2023-12-14T18:06:41.126 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-14T18:06:41.126 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} --verbose --name choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi --owner scheduled_choffman@teuthology --archive /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431 -- /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/orig.config.yaml 2023-12-14T18:06:41.181 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-14T18:06:41.266 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431 branch: wip-choffman-62793 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7491431' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62793-testing sha1: bf026da9d69000f3d4a9d16360156095d6612a2d owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4289 sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed sleep_before_teardown: 0 suite: fs suite_branch: wip-62793-testing suite_path: /home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa suite_relpath: qa suite_repo: https://github.com/chrisphoffman/ceph.git suite_sha1: bf026da9d69000f3d4a9d16360156095d6612a2d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBAqEFIrERJ0E4b5IMkedtnZUWsmlYIIy7lk3DB4N5nmW7/t2499qhZeWhvObY0t8ffVcjTUsLT6gglmQ8Mj6RU= smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEq2mpC03HKtWtJka6RjSZNr5uY9crXaq6+micl/UkHRcCYKzJAOoM8HKHc5xTs+7DOgHl4jUJunghqSEf3XmLE= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9DUm0HJt+g7/4B/7t7kUrNVx02zoDrxD3L4PZzs2A8y7LAoS09DOMhwuSXDTBK1APhaQkPKkTatUGeV1osDAg= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-14_17:59:30 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-14T18:06:41.266 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa; will attempt to use it 2023-12-14T18:06:41.267 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks 2023-12-14T18:06:41.267 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-14T18:06:41.269 INFO:teuthology.task.internal:Checking packages... 2023-12-14T18:06:41.292 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1faa353368ad4f5aeb56d7107b638336f58659ed' 2023-12-14T18:06:41.292 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-14T18:06:41.292 INFO:teuthology.packaging:ref: None 2023-12-14T18:06:41.292 INFO:teuthology.packaging:tag: None 2023-12-14T18:06:41.292 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-12-14T18:06:41.292 INFO:teuthology.packaging:sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:06:41.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-12-14T18:06:41.434 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3905.g1faa3533 2023-12-14T18:06:41.436 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-14T18:06:41.444 INFO:teuthology.task.internal:no buildpackages task found 2023-12-14T18:06:41.444 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-14T18:06:41.511 INFO:teuthology.task.internal:Saving configuration 2023-12-14T18:06:41.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-14T18:06:41.541 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:06:41.566 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:30.958178', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBAqEFIrERJ0E4b5IMkedtnZUWsmlYIIy7lk3DB4N5nmW7/t2499qhZeWhvObY0t8ffVcjTUsLT6gglmQ8Mj6RU='} 2023-12-14T18:06:41.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:30.960024', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEq2mpC03HKtWtJka6RjSZNr5uY9crXaq6+micl/UkHRcCYKzJAOoM8HKHc5xTs+7DOgHl4jUJunghqSEf3XmLE='} 2023-12-14T18:06:41.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:30.961444', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9DUm0HJt+g7/4B/7t7kUrNVx02zoDrxD3L4PZzs2A8y7LAoS09DOMhwuSXDTBK1APhaQkPKkTatUGeV1osDAg='} 2023-12-14T18:06:41.606 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-14T18:06:41.617 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-14T18:06:41.617 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-14T18:06:41.618 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-14T18:06:41.618 INFO:teuthology.run_tasks:Running task console_log... 2023-12-14T18:06:41.795 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe785de4dc0>, signals=[15]) 2023-12-14T18:06:41.795 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-14T18:06:41.805 INFO:teuthology.task.internal:Opening connections... 2023-12-14T18:06:41.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:06:41.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:06:41.876 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2023-12-14T18:06:41.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:06:41.945 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2023-12-14T18:06:41.946 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:06:42.012 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-14T18:06:42.022 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-12-14T18:06:42.044 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-12-14T18:06:42.044 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-12-14T18:06:42.105 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2023-12-14T18:06:42.105 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.6 (Ootpa)" 2023-12-14T18:06:42.105 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.6" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-14T18:06:42.106 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-14T18:06:42.107 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-14T18:06:42.107 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-12-14T18:06:42.135 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2023-12-14T18:06:42.155 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2023-12-14T18:06:42.155 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2023-12-14T18:06:42.214 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="Red Hat Enterprise Linux" 2023-12-14T18:06:42.214 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="8.6 (Ootpa)" 2023-12-14T18:06:42.214 INFO:teuthology.orchestra.run.smithi176.stdout:ID="rhel" 2023-12-14T18:06:42.214 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="fedora" 2023-12-14T18:06:42.214 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="8.6" 2023-12-14T18:06:42.214 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el8" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://www.redhat.com/" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-14T18:06:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-14T18:06:42.215 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2023-12-14T18:06:42.234 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2023-12-14T18:06:42.254 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2023-12-14T18:06:42.255 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2023-12-14T18:06:42.313 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Red Hat Enterprise Linux" 2023-12-14T18:06:42.313 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="8.6 (Ootpa)" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:ID="rhel" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="fedora" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="8.6" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el8" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.redhat.com/" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-14T18:06:42.314 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-14T18:06:42.315 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2023-12-14T18:06:42.334 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-14T18:06:42.347 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-14T18:06:42.442 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-14T18:06:42.442 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2023-12-14T18:06:42.445 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2023-12-14T18:06:42.447 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2023-12-14T18:06:42.466 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-14T18:06:42.476 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-14T18:06:42.476 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2023-12-14T18:06:42.505 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2023-12-14T18:06:42.507 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2023-12-14T18:06:42.542 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-14T18:06:42.678 INFO:teuthology.run_tasks:Running task kernel... 2023-12-14T18:06:42.691 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-12-14T18:06:42.691 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-12-14T18:06:42.691 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-12-14T18:06:42.692 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-14T18:06:42.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:42.712 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-12-14T18:06:42.773 INFO:teuthology.orchestra.run.smithi031.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:42.774 INFO:teuthology.task.kernel:Running kernel on smithi031: 4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:42.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2023-12-14T18:06:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:06:44.544 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-12-14T18:06:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:44.677 INFO:teuthology.orchestra.run.smithi031.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-14T18:06:44.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:46.581 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 10 kB/s | 3.5 kB 00:00 2023-12-14T18:06:48.392 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 16 MB 00:01 2023-12-14T18:06:52.298 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 419 kB/s | 24 kB 00:00 2023-12-14T18:06:53.117 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2023-12-14T18:06:53.118 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-12-14T18:06:53.214 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-12-14T18:06:53.216 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2023-12-14T18:06:53.216 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-12-14T18:06:53.347 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2023-12-14T18:06:53.776 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:06:53.777 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-12-14T18:06:53.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:53.863 INFO:teuthology.orchestra.run.smithi031.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-14T18:06:53.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:54.217 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:02 ago on Thu 14 Dec 2023 06:06:52 PM UTC. 2023-12-14T18:06:54.313 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-12-14T18:06:54.314 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-14T18:06:54.355 INFO:teuthology.orchestra.run.smithi031.stderr:Error: No packages marked for reinstall. 2023-12-14T18:06:54.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2023-12-14T18:06:54.793 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:06:54.794 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-12-14T18:06:54.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:54.862 INFO:teuthology.orchestra.run.smithi031.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-14T18:06:54.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:06:55.212 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Thu 14 Dec 2023 06:06:52 PM UTC. 2023-12-14T18:06:55.280 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-12-14T18:06:55.280 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-14T18:06:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:Error: No packages marked for reinstall. 2023-12-14T18:06:55.362 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2023-12-14T18:06:55.547 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:55.548 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:55.548 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-12-14T18:06:55.548 DEBUG:teuthology.parallel:result is None 2023-12-14T18:06:55.548 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-14T18:06:55.558 INFO:teuthology.task.internal:Creating test directory... 2023-12-14T18:06:55.558 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-14T18:06:55.560 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-14T18:06:55.562 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-14T18:06:55.583 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-14T18:06:55.589 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-14T18:06:55.598 INFO:teuthology.task.internal:Creating archive directory... 2023-12-14T18:06:55.598 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-14T18:06:55.619 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-14T18:06:55.622 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-14T18:06:55.666 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-14T18:06:55.678 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-14T18:06:55.678 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-14T18:06:55.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:55.710 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-14T18:06:55.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:55.728 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-14T18:06:55.731 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-14T18:06:55.753 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-14T18:06:55.775 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:55.796 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:55.798 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:55.811 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:55.815 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:55.827 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:55.829 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-14T18:06:55.836 INFO:teuthology.task.internal:Configuring sudo... 2023-12-14T18:06:55.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-14T18:06:55.838 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-14T18:06:55.840 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-14T18:06:55.872 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-14T18:06:55.880 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-14T18:06:55.880 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-14T18:06:55.915 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-14T18:06:55.917 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-14T18:06:55.935 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:56.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:56.081 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:56.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:56.218 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:06:56.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-14T18:06:56.292 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:56.338 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:56.392 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:56.477 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:56.521 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:06:56.521 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-14T18:06:56.589 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:56.635 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:56.692 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:56.775 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:56.821 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:06:56.821 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-14T18:06:56.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2023-12-14T18:06:56.892 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2023-12-14T18:06:56.894 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2023-12-14T18:06:56.949 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:06:56.960 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:06:56.993 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:06:57.351 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-14T18:06:57.361 INFO:teuthology.task.internal:Starting timer... 2023-12-14T18:06:57.362 INFO:teuthology.run_tasks:Running task pcp... 2023-12-14T18:06:57.405 INFO:teuthology.run_tasks:Running task selinux... 2023-12-14T18:06:57.415 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-12-14T18:06:57.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2023-12-14T18:06:57.485 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2023-12-14T18:06:57.487 DEBUG:teuthology.orchestra.run.smithi176:> sudo service auditd rotate 2023-12-14T18:06:57.549 INFO:teuthology.orchestra.run.smithi176.stdout:Rotating logs: 2023-12-14T18:06:57.550 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2023-12-14T18:06:57.643 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2023-12-14T18:06:57.645 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-14T18:06:57.645 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2023-12-14T18:06:57.669 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2023-12-14T18:06:57.670 DEBUG:teuthology.orchestra.run.smithi176:> /usr/sbin/getenforce 2023-12-14T18:06:57.695 INFO:teuthology.orchestra.run.smithi176.stdout:Permissive 2023-12-14T18:06:57.696 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2023-12-14T18:06:57.728 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2023-12-14T18:06:57.728 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi176.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2023-12-14T18:06:57.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:06:57.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:57.759 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:06:57.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:57.787 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:06:57.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:57.815 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-14T18:06:57.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2023-12-14T18:06:57.847 DEBUG:teuthology.orchestra.run.smithi176:> sudo /usr/sbin/setenforce permissive 2023-12-14T18:06:57.882 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2023-12-14T18:06:57.908 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-14T18:06:57.919 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-14T18:06:57.919 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-14T18:06:57.933 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-14T18:06:57.939 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi176.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-14T18:14:37.554 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi176.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2023-12-14T18:14:37.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-12-14T18:14:37.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:14:37.629 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-12-14T18:14:37.703 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-12-14T18:14:37.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2023-12-14T18:14:37.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:14:37.771 DEBUG:teuthology.orchestra.run.smithi176:> true 2023-12-14T18:14:37.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2023-12-14T18:14:37.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2023-12-14T18:14:37.841 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:14:37.916 DEBUG:teuthology.orchestra.run.smithi204:> true 2023-12-14T18:14:37.985 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2023-12-14T18:14:37.985 INFO:teuthology.run_tasks:Running task clock... 2023-12-14T18:14:37.994 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-14T18:14:37.994 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-14T18:14:37.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:14:37.997 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-14T18:14:37.997 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:14:38.000 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-14T18:14:38.000 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:14:38.034 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-14T18:14:38.034 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-14T18:14:38.050 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-14T18:14:38.052 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-14T18:14:38.060 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-14T18:14:38.075 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-14T18:14:38.121 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: ntpd: command not found 2023-12-14T18:14:38.134 INFO:teuthology.orchestra.run.smithi176.stdout:506 Cannot talk to daemon 2023-12-14T18:14:38.141 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2023-12-14T18:14:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2023-12-14T18:14:38.149 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-14T18:14:38.154 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2023-12-14T18:14:38.158 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2023-12-14T18:14:38.164 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-14T18:14:38.169 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-14T18:14:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-14T18:14:38.184 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-14T18:14:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-14T18:14:38.241 INFO:teuthology.orchestra.run.smithi176.stderr:bash: ntpq: command not found 2023-12-14T18:14:38.245 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:14:38.245 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2023-12-14T18:14:38.245 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.245 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.245 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.245 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.272 INFO:teuthology.orchestra.run.smithi204.stderr:bash: ntpq: command not found 2023-12-14T18:14:38.275 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:14:38.275 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2023-12-14T18:14:38.275 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.275 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.275 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.275 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.283 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-12-14T18:14:38.288 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:14:38.288 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-12-14T18:14:38.288 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.288 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.288 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.288 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:38.289 INFO:teuthology.run_tasks:Running task pexec... 2023-12-14T18:14:38.298 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-14T18:14:38.299 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:14:38.299 DEBUG:teuthology.orchestra.run.smithi031:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-14T18:14:38.299 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi176.front.sepia.ceph.com 2023-12-14T18:14:38.299 DEBUG:teuthology.orchestra.run.smithi176:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-14T18:14:38.300 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi204.front.sepia.ceph.com 2023-12-14T18:14:38.300 DEBUG:teuthology.orchestra.run.smithi204:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-14T18:14:38.675 INFO:teuthology.orchestra.run.smithi176.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:38.691 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:40.606 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2023-12-14T18:14:40.679 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 20 kB/s | 4.1 kB 00:00 2023-12-14T18:14:40.791 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2023-12-14T18:14:40.883 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 66 kB/s | 4.5 kB 00:00 2023-12-14T18:14:41.015 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2023-12-14T18:14:41.492 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2023-12-14T18:14:43.823 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2023-12-14T18:14:43.823 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout: Package Architecture Version Repository Size 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout:Resetting modules: 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout: container-tools 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2023-12-14T18:14:43.824 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:14:44.101 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2023-12-14T18:14:44.102 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2023-12-14T18:14:44.102 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2023-12-14T18:14:44.102 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2023-12-14T18:14:44.102 INFO:teuthology.orchestra.run.smithi176.stdout:Resetting modules: 2023-12-14T18:14:44.102 INFO:teuthology.orchestra.run.smithi176.stdout: container-tools 2023-12-14T18:14:44.102 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:14:44.103 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2023-12-14T18:14:44.103 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2023-12-14T18:14:44.103 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:14:44.715 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout:Resetting modules: 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: container-tools 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-12-14T18:14:44.717 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2023-12-14T18:14:44.718 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:14:56.814 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2023-12-14T18:14:57.295 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:57.758 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2023-12-14T18:14:58.245 INFO:teuthology.orchestra.run.smithi176.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:59.342 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2023-12-14T18:14:59.716 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2023-12-14T18:15:00.070 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-12-14T18:15:00.124 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2023-12-14T18:15:00.445 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 kB/s | 4.5 kB 00:00 2023-12-14T18:15:00.594 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:02.418 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 23 kB/s | 4.5 kB 00:00 2023-12-14T18:15:02.513 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================================== 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================================== 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout:Installing group/module packages: 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-14T18:15:02.667 INFO:teuthology.orchestra.run.smithi204.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:Installing module profiles: 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: container-tools/common 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:Enabling module streams: 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: container-tools rhel8 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:========================================================================================================== 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout:Install 13 Packages 2023-12-14T18:15:02.668 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:15:02.669 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 26 M 2023-12-14T18:15:02.669 INFO:teuthology.orchestra.run.smithi204.stdout:Installed size: 87 M 2023-12-14T18:15:02.669 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2023-12-14T18:15:02.872 INFO:teuthology.orchestra.run.smithi204.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 464 kB/s | 94 kB 00:00 2023-12-14T18:15:02.897 INFO:teuthology.orchestra.run.smithi204.stdout:(2/13): yajl-2.1.0-12.el8.x86_64.rpm 180 kB/s | 41 kB 00:00 2023-12-14T18:15:02.998 INFO:teuthology.orchestra.run.smithi204.stdout:(3/13): python3-pytoml-0.1.14-5.git7dea353.el8. 76 kB/s | 25 kB 00:00 2023-12-14T18:15:03.086 INFO:teuthology.orchestra.run.smithi204.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 4.7 MB/s | 1.0 MB 00:00 2023-12-14T18:15:03.156 INFO:teuthology.orchestra.run.smithi204.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 4.6 MB/s | 739 kB 00:00 2023-12-14T18:15:03.207 INFO:teuthology.orchestra.run.smithi204.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 327 kB/s | 39 kB 00:00 2023-12-14T18:15:03.207 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 18 kB/s | 4.1 kB 00:00 2023-12-14T18:15:03.265 INFO:teuthology.orchestra.run.smithi204.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 2.2 MB/s | 239 kB 00:00 2023-12-14T18:15:03.391 INFO:teuthology.orchestra.run.smithi204.stdout:(8/13): buildah-1.31.3-1.module+el8.9.0+20326+3 18 MB/s | 8.8 MB 00:00 2023-12-14T18:15:03.420 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2023-12-14T18:15:03.456 INFO:teuthology.orchestra.run.smithi204.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 808 kB/s | 153 kB 00:00 2023-12-14T18:15:03.541 INFO:teuthology.orchestra.run.smithi204.stdout:(10/13): netavark-1.7.0-1.module+el8.9.0+20326+ 11 MB/s | 3.7 MB 00:00 2023-12-14T18:15:03.568 INFO:teuthology.orchestra.run.smithi176.stdout:========================================================================================================== 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout:========================================================================================================== 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout:Installing group/module packages: 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-14T18:15:03.569 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout:Installing weak dependencies: 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout:Installing module profiles: 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: container-tools/common 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout:Enabling module streams: 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: container-tools rhel8 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout:========================================================================================================== 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout:Install 13 Packages 2023-12-14T18:15:03.570 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:15:03.571 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 26 M 2023-12-14T18:15:03.571 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 87 M 2023-12-14T18:15:03.571 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2023-12-14T18:15:03.725 INFO:teuthology.orchestra.run.smithi204.stdout:(11/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 24 MB/s | 8.1 MB 00:00 2023-12-14T18:15:03.739 INFO:teuthology.orchestra.run.smithi176.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 148 kB/s | 25 kB 00:00 2023-12-14T18:15:03.756 INFO:teuthology.orchestra.run.smithi176.stdout:(2/13): yajl-2.1.0-12.el8.x86_64.rpm 223 kB/s | 41 kB 00:00 2023-12-14T18:15:03.789 INFO:teuthology.orchestra.run.smithi176.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 432 kB/s | 94 kB 00:00 2023-12-14T18:15:03.792 INFO:teuthology.orchestra.run.smithi204.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 7.5 MB/s | 2.5 MB 00:00 2023-12-14T18:15:03.817 INFO:teuthology.orchestra.run.smithi204.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+20326+38 179 kB/s | 49 kB 00:00 2023-12-14T18:15:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:15:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:Total 22 MB/s | 26 MB 00:01 2023-12-14T18:15:03.956 INFO:teuthology.orchestra.run.smithi176.stdout:(4/13): cockpit-podman-75-1.module+el8.9.0+2032 4.3 MB/s | 739 kB 00:00 2023-12-14T18:15:03.998 INFO:teuthology.orchestra.run.smithi176.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 3.9 MB/s | 1.0 MB 00:00 2023-12-14T18:15:04.102 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2023-12-14T18:15:04.123 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2023-12-14T18:15:04.123 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2023-12-14T18:15:04.123 INFO:teuthology.orchestra.run.smithi176.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 236 kB/s | 39 kB 00:00 2023-12-14T18:15:04.148 INFO:teuthology.orchestra.run.smithi176.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 1.6 MB/s | 239 kB 00:00 2023-12-14T18:15:04.275 INFO:teuthology.orchestra.run.smithi176.stdout:(8/13): buildah-1.31.3-1.module+el8.9.0+20326+3 17 MB/s | 8.8 MB 00:00 2023-12-14T18:15:04.324 INFO:teuthology.orchestra.run.smithi176.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 874 kB/s | 153 kB 00:00 2023-12-14T18:15:04.408 INFO:teuthology.orchestra.run.smithi176.stdout:(10/13): netavark-1.7.0-1.module+el8.9.0+20326+ 13 MB/s | 3.7 MB 00:00 2023-12-14T18:15:04.492 INFO:teuthology.orchestra.run.smithi176.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 15 MB/s | 2.5 MB 00:00 2023-12-14T18:15:04.625 INFO:teuthology.orchestra.run.smithi176.stdout:(12/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 23 MB/s | 8.1 MB 00:00 2023-12-14T18:15:04.650 INFO:teuthology.orchestra.run.smithi176.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+20326+38 204 kB/s | 49 kB 00:00 2023-12-14T18:15:04.651 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:15:04.651 INFO:teuthology.orchestra.run.smithi176.stdout:Total 24 MB/s | 26 MB 00:01 2023-12-14T18:15:04.791 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2023-12-14T18:15:04.800 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2023-12-14T18:15:04.936 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2023-12-14T18:15:04.957 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2023-12-14T18:15:04.957 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2023-12-14T18:15:05.487 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2023-12-14T18:15:05.591 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2023-12-14T18:15:05.601 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2023-12-14T18:15:05.726 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-14T18:15:06.228 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2023-12-14T18:15:06.263 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-14T18:15:06.388 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-12-14T18:15:06.471 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-14T18:15:06.497 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:========================================================================================================== 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:========================================================================================================== 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:Installing group/module packages: 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-14T18:15:06.563 INFO:teuthology.orchestra.run.smithi031.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout:Installing module profiles: 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: container-tools/common 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling module streams: 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: container-tools rhel8 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-12-14T18:15:06.564 INFO:teuthology.orchestra.run.smithi031.stdout:========================================================================================================== 2023-12-14T18:15:06.565 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2023-12-14T18:15:06.565 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:15:06.565 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 26 M 2023-12-14T18:15:06.565 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 87 M 2023-12-14T18:15:06.565 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-12-14T18:15:06.677 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-14T18:15:06.724 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 158 kB/s | 25 kB 00:00 2023-12-14T18:15:06.749 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): yajl-2.1.0-12.el8.x86_64.rpm 225 kB/s | 41 kB 00:00 2023-12-14T18:15:06.774 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 452 kB/s | 94 kB 00:00 2023-12-14T18:15:06.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-14T18:15:06.908 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 5.5 MB/s | 1.0 MB 00:00 2023-12-14T18:15:07.008 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 3.1 MB/s | 739 kB 00:00 2023-12-14T18:15:07.040 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-14T18:15:07.050 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 277 kB/s | 39 kB 00:00 2023-12-14T18:15:07.100 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 2.5 MB/s | 239 kB 00:00 2023-12-14T18:15:07.165 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-14T18:15:07.209 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): buildah-1.31.3-1.module+el8.9.0+20326+3 19 MB/s | 8.8 MB 00:00 2023-12-14T18:15:07.234 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 1.1 MB/s | 153 kB 00:00 2023-12-14T18:15:07.248 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-14T18:15:07.351 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): netavark-1.7.0-1.module+el8.9.0+20326+ 12 MB/s | 3.7 MB 00:00 2023-12-14T18:15:07.425 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-14T18:15:07.443 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 12 MB/s | 2.5 MB 00:00 2023-12-14T18:15:07.474 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-14T18:15:07.502 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 329 kB/s | 49 kB 00:00 2023-12-14T18:15:07.636 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-14T18:15:07.644 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 19 MB/s | 8.1 MB 00:00 2023-12-14T18:15:07.644 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:15:07.644 INFO:teuthology.orchestra.run.smithi031.stdout:Total 24 MB/s | 26 MB 00:01 2023-12-14T18:15:07.870 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-14T18:15:07.950 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-12-14T18:15:07.975 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-12-14T18:15:07.975 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-12-14T18:15:08.134 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-14T18:15:08.369 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:15:08.618 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-12-14T18:15:08.628 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-12-14T18:15:08.661 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:15:09.071 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-14T18:15:09.081 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:15:09.330 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-12-14T18:15:09.361 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:15:09.625 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-14T18:15:09.780 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-14T18:15:09.993 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-14T18:15:10.221 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-14T18:15:10.256 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-14T18:15:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-14T18:15:10.455 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:15:10.657 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-14T18:15:10.671 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-14T18:15:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-14T18:15:10.906 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:15:10.906 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-14T18:15:10.906 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-14T18:15:10.906 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-14T18:15:10.906 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-14T18:15:10.906 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-14T18:15:10.907 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-14T18:15:10.914 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-14T18:15:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-14T18:15:11.101 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:15:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-14T18:15:11.519 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:15:11.520 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-14T18:15:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-14T18:15:11.521 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-14T18:15:12.376 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:15:12.651 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:15:13.138 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-14T18:15:14.100 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-14T18:15:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-14T18:15:14.554 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:15:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-14T18:15:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-14T18:15:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-14T18:15:22.602 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-14T18:15:22.602 INFO:teuthology.orchestra.run.smithi204.stdout:Installed products updated. 2023-12-14T18:15:23.131 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:15:23.131 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2023-12-14T18:15:23.131 INFO:teuthology.orchestra.run.smithi204.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.131 INFO:teuthology.orchestra.run.smithi204.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-14T18:15:23.132 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:15:23.133 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2023-12-14T18:15:23.309 DEBUG:teuthology.parallel:result is None 2023-12-14T18:15:25.687 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-14T18:15:25.687 INFO:teuthology.orchestra.run.smithi176.stdout:Installed products updated. 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-14T18:15:26.241 INFO:teuthology.orchestra.run.smithi176.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.242 INFO:teuthology.orchestra.run.smithi176.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:26.242 INFO:teuthology.orchestra.run.smithi176.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:26.242 INFO:teuthology.orchestra.run.smithi176.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-14T18:15:26.242 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:15:26.242 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2023-12-14T18:15:26.408 DEBUG:teuthology.parallel:result is None 2023-12-14T18:15:30.266 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-14T18:15:30.266 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2023-12-14T18:15:30.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:15:30.863 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:15:30.864 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-12-14T18:15:31.084 DEBUG:teuthology.parallel:result is None 2023-12-14T18:15:31.084 INFO:teuthology.run_tasks:Running task install... 2023-12-14T18:15:31.092 DEBUG:teuthology.task.install:project ceph 2023-12-14T18:15:31.093 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed'}} 2023-12-14T18:15:31.093 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed'} 2023-12-14T18:15:31.093 INFO:teuthology.task.install:Using flavor: default 2023-12-14T18:15:31.098 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-14T18:15:31.099 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-14T18:15:31.099 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'tag': None, 'wait_for_package': False} 2023-12-14T18:15:31.099 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:15:31.101 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'tag': None, 'wait_for_package': False} 2023-12-14T18:15:31.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:15:31.102 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'tag': None, 'wait_for_package': False} 2023-12-14T18:15:31.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:15:31.334 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/ 2023-12-14T18:15:31.334 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g1faa3533 2023-12-14T18:15:31.337 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/ 2023-12-14T18:15:31.338 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g1faa3533 2023-12-14T18:15:31.346 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/ 2023-12-14T18:15:31.346 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g1faa3533 2023-12-14T18:15:31.467 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-14T18:15:31.467 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:15:31.467 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-14T18:15:31.475 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-14T18:15:31.475 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:15:31.475 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-14T18:15:31.500 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-14T18:15:31.500 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1faa353368ad4f5aeb56d7107b638336f58659ed/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-14T18:15:31.508 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-14T18:15:31.508 DEBUG:teuthology.orchestra.run.smithi176:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1faa353368ad4f5aeb56d7107b638336f58659ed/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-14T18:15:31.530 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-14T18:15:31.530 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:15:31.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-14T18:15:31.566 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-14T18:15:31.566 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1faa353368ad4f5aeb56d7107b638336f58659ed/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-14T18:15:31.572 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-14T18:15:31.580 DEBUG:teuthology.orchestra.run.smithi176:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-14T18:15:31.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-14T18:15:31.652 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-14T18:15:31.659 DEBUG:teuthology.orchestra.run.smithi176:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-14T18:15:31.720 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2023-12-14T18:15:31.721 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2023-12-14T18:15:31.726 INFO:teuthology.orchestra.run.smithi176.stdout:check_obsoletes = 1 2023-12-14T18:15:31.728 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum clean all 2023-12-14T18:15:31.730 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-14T18:15:31.762 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2023-12-14T18:15:31.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2023-12-14T18:15:32.100 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:32.105 INFO:teuthology.orchestra.run.smithi176.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:32.198 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:33.512 INFO:teuthology.orchestra.run.smithi031.stdout:52 files removed 2023-12-14T18:15:33.531 INFO:teuthology.orchestra.run.smithi176.stdout:52 files removed 2023-12-14T18:15:33.543 INFO:teuthology.orchestra.run.smithi204.stdout:52 files removed 2023-12-14T18:15:33.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-14T18:15:33.583 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-14T18:15:33.596 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-14T18:15:33.920 INFO:teuthology.orchestra.run.smithi176.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:33.933 INFO:teuthology.orchestra.run.smithi204.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:33.969 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-12-14T18:15:36.112 INFO:teuthology.orchestra.run.smithi176.stdout:ceph packages for x86_64 108 kB/s | 80 kB 00:00 2023-12-14T18:15:36.113 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 105 kB/s | 80 kB 00:00 2023-12-14T18:15:36.231 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 123 kB/s | 80 kB 00:00 2023-12-14T18:15:36.538 INFO:teuthology.orchestra.run.smithi176.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-12-14T18:15:36.614 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-12-14T18:15:36.682 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 23 kB/s | 9.4 kB 00:00 2023-12-14T18:15:37.182 INFO:teuthology.orchestra.run.smithi176.stdout:ceph source packages 2.4 kB/s | 1.5 kB 00:00 2023-12-14T18:15:37.190 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-12-14T18:15:37.308 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 2.5 kB/s | 1.5 kB 00:00 2023-12-14T18:15:37.348 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-12-14T18:15:37.450 INFO:teuthology.orchestra.run.smithi204.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-12-14T18:15:37.523 INFO:teuthology.orchestra.run.smithi176.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-12-14T18:15:38.086 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2023-12-14T18:15:38.183 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 20 MB/s | 16 MB 00:00 2023-12-14T18:15:39.690 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 16 MB 00:02 2023-12-14T18:15:41.660 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 398 kB/s | 24 kB 00:00 2023-12-14T18:15:42.166 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 405 kB/s | 24 kB 00:00 2023-12-14T18:15:42.329 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2023-12-14T18:15:43.257 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 399 kB/s | 24 kB 00:00 2023-12-14T18:15:44.152 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 MB/s | 58 MB 00:01 2023-12-14T18:15:44.609 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 64 MB 00:01 2023-12-14T18:15:45.222 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 64 MB 00:01 2023-12-14T18:15:55.218 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 58 MB 00:01 2023-12-14T18:15:55.880 INFO:teuthology.orchestra.run.smithi204.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 58 MB 00:01 2023-12-14T18:15:56.286 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 64 MB 00:01 2023-12-14T18:16:05.701 INFO:teuthology.orchestra.run.smithi176.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2023-12-14T18:16:07.502 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2023-12-14T18:16:11.306 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:16:11.307 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:16:11.325 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-14T18:16:11.327 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-14T18:16:11.328 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-14T18:16:11.330 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-14T18:16:11.332 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-14T18:16:11.337 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-14T18:16:11.338 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-14T18:16:11.543 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout:=========================================================================================================================== 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout:=========================================================================================================================== 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-14T18:16:11.558 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.4 k 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 5.3 M 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 847 k 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 165 k 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 1.5 M 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 127 k 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 1.7 M 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 7.4 M 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 47 k 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 13 M 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 41 M 2023-12-14T18:16:11.559 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 265 k 2023-12-14T18:16:11.560 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 83 k 2023-12-14T18:16:11.560 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 258 k 2023-12-14T18:16:11.560 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 18 k 2023-12-14T18:16:11.560 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-14T18:16:11.560 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 31 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 771 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 139 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-14T18:16:11.561 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 197 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 370 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 364 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 107 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 94 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.4 M 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 179 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-14T18:16:11.562 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.7 M 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.6 M 2023-12-14T18:16:11.563 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-14T18:16:11.564 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 22 M 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 23 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 2.3 M 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 240 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 4.6 M 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 18 M 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 15 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 24 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-14T18:16:11.565 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 137 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 473 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.5 M 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-14T18:16:11.566 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 45 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 119 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-14T18:16:11.567 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-14T18:16:11.568 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:16:11.569 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-14T18:16:11.570 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:Enabling module streams: 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql 10 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:=========================================================================================================================== 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:Install 129 Packages 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 16 Packages 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 205 M 2023-12-14T18:16:11.571 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2023-12-14T18:16:11.848 INFO:teuthology.orchestra.run.smithi204.stdout:(1/145): ceph-18.0.0-3905.g1faa3533.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2023-12-14T18:16:12.115 INFO:teuthology.orchestra.run.smithi204.stdout:(2/145): ceph-fuse-18.0.0-3905.g1faa3533.el8.x8 3.1 MB/s | 847 kB 00:00 2023-12-14T18:16:12.181 INFO:teuthology.orchestra.run.smithi204.stdout:(3/145): ceph-immutable-object-cache-18.0.0-390 2.4 MB/s | 165 kB 00:00 2023-12-14T18:16:12.557 INFO:teuthology.orchestra.run.smithi204.stdout:(4/145): ceph-mds-18.0.0-3905.g1faa3533.el8.x86 6.1 MB/s | 2.3 MB 00:00 2023-12-14T18:16:12.641 INFO:teuthology.orchestra.run.smithi176.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:16:12.642 INFO:teuthology.orchestra.run.smithi176.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:16:12.660 INFO:teuthology.orchestra.run.smithi176.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-14T18:16:12.662 INFO:teuthology.orchestra.run.smithi176.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-14T18:16:12.663 INFO:teuthology.orchestra.run.smithi176.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-14T18:16:12.664 INFO:teuthology.orchestra.run.smithi176.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-14T18:16:12.667 INFO:teuthology.orchestra.run.smithi176.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-14T18:16:12.671 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-14T18:16:12.672 INFO:teuthology.orchestra.run.smithi176.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-14T18:16:12.690 INFO:teuthology.orchestra.run.smithi204.stdout:(5/145): ceph-base-18.0.0-3905.g1faa3533.el8.x8 5.5 MB/s | 5.3 MB 00:00 2023-12-14T18:16:12.824 INFO:teuthology.orchestra.run.smithi204.stdout:(6/145): ceph-mgr-18.0.0-3905.g1faa3533.el8.x86 5.8 MB/s | 1.5 MB 00:00 2023-12-14T18:16:12.874 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout:=========================================================================================================================== 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout:=========================================================================================================================== 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-14T18:16:12.888 INFO:teuthology.orchestra.run.smithi176.stdout: ceph x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.4 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 5.3 M 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 847 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 165 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 1.5 M 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 127 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 1.7 M 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 7.4 M 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 47 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 13 M 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 41 M 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 265 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 83 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 258 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 18 k 2023-12-14T18:16:12.889 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 31 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 771 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 139 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 197 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 370 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 364 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 107 k 2023-12-14T18:16:12.890 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 94 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.4 M 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 179 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: librados2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.7 M 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.6 M 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-14T18:16:12.891 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 22 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 23 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 2.3 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 240 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 4.6 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 18 M 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 15 k 2023-12-14T18:16:12.892 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 24 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 137 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-14T18:16:12.893 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 473 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.5 M 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-14T18:16:12.894 INFO:teuthology.orchestra.run.smithi176.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 45 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 119 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-14T18:16:12.895 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-14T18:16:12.896 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-14T18:16:12.897 INFO:teuthology.orchestra.run.smithi176.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout:Enabling module streams: 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql 10 2023-12-14T18:16:12.898 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout:=========================================================================================================================== 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout:Install 129 Packages 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 16 Packages 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 205 M 2023-12-14T18:16:12.899 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2023-12-14T18:16:13.283 INFO:teuthology.orchestra.run.smithi204.stdout:(7/145): ceph-mon-18.0.0-3905.g1faa3533.el8.x86 7.8 MB/s | 4.6 MB 00:00 2023-12-14T18:16:13.318 INFO:teuthology.orchestra.run.smithi176.stdout:(1/145): ceph-18.0.0-3905.g1faa3533.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-12-14T18:16:14.410 INFO:teuthology.orchestra.run.smithi176.stdout:(2/145): ceph-fuse-18.0.0-3905.g1faa3533.el8.x8 776 kB/s | 847 kB 00:01 2023-12-14T18:16:14.777 INFO:teuthology.orchestra.run.smithi176.stdout:(3/145): ceph-immutable-object-cache-18.0.0-390 451 kB/s | 165 kB 00:00 2023-12-14T18:16:15.400 INFO:teuthology.orchestra.run.smithi204.stdout:(8/145): ceph-radosgw-18.0.0-3905.g1faa3533.el8 6.2 MB/s | 13 MB 00:02 2023-12-14T18:16:15.463 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:16:15.464 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:16:15.483 INFO:teuthology.orchestra.run.smithi204.stdout:(9/145): ceph-selinux-18.0.0-3905.g1faa3533.el8 292 kB/s | 24 kB 00:00 2023-12-14T18:16:15.484 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-14T18:16:15.486 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-14T18:16:15.487 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-14T18:16:15.489 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-14T18:16:15.492 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-14T18:16:15.496 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-14T18:16:15.497 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-14T18:16:15.667 INFO:teuthology.orchestra.run.smithi204.stdout:(10/145): ceph-osd-18.0.0-3905.g1faa3533.el8.x8 6.2 MB/s | 18 MB 00:02 2023-12-14T18:16:15.733 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-12-14T18:16:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-12-14T18:16:15.749 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-12-14T18:16:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-12-14T18:16:15.749 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.4 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 5.3 M 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 847 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 165 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 1.5 M 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 127 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 1.7 M 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 7.4 M 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 47 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 13 M 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 41 M 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 265 k 2023-12-14T18:16:15.750 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 83 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 258 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 18 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 31 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 771 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 139 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-14T18:16:15.751 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 197 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 370 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 364 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 107 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 94 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.4 M 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 179 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-14T18:16:15.752 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.7 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.6 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 22 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 23 k 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 2.3 M 2023-12-14T18:16:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 240 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 4.6 M 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 18 M 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 15 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 24 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 137 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-14T18:16:15.754 INFO:teuthology.orchestra.run.smithi031.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 473 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.5 M 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-14T18:16:15.755 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 45 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 119 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-14T18:16:15.756 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-14T18:16:15.757 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-14T18:16:15.758 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-14T18:16:15.759 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling module streams: 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql 10 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:Install 129 Packages 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 16 Packages 2023-12-14T18:16:15.760 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:16:15.761 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 205 M 2023-12-14T18:16:15.761 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-12-14T18:16:15.767 INFO:teuthology.orchestra.run.smithi204.stdout:(11/145): cephfs-mirror-18.0.0-3905.g1faa3533.e 2.5 MB/s | 258 kB 00:00 2023-12-14T18:16:15.859 INFO:teuthology.orchestra.run.smithi204.stdout:(12/145): libcephfs-devel-18.0.0-3905.g1faa3533 338 kB/s | 31 kB 00:00 2023-12-14T18:16:16.053 INFO:teuthology.orchestra.run.smithi204.stdout:(13/145): libcephfs2-18.0.0-3905.g1faa3533.el8. 3.9 MB/s | 771 kB 00:00 2023-12-14T18:16:16.151 INFO:teuthology.orchestra.run.smithi204.stdout:(14/145): libcephsqlite-18.0.0-3905.g1faa3533.e 1.4 MB/s | 137 kB 00:00 2023-12-14T18:16:16.235 INFO:teuthology.orchestra.run.smithi204.stdout:(15/145): librados-devel-18.0.0-3905.g1faa3533. 1.6 MB/s | 139 kB 00:00 2023-12-14T18:16:16.285 INFO:teuthology.orchestra.run.smithi176.stdout:(4/145): ceph-base-18.0.0-3905.g1faa3533.el8.x8 1.6 MB/s | 5.3 MB 00:03 2023-12-14T18:16:16.335 INFO:teuthology.orchestra.run.smithi204.stdout:(16/145): libradosstriper1-18.0.0-3905.g1faa353 4.6 MB/s | 473 kB 00:00 2023-12-14T18:16:16.486 INFO:teuthology.orchestra.run.smithi031.stdout:(1/145): ceph-18.0.0-3905.g1faa3533.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-12-14T18:16:16.836 INFO:teuthology.orchestra.run.smithi176.stdout:(5/145): ceph-mgr-18.0.0-3905.g1faa3533.el8.x86 2.8 MB/s | 1.5 MB 00:00 2023-12-14T18:16:16.969 INFO:teuthology.orchestra.run.smithi176.stdout:(6/145): ceph-mds-18.0.0-3905.g1faa3533.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-12-14T18:16:17.519 INFO:teuthology.orchestra.run.smithi204.stdout:(17/145): ceph-common-18.0.0-3905.g1faa3533.el8 3.7 MB/s | 22 MB 00:05 2023-12-14T18:16:17.545 INFO:teuthology.orchestra.run.smithi031.stdout:(2/145): ceph-fuse-18.0.0-3905.g1faa3533.el8.x8 801 kB/s | 847 kB 00:01 2023-12-14T18:16:17.611 INFO:teuthology.orchestra.run.smithi204.stdout:(18/145): librgw2-18.0.0-3905.g1faa3533.el8.x86 5.1 MB/s | 6.5 MB 00:01 2023-12-14T18:16:17.636 INFO:teuthology.orchestra.run.smithi204.stdout:(19/145): python3-ceph-argparse-18.0.0-3905.g1f 382 kB/s | 45 kB 00:00 2023-12-14T18:16:17.695 INFO:teuthology.orchestra.run.smithi204.stdout:(20/145): python3-ceph-common-18.0.0-3905.g1faa 1.4 MB/s | 119 kB 00:00 2023-12-14T18:16:17.705 INFO:teuthology.orchestra.run.smithi031.stdout:(3/145): ceph-immutable-object-cache-18.0.0-390 1.0 MB/s | 165 kB 00:00 2023-12-14T18:16:17.787 INFO:teuthology.orchestra.run.smithi204.stdout:(21/145): python3-rados-18.0.0-3905.g1faa3533.e 4.0 MB/s | 370 kB 00:00 2023-12-14T18:16:17.844 INFO:teuthology.orchestra.run.smithi204.stdout:(22/145): python3-cephfs-18.0.0-3905.g1faa3533. 950 kB/s | 197 kB 00:00 2023-12-14T18:16:17.887 INFO:teuthology.orchestra.run.smithi204.stdout:(23/145): python3-rbd-18.0.0-3905.g1faa3533.el8 3.5 MB/s | 364 kB 00:00 2023-12-14T18:16:17.963 INFO:teuthology.orchestra.run.smithi204.stdout:(24/145): python3-rgw-18.0.0-3905.g1faa3533.el8 903 kB/s | 107 kB 00:00 2023-12-14T18:16:17.988 INFO:teuthology.orchestra.run.smithi204.stdout:(25/145): rbd-fuse-18.0.0-3905.g1faa3533.el8.x8 943 kB/s | 94 kB 00:00 2023-12-14T18:16:18.071 INFO:teuthology.orchestra.run.smithi204.stdout:(26/145): rbd-nbd-18.0.0-3905.g1faa3533.el8.x86 2.1 MB/s | 179 kB 00:00 2023-12-14T18:16:18.155 INFO:teuthology.orchestra.run.smithi204.stdout:(27/145): ceph-grafana-dashboards-18.0.0-3905.g 276 kB/s | 23 kB 00:00 2023-12-14T18:16:18.203 INFO:teuthology.orchestra.run.smithi176.stdout:(7/145): ceph-mon-18.0.0-3905.g1faa3533.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-12-14T18:16:18.239 INFO:teuthology.orchestra.run.smithi204.stdout:(28/145): ceph-mgr-cephadm-18.0.0-3905.g1faa353 1.5 MB/s | 127 kB 00:00 2023-12-14T18:16:18.764 INFO:teuthology.orchestra.run.smithi204.stdout:(29/145): ceph-mgr-dashboard-18.0.0-3905.g1faa3 3.3 MB/s | 1.7 MB 00:00 2023-12-14T18:16:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:(30/145): rbd-mirror-18.0.0-3905.g1faa3533.el8. 2.5 MB/s | 3.4 MB 00:01 2023-12-14T18:16:19.554 INFO:teuthology.orchestra.run.smithi204.stdout:(31/145): ceph-mgr-modules-core-18.0.0-3905.g1f 1.1 MB/s | 240 kB 00:00 2023-12-14T18:16:19.628 INFO:teuthology.orchestra.run.smithi031.stdout:(4/145): ceph-mds-18.0.0-3905.g1faa3533.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-12-14T18:16:19.665 INFO:teuthology.orchestra.run.smithi204.stdout:(32/145): ceph-mgr-rook-18.0.0-3905.g1faa3533.e 428 kB/s | 47 kB 00:00 2023-12-14T18:16:19.765 INFO:teuthology.orchestra.run.smithi204.stdout:(33/145): ceph-prometheus-alerts-18.0.0-3905.g1 145 kB/s | 15 kB 00:00 2023-12-14T18:16:19.982 INFO:teuthology.orchestra.run.smithi204.stdout:(34/145): ceph-volume-18.0.0-3905.g1faa3533.el8 1.2 MB/s | 265 kB 00:00 2023-12-14T18:16:20.091 INFO:teuthology.orchestra.run.smithi204.stdout:(35/145): cephadm-18.0.0-3905.g1faa3533.el8.noa 770 kB/s | 83 kB 00:00 2023-12-14T18:16:20.228 INFO:teuthology.orchestra.run.smithi204.stdout:(36/145): cephfs-top-18.0.0-3905.g1faa3533.el8. 132 kB/s | 18 kB 00:00 2023-12-14T18:16:20.358 INFO:teuthology.orchestra.run.smithi204.stdout:(37/145): ceph-mgr-diskprediction-local-18.0.0- 4.7 MB/s | 7.4 MB 00:01 2023-12-14T18:16:20.992 INFO:teuthology.orchestra.run.smithi204.stdout:(38/145): liboath-2.6.2-3.el8.x86_64.rpm 93 kB/s | 59 kB 00:00 2023-12-14T18:16:21.053 INFO:teuthology.orchestra.run.smithi031.stdout:(5/145): ceph-mgr-18.0.0-3905.g1faa3533.el8.x86 1.1 MB/s | 1.5 MB 00:01 2023-12-14T18:16:21.192 INFO:teuthology.orchestra.run.smithi204.stdout:(39/145): gperftools-libs-2.7-9.el8.x86_64.rpm 318 kB/s | 306 kB 00:00 2023-12-14T18:16:21.263 INFO:teuthology.orchestra.run.smithi204.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 277 kB/s | 75 kB 00:00 2023-12-14T18:16:21.401 INFO:teuthology.orchestra.run.smithi204.stdout:(41/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 317 kB/s | 44 kB 00:00 2023-12-14T18:16:21.434 INFO:teuthology.orchestra.run.smithi204.stdout:(42/145): python3-asyncssh-2.7.0-2.el8.noarch.r 1.9 MB/s | 476 kB 00:00 2023-12-14T18:16:21.560 INFO:teuthology.orchestra.run.smithi204.stdout:(43/145): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-12-14T18:16:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:(8/145): ceph-radosgw-18.0.0-3905.g1faa3533.el8 3.8 MB/s | 13 MB 00:03 2023-12-14T18:16:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:(44/145): python3-beautifulsoup4-4.6.3-2.el8.1. 765 kB/s | 185 kB 00:00 2023-12-14T18:16:21.685 INFO:teuthology.orchestra.run.smithi204.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-12-14T18:16:21.728 INFO:teuthology.orchestra.run.smithi176.stdout:(9/145): ceph-selinux-18.0.0-3905.g1faa3533.el8 209 kB/s | 24 kB 00:00 2023-12-14T18:16:21.853 INFO:teuthology.orchestra.run.smithi204.stdout:(46/145): python3-cheroot-8.5.2-1.el8.noarch.rp 826 kB/s | 173 kB 00:00 2023-12-14T18:16:21.878 INFO:teuthology.orchestra.run.smithi204.stdout:(47/145): python3-cherrypy-18.4.0-1.el8.noarch. 2.0 MB/s | 384 kB 00:00 2023-12-14T18:16:22.003 INFO:teuthology.orchestra.run.smithi204.stdout:(48/145): python3-google-auth-1.1.1-10.el8.noar 541 kB/s | 81 kB 00:00 2023-12-14T18:16:22.028 INFO:teuthology.orchestra.run.smithi204.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 74 kB/s | 11 kB 00:00 2023-12-14T18:16:22.112 INFO:teuthology.orchestra.run.smithi204.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-12-14T18:16:22.296 INFO:teuthology.orchestra.run.smithi204.stdout:(51/145): python3-logutils-0.3.5-11.el8.noarch. 265 kB/s | 49 kB 00:00 2023-12-14T18:16:22.329 INFO:teuthology.orchestra.run.smithi204.stdout:(52/145): python3-kubernetes-11.0.0-6.el8.noarc 3.9 MB/s | 1.2 MB 00:00 2023-12-14T18:16:22.443 INFO:teuthology.orchestra.run.smithi031.stdout:(6/145): ceph-base-18.0.0-3905.g1faa3533.el8.x8 862 kB/s | 5.3 MB 00:06 2023-12-14T18:16:22.454 INFO:teuthology.orchestra.run.smithi204.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 371 kB/s | 59 kB 00:00 2023-12-14T18:16:22.480 INFO:teuthology.orchestra.run.smithi204.stdout:(54/145): python3-natsort-7.1.1-2.el8.noarch.rp 401 kB/s | 60 kB 00:00 2023-12-14T18:16:22.622 INFO:teuthology.orchestra.run.smithi204.stdout:(55/145): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-12-14T18:16:22.655 INFO:teuthology.orchestra.run.smithi204.stdout:(56/145): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-12-14T18:16:22.722 INFO:teuthology.orchestra.run.smithi204.stdout:(57/145): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-12-14T18:16:22.798 INFO:teuthology.orchestra.run.smithi204.stdout:(58/145): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-12-14T18:16:22.823 INFO:teuthology.orchestra.run.smithi204.stdout:(59/145): python3-rsa-4.9-2.el8.noarch.rpm 640 kB/s | 64 kB 00:00 2023-12-14T18:16:22.898 INFO:teuthology.orchestra.run.smithi204.stdout:(60/145): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-12-14T18:16:22.954 INFO:teuthology.orchestra.run.smithi204.stdout:(61/145): python3-singledispatch-3.4.0.3-18.el8 184 kB/s | 24 kB 00:00 2023-12-14T18:16:23.307 INFO:teuthology.orchestra.run.smithi204.stdout:(62/145): python3-tempora-1.14.1-5.el8.noarch.r 72 kB/s | 29 kB 00:00 2023-12-14T18:16:23.491 INFO:teuthology.orchestra.run.smithi204.stdout:(63/145): python3-trustme-0.6.0-4.el8.noarch.rp 51 kB/s | 27 kB 00:00 2023-12-14T18:16:23.550 INFO:teuthology.orchestra.run.smithi204.stdout:(64/145): python3-waitress-1.4.3-1.el8.noarch.r 1.0 MB/s | 255 kB 00:00 2023-12-14T18:16:23.667 INFO:teuthology.orchestra.run.smithi204.stdout:(65/145): python3-webob-1.8.5-1.el8.1.noarch.rp 1.4 MB/s | 251 kB 00:00 2023-12-14T18:16:23.692 INFO:teuthology.orchestra.run.smithi204.stdout:(66/145): python3-websocket-client-0.56.0-5.el8 431 kB/s | 61 kB 00:00 2023-12-14T18:16:23.784 INFO:teuthology.orchestra.run.smithi204.stdout:(67/145): python3-webtest-2.0.33-1.el8.noarch.r 732 kB/s | 85 kB 00:00 2023-12-14T18:16:23.809 INFO:teuthology.orchestra.run.smithi204.stdout:(68/145): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-12-14T18:16:23.835 INFO:teuthology.orchestra.run.smithi204.stdout:(69/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-14T18:16:23.988 INFO:teuthology.orchestra.run.smithi204.stdout:(70/145): libverto-devel-0.3.0-5.el8.x86_64.rpm 117 kB/s | 18 kB 00:00 2023-12-14T18:16:24.037 INFO:teuthology.orchestra.run.smithi204.stdout:(71/145): thrift-0.13.0-2.el8.x86_64.rpm 6.9 MB/s | 1.7 MB 00:00 2023-12-14T18:16:24.127 INFO:teuthology.orchestra.run.smithi204.stdout:(72/145): python3-jwt-1.6.1-2.el8.noarch.rpm 307 kB/s | 43 kB 00:00 2023-12-14T18:16:24.177 INFO:teuthology.orchestra.run.smithi204.stdout:(73/145): python3-oauthlib-2.1.0-1.el8.noarch.r 1.1 MB/s | 155 kB 00:00 2023-12-14T18:16:24.278 INFO:teuthology.orchestra.run.smithi204.stdout:(74/145): python3-pycparser-2.14-14.el8.noarch. 725 kB/s | 109 kB 00:00 2023-12-14T18:16:24.324 INFO:teuthology.orchestra.run.smithi204.stdout:(75/145): python3-requests-oauthlib-1.0.0-1.el8 291 kB/s | 43 kB 00:00 2023-12-14T18:16:24.370 INFO:teuthology.orchestra.run.smithi204.stdout:(76/145): libaio-devel-0.3.112-1.el8.x86_64.rpm 203 kB/s | 19 kB 00:00 2023-12-14T18:16:24.395 INFO:teuthology.orchestra.run.smithi204.stdout:(77/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 622 kB/s | 44 kB 00:00 2023-12-14T18:16:24.495 INFO:teuthology.orchestra.run.smithi204.stdout:(78/145): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-12-14T18:16:24.521 INFO:teuthology.orchestra.run.smithi204.stdout:(79/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-12-14T18:16:24.555 INFO:teuthology.orchestra.run.smithi204.stdout:(80/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-12-14T18:16:24.628 INFO:teuthology.orchestra.run.smithi031.stdout:(7/145): ceph-mon-18.0.0-3905.g1faa3533.el8.x86 1.3 MB/s | 4.6 MB 00:03 2023-12-14T18:16:24.630 INFO:teuthology.orchestra.run.smithi204.stdout:(81/145): libselinux-devel-2.9-5.el8.x86_64.rpm 1.8 MB/s | 200 kB 00:00 2023-12-14T18:16:24.680 INFO:teuthology.orchestra.run.smithi204.stdout:(82/145): keyutils-libs-devel-1.5.10-9.el8.x86_ 384 kB/s | 48 kB 00:00 2023-12-14T18:16:24.747 INFO:teuthology.orchestra.run.smithi204.stdout:(83/145): libsepol-devel-2.9-3.el8.x86_64.rpm 745 kB/s | 87 kB 00:00 2023-12-14T18:16:24.805 INFO:teuthology.orchestra.run.smithi204.stdout:(84/145): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 460 kB/s | 58 kB 00:00 2023-12-14T18:16:24.839 INFO:teuthology.orchestra.run.smithi204.stdout:(85/145): libcom_err-devel-1.45.6-4.el8.x86_64. 420 kB/s | 39 kB 00:00 2023-12-14T18:16:24.997 INFO:teuthology.orchestra.run.smithi204.stdout:(86/145): fuse-2.9.7-17.el8.x86_64.rpm 527 kB/s | 83 kB 00:00 2023-12-14T18:16:25.073 INFO:teuthology.orchestra.run.smithi204.stdout:(87/145): openssl-devel-1.1.1k-9.el8_7.x86_64.r 8.7 MB/s | 2.3 MB 00:00 2023-12-14T18:16:25.165 INFO:teuthology.orchestra.run.smithi204.stdout:(88/145): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2023-12-14T18:16:25.216 INFO:teuthology.orchestra.run.smithi204.stdout:(89/145): libquadmath-8.5.0-20.el8.x86_64.rpm 1.2 MB/s | 172 kB 00:00 2023-12-14T18:16:25.266 INFO:teuthology.orchestra.run.smithi204.stdout:(90/145): libuuid-devel-2.32.1-43.el8.x86_64.rp 993 kB/s | 100 kB 00:00 2023-12-14T18:16:25.366 INFO:teuthology.orchestra.run.smithi204.stdout:(91/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-12-14T18:16:25.475 INFO:teuthology.orchestra.run.smithi204.stdout:(92/145): python3-cryptography-3.2.1-6.el8.x86_ 2.6 MB/s | 559 kB 00:00 2023-12-14T18:16:25.500 INFO:teuthology.orchestra.run.smithi204.stdout:(93/145): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 544 kB/s | 73 kB 00:00 2023-12-14T18:16:25.559 INFO:teuthology.orchestra.run.smithi204.stdout:(94/145): NetworkManager-initscripts-updown-1.4 1.7 MB/s | 143 kB 00:00 2023-12-14T18:16:25.676 INFO:teuthology.orchestra.run.smithi204.stdout:(95/145): elfutils-libelf-devel-0.189-3.el8.x86 352 kB/s | 62 kB 00:00 2023-12-14T18:16:25.759 INFO:teuthology.orchestra.run.smithi204.stdout:(96/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 234 kB/s | 47 kB 00:00 2023-12-14T18:16:25.868 INFO:teuthology.orchestra.run.smithi204.stdout:(97/145): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 2.9 MB/s | 562 kB 00:00 2023-12-14T18:16:25.893 INFO:teuthology.orchestra.run.smithi204.stdout:(98/145): python3-jsonpointer-1.10-11.el8.noarc 150 kB/s | 20 kB 00:00 2023-12-14T18:16:25.918 INFO:teuthology.orchestra.run.smithi204.stdout:(99/145): python3-prettytable-0.7.2-14.el8.noar 875 kB/s | 44 kB 00:00 2023-12-14T18:16:26.027 INFO:teuthology.orchestra.run.smithi204.stdout:(100/145): python3-pyasn1-modules-0.3.7-6.el8.n 825 kB/s | 110 kB 00:00 2023-12-14T18:16:26.052 INFO:teuthology.orchestra.run.smithi204.stdout:(101/145): python3-jsonpatch-1.21-2.el8.noarch. 198 kB/s | 27 kB 00:00 2023-12-14T18:16:26.153 INFO:teuthology.orchestra.run.smithi204.stdout:(102/145): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2023-12-14T18:16:26.186 INFO:teuthology.orchestra.run.smithi204.stdout:(103/145): python3-werkzeug-0.12.2-4.el8.noarch 3.3 MB/s | 457 kB 00:00 2023-12-14T18:16:26.245 INFO:teuthology.orchestra.run.smithi204.stdout:(104/145): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-12-14T18:16:26.359 INFO:teuthology.orchestra.run.smithi204.stdout:(105/145): flex-2.6.1-9.el8.x86_64.rpm 1.8 MB/s | 320 kB 00:00 2023-12-14T18:16:26.404 INFO:teuthology.orchestra.run.smithi204.stdout:(106/145): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2023-12-14T18:16:26.496 INFO:teuthology.orchestra.run.smithi204.stdout:(107/145): python3-markupsafe-0.23-19.el8.x86_6 285 kB/s | 39 kB 00:00 2023-12-14T18:16:26.580 INFO:teuthology.orchestra.run.smithi204.stdout:(108/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-12-14T18:16:26.638 INFO:teuthology.orchestra.run.smithi204.stdout:(109/145): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-12-14T18:16:26.664 INFO:teuthology.orchestra.run.smithi204.stdout:(110/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-12-14T18:16:26.739 INFO:teuthology.orchestra.run.smithi204.stdout:(111/145): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.0 MB/s | 103 kB 00:00 2023-12-14T18:16:27.289 INFO:teuthology.orchestra.run.smithi204.stdout:(112/145): python3-babel-2.5.1-7.el8.noarch.rpm 8.7 MB/s | 4.8 MB 00:00 2023-12-14T18:16:27.506 INFO:teuthology.orchestra.run.smithi204.stdout:(113/145): python3-scipy-1.0.0-21.module+el8.5. 16 MB/s | 14 MB 00:00 2023-12-14T18:16:27.540 INFO:teuthology.orchestra.run.smithi204.stdout:(114/145): python3-jinja2-2.10.1-3.el8.noarch.r 2.1 MB/s | 538 kB 00:00 2023-12-14T18:16:27.607 INFO:teuthology.orchestra.run.smithi204.stdout:(115/145): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-12-14T18:16:27.729 INFO:teuthology.orchestra.run.smithi204.stdout:(116/145): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-12-14T18:16:27.766 INFO:teuthology.orchestra.run.smithi204.stdout:(117/145): python3-numpy-f2py-1.14.3-10.el8.x86 1.4 MB/s | 225 kB 00:00 2023-12-14T18:16:27.832 INFO:teuthology.orchestra.run.smithi204.stdout:(118/145): librdkafka-0.11.4-3.el8.x86_64.rpm 3.3 MB/s | 354 kB 00:00 2023-12-14T18:16:27.950 INFO:teuthology.orchestra.run.smithi204.stdout:(119/145): openblas-threads-0.3.15-6.el8.x86_64 28 MB/s | 5.2 MB 00:00 2023-12-14T18:16:28.016 INFO:teuthology.orchestra.run.smithi204.stdout:(120/145): python3-mako-1.0.6-14.el8.noarch.rpm 855 kB/s | 157 kB 00:00 2023-12-14T18:16:28.192 INFO:teuthology.orchestra.run.smithi204.stdout:(121/145): openblas-0.3.15-6.el8.x86_64.rpm 21 MB/s | 5.0 MB 00:00 2023-12-14T18:16:28.242 INFO:teuthology.orchestra.run.smithi204.stdout:(122/145): postgresql-10.23-2.module+el8.8.0+19 6.7 MB/s | 1.5 MB 00:00 2023-12-14T18:16:28.367 INFO:teuthology.orchestra.run.smithi204.stdout:(123/145): postgresql-contrib-10.23-2.module+el 4.5 MB/s | 811 kB 00:00 2023-12-14T18:16:28.501 INFO:teuthology.orchestra.run.smithi204.stdout:(124/145): postgresql-server-10.23-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-12-14T18:16:28.535 INFO:teuthology.orchestra.run.smithi204.stdout:(125/145): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2023-12-14T18:16:28.651 INFO:teuthology.orchestra.run.smithi204.stdout:(126/145): libpq-13.11-1.el8.x86_64.rpm 1.3 MB/s | 198 kB 00:00 2023-12-14T18:16:28.685 INFO:teuthology.orchestra.run.smithi204.stdout:(127/145): jq-1.6-7.el8.x86_64.rpm 1.3 MB/s | 202 kB 00:00 2023-12-14T18:16:28.819 INFO:teuthology.orchestra.run.smithi204.stdout:(128/145): python3-pytz-2017.2-11.el8.noarch.rp 324 kB/s | 54 kB 00:00 2023-12-14T18:16:29.370 INFO:teuthology.orchestra.run.smithi031.stdout:(8/145): ceph-common-18.0.0-3905.g1faa3533.el8. 1.6 MB/s | 22 MB 00:13 2023-12-14T18:16:29.486 INFO:teuthology.orchestra.run.smithi031.stdout:(9/145): ceph-selinux-18.0.0-3905.g1faa3533.el8 209 kB/s | 24 kB 00:00 2023-12-14T18:16:30.253 INFO:teuthology.orchestra.run.smithi204.stdout:(129/145): librados2-18.0.0-3905.g1faa3533.el8. 2.3 MB/s | 3.7 MB 00:01 2023-12-14T18:16:30.450 INFO:teuthology.orchestra.run.smithi204.stdout:(130/145): openssl-1.1.1k-9.el8_7.x86_64.rpm 3.5 MB/s | 710 kB 00:00 2023-12-14T18:16:30.628 INFO:teuthology.orchestra.run.smithi204.stdout:(131/145): librbd1-18.0.0-3905.g1faa3533.el8.x8 2.0 MB/s | 3.6 MB 00:01 2023-12-14T18:16:30.662 INFO:teuthology.orchestra.run.smithi204.stdout:(132/145): openssl-libs-1.1.1k-9.el8_7.x86_64.r 6.9 MB/s | 1.5 MB 00:00 2023-12-14T18:16:30.737 INFO:teuthology.orchestra.run.smithi204.stdout:(133/145): elfutils-libelf-0.189-3.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2023-12-14T18:16:30.829 INFO:teuthology.orchestra.run.smithi204.stdout:(134/145): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2023-12-14T18:16:30.888 INFO:teuthology.orchestra.run.smithi204.stdout:(135/145): libmount-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 237 kB 00:00 2023-12-14T18:16:30.971 INFO:teuthology.orchestra.run.smithi204.stdout:(136/145): libsmartcols-2.32.1-43.el8.x86_64.rp 1.2 MB/s | 179 kB 00:00 2023-12-14T18:16:31.138 INFO:teuthology.orchestra.run.smithi204.stdout:(137/145): libuuid-2.32.1-43.el8.x86_64.rpm 395 kB/s | 99 kB 00:00 2023-12-14T18:16:31.230 INFO:teuthology.orchestra.run.smithi204.stdout:(138/145): util-linux-2.32.1-43.el8.x86_64.rpm 9.6 MB/s | 2.5 MB 00:00 2023-12-14T18:16:31.322 INFO:teuthology.orchestra.run.smithi204.stdout:(139/145): util-linux-user-2.32.1-43.el8.x86_64 559 kB/s | 103 kB 00:00 2023-12-14T18:16:31.339 INFO:teuthology.orchestra.run.smithi204.stdout:(140/145): xfsdump-3.1.8-6.el8.x86_64.rpm 3.0 MB/s | 333 kB 00:00 2023-12-14T18:16:31.386 INFO:teuthology.orchestra.run.smithi031.stdout:(10/145): ceph-radosgw-18.0.0-3905.g1faa3533.el 1.9 MB/s | 13 MB 00:06 2023-12-14T18:16:31.561 INFO:teuthology.orchestra.run.smithi031.stdout:(11/145): cephfs-mirror-18.0.0-3905.g1faa3533.e 1.4 MB/s | 258 kB 00:00 2023-12-14T18:16:31.654 INFO:teuthology.orchestra.run.smithi031.stdout:(12/145): libcephfs-devel-18.0.0-3905.g1faa3533 338 kB/s | 31 kB 00:00 2023-12-14T18:16:31.789 INFO:teuthology.orchestra.run.smithi204.stdout:(141/145): ceph-test-18.0.0-3905.g1faa3533.el8. 2.5 MB/s | 41 MB 00:16 2023-12-14T18:16:31.823 INFO:teuthology.orchestra.run.smithi204.stdout:(142/145): libblkid-2.32.1-43.el8.x86_64.rpm 456 kB/s | 220 kB 00:00 2023-12-14T18:16:31.865 INFO:teuthology.orchestra.run.smithi204.stdout:(143/145): iproute-6.2.0-5.el8_9.x86_64.rpm 1.5 MB/s | 854 kB 00:00 2023-12-14T18:16:31.982 INFO:teuthology.orchestra.run.smithi204.stdout:(144/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.0 MB/s | 1.1 MB 00:00 2023-12-14T18:16:32.007 INFO:teuthology.orchestra.run.smithi204.stdout:(145/145): libfdisk-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 254 kB 00:00 2023-12-14T18:16:32.012 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:16:32.012 INFO:teuthology.orchestra.run.smithi204.stdout:Total 10 MB/s | 205 MB 00:20 2023-12-14T18:16:32.112 INFO:teuthology.orchestra.run.smithi031.stdout:(13/145): libcephfs2-18.0.0-3905.g1faa3533.el8. 1.6 MB/s | 771 kB 00:00 2023-12-14T18:16:32.221 INFO:teuthology.orchestra.run.smithi031.stdout:(14/145): libcephsqlite-18.0.0-3905.g1faa3533.e 1.2 MB/s | 137 kB 00:00 2023-12-14T18:16:32.371 INFO:teuthology.orchestra.run.smithi031.stdout:(15/145): librados-devel-18.0.0-3905.g1faa3533. 927 kB/s | 139 kB 00:00 2023-12-14T18:16:32.680 INFO:teuthology.orchestra.run.smithi031.stdout:(16/145): libradosstriper1-18.0.0-3905.g1faa353 1.5 MB/s | 473 kB 00:00 2023-12-14T18:16:32.835 INFO:teuthology.orchestra.run.smithi176.stdout:(10/145): ceph-osd-18.0.0-3905.g1faa3533.el8.x8 1.1 MB/s | 18 MB 00:15 2023-12-14T18:16:33.089 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2023-12-14T18:16:33.252 INFO:teuthology.orchestra.run.smithi176.stdout:(11/145): cephfs-mirror-18.0.0-3905.g1faa3533.e 620 kB/s | 258 kB 00:00 2023-12-14T18:16:33.369 INFO:teuthology.orchestra.run.smithi176.stdout:(12/145): libcephfs-devel-18.0.0-3905.g1faa3533 266 kB/s | 31 kB 00:00 2023-12-14T18:16:33.485 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2023-12-14T18:16:33.485 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2023-12-14T18:16:33.513 INFO:teuthology.orchestra.run.smithi031.stdout:(17/145): ceph-osd-18.0.0-3905.g1faa3533.el8.x8 1.6 MB/s | 18 MB 00:11 2023-12-14T18:16:33.588 INFO:teuthology.orchestra.run.smithi031.stdout:(18/145): python3-ceph-argparse-18.0.0-3905.g1f 601 kB/s | 45 kB 00:00 2023-12-14T18:16:33.663 INFO:teuthology.orchestra.run.smithi031.stdout:(19/145): python3-ceph-common-18.0.0-3905.g1faa 1.5 MB/s | 119 kB 00:00 2023-12-14T18:16:33.811 INFO:teuthology.orchestra.run.smithi031.stdout:(20/145): python3-cephfs-18.0.0-3905.g1faa3533. 1.3 MB/s | 197 kB 00:00 2023-12-14T18:16:33.989 INFO:teuthology.orchestra.run.smithi031.stdout:(21/145): python3-rados-18.0.0-3905.g1faa3533.e 2.0 MB/s | 370 kB 00:00 2023-12-14T18:16:34.165 INFO:teuthology.orchestra.run.smithi031.stdout:(22/145): python3-rbd-18.0.0-3905.g1faa3533.el8 2.0 MB/s | 364 kB 00:00 2023-12-14T18:16:34.240 INFO:teuthology.orchestra.run.smithi031.stdout:(23/145): python3-rgw-18.0.0-3905.g1faa3533.el8 1.4 MB/s | 107 kB 00:00 2023-12-14T18:16:34.241 INFO:teuthology.orchestra.run.smithi176.stdout:(13/145): libcephfs2-18.0.0-3905.g1faa3533.el8. 884 kB/s | 771 kB 00:00 2023-12-14T18:16:34.299 INFO:teuthology.orchestra.run.smithi031.stdout:(24/145): rbd-fuse-18.0.0-3905.g1faa3533.el8.x8 1.6 MB/s | 94 kB 00:00 2023-12-14T18:16:34.453 INFO:teuthology.orchestra.run.smithi176.stdout:(14/145): libcephsqlite-18.0.0-3905.g1faa3533.e 651 kB/s | 137 kB 00:00 2023-12-14T18:16:34.628 INFO:teuthology.orchestra.run.smithi176.stdout:(15/145): librados-devel-18.0.0-3905.g1faa3533. 794 kB/s | 139 kB 00:00 2023-12-14T18:16:35.041 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2023-12-14T18:16:35.052 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2023-12-14T18:16:35.572 INFO:teuthology.orchestra.run.smithi031.stdout:(25/145): rbd-mirror-18.0.0-3905.g1faa3533.el8. 2.7 MB/s | 3.4 MB 00:01 2023-12-14T18:16:35.655 INFO:teuthology.orchestra.run.smithi176.stdout:(16/145): ceph-test-18.0.0-3905.g1faa3533.el8.x 2.9 MB/s | 41 MB 00:13 2023-12-14T18:16:35.707 INFO:teuthology.orchestra.run.smithi031.stdout:(26/145): librgw2-18.0.0-3905.g1faa3533.el8.x86 2.2 MB/s | 6.5 MB 00:03 2023-12-14T18:16:35.732 INFO:teuthology.orchestra.run.smithi031.stdout:(27/145): rbd-nbd-18.0.0-3905.g1faa3533.el8.x86 1.1 MB/s | 179 kB 00:00 2023-12-14T18:16:35.800 INFO:teuthology.orchestra.run.smithi031.stdout:(28/145): ceph-grafana-dashboards-18.0.0-3905.g 251 kB/s | 23 kB 00:00 2023-12-14T18:16:35.812 INFO:teuthology.orchestra.run.smithi176.stdout:(17/145): libradosstriper1-18.0.0-3905.g1faa353 400 kB/s | 473 kB 00:01 2023-12-14T18:16:35.816 INFO:teuthology.orchestra.run.smithi031.stdout:(29/145): ceph-mgr-cephadm-18.0.0-3905.g1faa353 1.5 MB/s | 127 kB 00:00 2023-12-14T18:16:35.996 INFO:teuthology.orchestra.run.smithi176.stdout:(18/145): python3-ceph-argparse-18.0.0-3905.g1f 244 kB/s | 45 kB 00:00 2023-12-14T18:16:36.179 INFO:teuthology.orchestra.run.smithi176.stdout:(19/145): python3-ceph-common-18.0.0-3905.g1faa 648 kB/s | 119 kB 00:00 2023-12-14T18:16:36.429 INFO:teuthology.orchestra.run.smithi176.stdout:(20/145): python3-cephfs-18.0.0-3905.g1faa3533. 788 kB/s | 197 kB 00:00 2023-12-14T18:16:36.558 INFO:teuthology.orchestra.run.smithi031.stdout:(30/145): ceph-mgr-dashboard-18.0.0-3905.g1faa3 2.3 MB/s | 1.7 MB 00:00 2023-12-14T18:16:36.738 INFO:teuthology.orchestra.run.smithi176.stdout:(21/145): python3-rados-18.0.0-3905.g1faa3533.e 1.2 MB/s | 370 kB 00:00 2023-12-14T18:16:36.750 INFO:teuthology.orchestra.run.smithi031.stdout:(31/145): ceph-mgr-modules-core-18.0.0-3905.g1f 1.2 MB/s | 240 kB 00:00 2023-12-14T18:16:36.834 INFO:teuthology.orchestra.run.smithi031.stdout:(32/145): ceph-mgr-rook-18.0.0-3905.g1faa3533.e 570 kB/s | 47 kB 00:00 2023-12-14T18:16:36.917 INFO:teuthology.orchestra.run.smithi031.stdout:(33/145): ceph-prometheus-alerts-18.0.0-3905.g1 175 kB/s | 15 kB 00:00 2023-12-14T18:16:36.988 INFO:teuthology.orchestra.run.smithi176.stdout:(22/145): python3-rbd-18.0.0-3905.g1faa3533.el8 1.4 MB/s | 364 kB 00:00 2023-12-14T18:16:37.059 INFO:teuthology.orchestra.run.smithi031.stdout:(34/145): ceph-volume-18.0.0-3905.g1faa3533.el8 1.8 MB/s | 265 kB 00:00 2023-12-14T18:16:37.122 INFO:teuthology.orchestra.run.smithi176.stdout:(23/145): python3-rgw-18.0.0-3905.g1faa3533.el8 800 kB/s | 107 kB 00:00 2023-12-14T18:16:37.134 INFO:teuthology.orchestra.run.smithi031.stdout:(35/145): cephadm-18.0.0-3905.g1faa3533.el8.noa 1.1 MB/s | 83 kB 00:00 2023-12-14T18:16:37.206 INFO:teuthology.orchestra.run.smithi176.stdout:(24/145): rbd-fuse-18.0.0-3905.g1faa3533.el8.x8 1.1 MB/s | 94 kB 00:00 2023-12-14T18:16:37.209 INFO:teuthology.orchestra.run.smithi031.stdout:(36/145): cephfs-top-18.0.0-3905.g1faa3533.el8. 240 kB/s | 18 kB 00:00 2023-12-14T18:16:37.735 INFO:teuthology.orchestra.run.smithi031.stdout:(37/145): gperftools-libs-2.7-9.el8.x86_64.rpm 583 kB/s | 306 kB 00:00 2023-12-14T18:16:37.894 INFO:teuthology.orchestra.run.smithi031.stdout:(38/145): ceph-mgr-diskprediction-local-18.0.0- 3.6 MB/s | 7.4 MB 00:02 2023-12-14T18:16:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:(39/145): liboath-2.6.2-3.el8.x86_64.rpm 336 kB/s | 59 kB 00:00 2023-12-14T18:16:37.964 INFO:teuthology.orchestra.run.smithi176.stdout:(25/145): ceph-common-18.0.0-3905.g1faa3533.el8 887 kB/s | 22 MB 00:24 2023-12-14T18:16:38.039 INFO:teuthology.orchestra.run.smithi176.stdout:(26/145): librgw2-18.0.0-3905.g1faa3533.el8.x86 2.7 MB/s | 6.5 MB 00:02 2023-12-14T18:16:38.098 INFO:teuthology.orchestra.run.smithi176.stdout:(27/145): rbd-nbd-18.0.0-3905.g1faa3533.el8.x86 1.3 MB/s | 179 kB 00:00 2023-12-14T18:16:38.111 INFO:teuthology.orchestra.run.smithi031.stdout:(40/145): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2023-12-14T18:16:38.123 INFO:teuthology.orchestra.run.smithi176.stdout:(28/145): ceph-grafana-dashboards-18.0.0-3905.g 277 kB/s | 23 kB 00:00 2023-12-14T18:16:38.165 INFO:teuthology.orchestra.run.smithi176.stdout:(29/145): ceph-mgr-cephadm-18.0.0-3905.g1faa353 1.9 MB/s | 127 kB 00:00 2023-12-14T18:16:38.220 INFO:teuthology.orchestra.run.smithi031.stdout:(41/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-12-14T18:16:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:(42/145): libunwind-1.3.1-3.el8.x86_64.rpm 209 kB/s | 75 kB 00:00 2023-12-14T18:16:38.328 INFO:teuthology.orchestra.run.smithi031.stdout:(43/145): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-12-14T18:16:38.370 INFO:teuthology.orchestra.run.smithi031.stdout:(44/145): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-12-14T18:16:38.460 INFO:teuthology.orchestra.run.smithi031.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 124 kB/s | 16 kB 00:00 2023-12-14T18:16:38.588 INFO:teuthology.orchestra.run.smithi031.stdout:(46/145): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-12-14T18:16:38.607 INFO:teuthology.orchestra.run.smithi176.stdout:(30/145): ceph-mgr-dashboard-18.0.0-3905.g1faa3 3.5 MB/s | 1.7 MB 00:00 2023-12-14T18:16:38.613 INFO:teuthology.orchestra.run.smithi031.stdout:(47/145): python3-cheroot-8.5.2-1.el8.noarch.rp 713 kB/s | 173 kB 00:00 2023-12-14T18:16:38.688 INFO:teuthology.orchestra.run.smithi031.stdout:(48/145): python3-google-auth-1.1.1-10.el8.noar 812 kB/s | 81 kB 00:00 2023-12-14T18:16:38.732 INFO:teuthology.orchestra.run.smithi176.stdout:(31/145): ceph-mgr-modules-core-18.0.0-3905.g1f 1.9 MB/s | 240 kB 00:00 2023-12-14T18:16:38.763 INFO:teuthology.orchestra.run.smithi031.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 74 kB/s | 11 kB 00:00 2023-12-14T18:16:38.797 INFO:teuthology.orchestra.run.smithi031.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-12-14T18:16:38.807 INFO:teuthology.orchestra.run.smithi176.stdout:(32/145): ceph-mgr-rook-18.0.0-3905.g1faa3533.e 632 kB/s | 47 kB 00:00 2023-12-14T18:16:38.883 INFO:teuthology.orchestra.run.smithi176.stdout:(33/145): ceph-prometheus-alerts-18.0.0-3905.g1 194 kB/s | 15 kB 00:00 2023-12-14T18:16:38.923 INFO:teuthology.orchestra.run.smithi031.stdout:(51/145): python3-logutils-0.3.5-11.el8.noarch. 389 kB/s | 49 kB 00:00 2023-12-14T18:16:38.949 INFO:teuthology.orchestra.run.smithi176.stdout:(34/145): rbd-mirror-18.0.0-3905.g1faa3533.el8. 2.0 MB/s | 3.4 MB 00:01 2023-12-14T18:16:39.017 INFO:teuthology.orchestra.run.smithi176.stdout:(35/145): ceph-volume-18.0.0-3905.g1faa3533.el8 1.9 MB/s | 265 kB 00:00 2023-12-14T18:16:39.042 INFO:teuthology.orchestra.run.smithi176.stdout:(36/145): cephadm-18.0.0-3905.g1faa3533.el8.noa 911 kB/s | 83 kB 00:00 2023-12-14T18:16:39.075 INFO:teuthology.orchestra.run.smithi176.stdout:(37/145): cephfs-top-18.0.0-3905.g1faa3533.el8. 307 kB/s | 18 kB 00:00 2023-12-14T18:16:39.181 INFO:teuthology.orchestra.run.smithi031.stdout:(52/145): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2023-12-14T18:16:39.206 INFO:teuthology.orchestra.run.smithi031.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 207 kB/s | 59 kB 00:00 2023-12-14T18:16:39.298 INFO:teuthology.orchestra.run.smithi031.stdout:(54/145): python3-natsort-7.1.1-2.el8.noarch.rp 516 kB/s | 60 kB 00:00 2023-12-14T18:16:39.317 INFO:teuthology.orchestra.run.smithi176.stdout:(38/145): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2023-12-14T18:16:39.357 INFO:teuthology.orchestra.run.smithi031.stdout:(55/145): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-12-14T18:16:39.399 INFO:teuthology.orchestra.run.smithi031.stdout:(56/145): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-12-14T18:16:39.417 INFO:teuthology.orchestra.run.smithi176.stdout:(39/145): gperftools-libs-2.7-9.el8.x86_64.rpm 816 kB/s | 306 kB 00:00 2023-12-14T18:16:39.443 INFO:teuthology.orchestra.run.smithi176.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 600 kB/s | 75 kB 00:00 2023-12-14T18:16:39.458 INFO:teuthology.orchestra.run.smithi031.stdout:(57/145): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-12-14T18:16:39.499 INFO:teuthology.orchestra.run.smithi031.stdout:(58/145): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-12-14T18:16:39.501 INFO:teuthology.orchestra.run.smithi176.stdout:(41/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 747 kB/s | 44 kB 00:00 2023-12-14T18:16:39.635 INFO:teuthology.orchestra.run.smithi176.stdout:(42/145): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-12-14T18:16:39.650 INFO:teuthology.orchestra.run.smithi031.stdout:(59/145): python3-rsa-4.9-2.el8.noarch.rpm 334 kB/s | 64 kB 00:00 2023-12-14T18:16:39.675 INFO:teuthology.orchestra.run.smithi031.stdout:(60/145): python3-simplegeneric-0.8.1-17.el8.no 107 kB/s | 19 kB 00:00 2023-12-14T18:16:39.694 INFO:teuthology.orchestra.run.smithi176.stdout:(43/145): python3-cachetools-3.1.1-4.el8.noarch 566 kB/s | 33 kB 00:00 2023-12-14T18:16:39.719 INFO:teuthology.orchestra.run.smithi176.stdout:(44/145): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2023-12-14T18:16:39.769 INFO:teuthology.orchestra.run.smithi176.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-12-14T18:16:39.853 INFO:teuthology.orchestra.run.smithi176.stdout:(46/145): ceph-mgr-diskprediction-local-18.0.0- 4.4 MB/s | 7.4 MB 00:01 2023-12-14T18:16:39.870 INFO:teuthology.orchestra.run.smithi176.stdout:(47/145): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-12-14T18:16:39.920 INFO:teuthology.orchestra.run.smithi176.stdout:(48/145): python3-jaraco-6.2-6.el8.noarch.rpm 223 kB/s | 11 kB 00:00 2023-12-14T18:16:39.971 INFO:teuthology.orchestra.run.smithi176.stdout:(49/145): python3-jaraco-functools-2.0-4.el8.no 351 kB/s | 18 kB 00:00 2023-12-14T18:16:40.029 INFO:teuthology.orchestra.run.smithi176.stdout:(50/145): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2023-12-14T18:16:40.079 INFO:teuthology.orchestra.run.smithi176.stdout:(51/145): python3-logutils-0.3.5-11.el8.noarch. 975 kB/s | 49 kB 00:00 2023-12-14T18:16:40.109 INFO:teuthology.orchestra.run.smithi031.stdout:(61/145): ceph-test-18.0.0-3905.g1faa3533.el8.x 3.8 MB/s | 41 MB 00:10 2023-12-14T18:16:40.113 INFO:teuthology.orchestra.run.smithi176.stdout:(52/145): python3-google-auth-1.1.1-10.el8.noar 313 kB/s | 81 kB 00:00 2023-12-14T18:16:40.130 INFO:teuthology.orchestra.run.smithi176.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-12-14T18:16:40.134 INFO:teuthology.orchestra.run.smithi031.stdout:(62/145): python3-singledispatch-3.4.0.3-18.el8 49 kB/s | 24 kB 00:00 2023-12-14T18:16:40.159 INFO:teuthology.orchestra.run.smithi031.stdout:(63/145): python3-tempora-1.14.1-5.el8.noarch.r 60 kB/s | 29 kB 00:00 2023-12-14T18:16:40.163 INFO:teuthology.orchestra.run.smithi176.stdout:(54/145): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-12-14T18:16:40.222 INFO:teuthology.orchestra.run.smithi176.stdout:(55/145): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-12-14T18:16:40.281 INFO:teuthology.orchestra.run.smithi176.stdout:(56/145): python3-repoze-lru-0.7-6.el8.noarch.r 577 kB/s | 34 kB 00:00 2023-12-14T18:16:40.339 INFO:teuthology.orchestra.run.smithi176.stdout:(57/145): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-12-14T18:16:40.351 INFO:teuthology.orchestra.run.smithi031.stdout:(64/145): python3-waitress-1.4.3-1.el8.noarch.r 1.3 MB/s | 255 kB 00:00 2023-12-14T18:16:40.390 INFO:teuthology.orchestra.run.smithi176.stdout:(58/145): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-12-14T18:16:40.409 INFO:teuthology.orchestra.run.smithi031.stdout:(65/145): python3-webob-1.8.5-1.el8.1.noarch.rp 1.0 MB/s | 251 kB 00:00 2023-12-14T18:16:40.415 INFO:teuthology.orchestra.run.smithi176.stdout:(59/145): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-12-14T18:16:40.435 INFO:teuthology.orchestra.run.smithi031.stdout:(66/145): python3-trustme-0.6.0-4.el8.noarch.rp 84 kB/s | 27 kB 00:00 2023-12-14T18:16:40.449 INFO:teuthology.orchestra.run.smithi176.stdout:(60/145): python3-simplegeneric-0.8.1-17.el8.no 320 kB/s | 19 kB 00:00 2023-12-14T18:16:40.460 INFO:teuthology.orchestra.run.smithi031.stdout:(67/145): python3-websocket-client-0.56.0-5.el8 563 kB/s | 61 kB 00:00 2023-12-14T18:16:40.474 INFO:teuthology.orchestra.run.smithi176.stdout:(61/145): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-12-14T18:16:40.508 INFO:teuthology.orchestra.run.smithi176.stdout:(62/145): python3-tempora-1.14.1-5.el8.noarch.r 498 kB/s | 29 kB 00:00 2023-12-14T18:16:40.510 INFO:teuthology.orchestra.run.smithi031.stdout:(68/145): python3-webtest-2.0.33-1.el8.noarch.r 850 kB/s | 85 kB 00:00 2023-12-14T18:16:40.533 INFO:teuthology.orchestra.run.smithi176.stdout:(63/145): python3-trustme-0.6.0-4.el8.noarch.rp 469 kB/s | 27 kB 00:00 2023-12-14T18:16:40.544 INFO:teuthology.orchestra.run.smithi031.stdout:(69/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-12-14T18:16:40.577 INFO:teuthology.orchestra.run.smithi031.stdout:(70/145): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2023-12-14T18:16:40.683 INFO:teuthology.orchestra.run.smithi176.stdout:(64/145): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-12-14T18:16:40.708 INFO:teuthology.orchestra.run.smithi176.stdout:(65/145): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-12-14T18:16:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:(71/145): python3-jsonpointer-1.10-11.el8.noarc 120 kB/s | 20 kB 00:00 2023-12-14T18:16:40.733 INFO:teuthology.orchestra.run.smithi176.stdout:(66/145): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-12-14T18:16:40.753 INFO:teuthology.orchestra.run.smithi031.stdout:(72/145): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2023-12-14T18:16:40.759 INFO:teuthology.orchestra.run.smithi176.stdout:(67/145): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2023-12-14T18:16:40.778 INFO:teuthology.orchestra.run.smithi031.stdout:(73/145): python3-prettytable-0.7.2-14.el8.noar 229 kB/s | 44 kB 00:00 2023-12-14T18:16:40.851 INFO:teuthology.orchestra.run.smithi176.stdout:(68/145): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-12-14T18:16:40.876 INFO:teuthology.orchestra.run.smithi176.stdout:(69/145): python3-webtest-2.0.33-1.el8.noarch.r 601 kB/s | 85 kB 00:00 2023-12-14T18:16:40.879 INFO:teuthology.orchestra.run.smithi031.stdout:(74/145): python3-pyasn1-modules-0.3.7-6.el8.no 659 kB/s | 110 kB 00:00 2023-12-14T18:16:40.895 INFO:teuthology.orchestra.run.smithi031.stdout:(75/145): python3-jsonpatch-1.21-2.el8.noarch.r 187 kB/s | 27 kB 00:00 2023-12-14T18:16:40.901 INFO:teuthology.orchestra.run.smithi176.stdout:(70/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-14T18:16:40.929 INFO:teuthology.orchestra.run.smithi031.stdout:(76/145): python3-pyasn1-0.3.7-6.el8.noarch.rpm 891 kB/s | 126 kB 00:00 2023-12-14T18:16:41.027 INFO:teuthology.orchestra.run.smithi176.stdout:(71/145): python3-jwt-1.6.1-2.el8.noarch.rpm 357 kB/s | 43 kB 00:00 2023-12-14T18:16:41.037 INFO:teuthology.orchestra.run.smithi031.stdout:(77/145): python3-werkzeug-0.12.2-4.el8.noarch. 2.8 MB/s | 457 kB 00:00 2023-12-14T18:16:41.052 INFO:teuthology.orchestra.run.smithi176.stdout:(72/145): libverto-devel-0.3.0-5.el8.x86_64.rpm 102 kB/s | 18 kB 00:00 2023-12-14T18:16:41.079 INFO:teuthology.orchestra.run.smithi031.stdout:(78/145): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-12-14T18:16:41.127 INFO:teuthology.orchestra.run.smithi176.stdout:(73/145): python3-oauthlib-2.1.0-1.el8.noarch.r 1.5 MB/s | 155 kB 00:00 2023-12-14T18:16:41.146 INFO:teuthology.orchestra.run.smithi031.stdout:(79/145): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-12-14T18:16:41.173 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2023-12-14T18:16:41.197 INFO:teuthology.orchestra.run.smithi031.stdout:(80/145): bison-3.0.4-10.el8.x86_64.rpm 2.2 MB/s | 688 kB 00:00 2023-12-14T18:16:41.211 INFO:teuthology.orchestra.run.smithi176.stdout:(74/145): python3-pycparser-2.14-14.el8.noarch. 688 kB/s | 109 kB 00:00 2023-12-14T18:16:41.236 INFO:teuthology.orchestra.run.smithi176.stdout:(75/145): python3-requests-oauthlib-1.0.0-1.el8 392 kB/s | 43 kB 00:00 2023-12-14T18:16:41.281 INFO:teuthology.orchestra.run.smithi031.stdout:(81/145): python3-markupsafe-0.23-19.el8.x86_64 193 kB/s | 39 kB 00:00 2023-12-14T18:16:41.345 INFO:teuthology.orchestra.run.smithi176.stdout:(76/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 405 kB/s | 44 kB 00:00 2023-12-14T18:16:41.364 INFO:teuthology.orchestra.run.smithi031.stdout:(82/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-12-14T18:16:41.373 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-14T18:16:41.389 INFO:teuthology.orchestra.run.smithi031.stdout:(83/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2023-12-14T18:16:41.412 INFO:teuthology.orchestra.run.smithi176.stdout:(77/145): libaio-devel-0.3.112-1.el8.x86_64.rpm 93 kB/s | 19 kB 00:00 2023-12-14T18:16:41.423 INFO:teuthology.orchestra.run.smithi031.stdout:(84/145): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2023-12-14T18:16:41.507 INFO:teuthology.orchestra.run.smithi031.stdout:(85/145): python3-pyOpenSSL-19.0.0-1.el8.noarch 728 kB/s | 103 kB 00:00 2023-12-14T18:16:41.512 INFO:teuthology.orchestra.run.smithi176.stdout:(78/145): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2023-12-14T18:16:41.562 INFO:teuthology.orchestra.run.smithi176.stdout:(79/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-12-14T18:16:41.603 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:41.604 INFO:teuthology.orchestra.run.smithi176.stdout:(80/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-12-14T18:16:41.699 INFO:teuthology.orchestra.run.smithi031.stdout:(86/145): python3-jinja2-2.10.1-3.el8.noarch.rp 2.7 MB/s | 538 kB 00:00 2023-12-14T18:16:41.713 INFO:teuthology.orchestra.run.smithi176.stdout:(81/145): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2023-12-14T18:16:41.765 INFO:teuthology.orchestra.run.smithi031.stdout:(87/145): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-12-14T18:16:41.797 INFO:teuthology.orchestra.run.smithi176.stdout:(82/145): keyutils-libs-devel-1.5.10-9.el8.x86_ 251 kB/s | 48 kB 00:00 2023-12-14T18:16:41.867 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:41.872 INFO:teuthology.orchestra.run.smithi176.stdout:(83/145): thrift-0.13.0-2.el8.x86_64.rpm 1.6 MB/s | 1.7 MB 00:01 2023-12-14T18:16:41.938 INFO:teuthology.orchestra.run.smithi176.stdout:(84/145): libsepol-devel-2.9-3.el8.x86_64.rpm 386 kB/s | 87 kB 00:00 2023-12-14T18:16:41.964 INFO:teuthology.orchestra.run.smithi176.stdout:(85/145): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 345 kB/s | 58 kB 00:00 2023-12-14T18:16:42.047 INFO:teuthology.orchestra.run.smithi176.stdout:(86/145): fuse-2.9.7-17.el8.x86_64.rpm 997 kB/s | 83 kB 00:00 2023-12-14T18:16:42.065 INFO:teuthology.orchestra.run.smithi176.stdout:(87/145): libcom_err-devel-1.45.6-4.el8.x86_64. 201 kB/s | 39 kB 00:00 2023-12-14T18:16:42.090 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2023-12-14T18:16:42.099 INFO:teuthology.orchestra.run.smithi031.stdout:(88/145): python3-scipy-1.0.0-21.module+el8.5.0 19 MB/s | 14 MB 00:00 2023-12-14T18:16:42.131 INFO:teuthology.orchestra.run.smithi176.stdout:(88/145): openssl-devel-1.1.1k-9.el8_7.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-12-14T18:16:42.157 INFO:teuthology.orchestra.run.smithi031.stdout:(89/145): socat-1.7.4.1-1.el8.x86_64.rpm 826 kB/s | 323 kB 00:00 2023-12-14T18:16:42.215 INFO:teuthology.orchestra.run.smithi176.stdout:(89/145): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2023-12-14T18:16:42.233 INFO:teuthology.orchestra.run.smithi031.stdout:(90/145): python3-numpy-1.14.3-10.el8.x86_64.rp 7.0 MB/s | 3.7 MB 00:00 2023-12-14T18:16:42.248 INFO:teuthology.orchestra.run.smithi176.stdout:(90/145): libquadmath-8.5.0-20.el8.x86_64.rpm 938 kB/s | 172 kB 00:00 2023-12-14T18:16:42.251 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:42.266 INFO:teuthology.orchestra.run.smithi031.stdout:(91/145): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-12-14T18:16:42.308 INFO:teuthology.orchestra.run.smithi031.stdout:(92/145): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2023-12-14T18:16:42.315 INFO:teuthology.orchestra.run.smithi176.stdout:(91/145): libuuid-devel-2.32.1-43.el8.x86_64.rp 542 kB/s | 100 kB 00:00 2023-12-14T18:16:42.355 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:42.367 INFO:teuthology.orchestra.run.smithi031.stdout:(93/145): python3-mako-1.0.6-14.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-12-14T18:16:42.374 INFO:teuthology.orchestra.run.smithi176.stdout:(92/145): python3-cryptography-3.2.1-6.el8.x86_ 4.4 MB/s | 559 kB 00:00 2023-12-14T18:16:42.475 INFO:teuthology.orchestra.run.smithi176.stdout:(93/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 917 kB/s | 238 kB 00:00 2023-12-14T18:16:42.491 INFO:teuthology.orchestra.run.smithi176.stdout:(94/145): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 413 kB/s | 73 kB 00:00 2023-12-14T18:16:42.501 INFO:teuthology.orchestra.run.smithi031.stdout:(94/145): postgresql-10.23-2.module+el8.8.0+194 11 MB/s | 1.5 MB 00:00 2023-12-14T18:16:42.508 INFO:teuthology.orchestra.run.smithi176.stdout:(95/145): NetworkManager-initscripts-updown-1.4 1.0 MB/s | 143 kB 00:00 2023-12-14T18:16:42.544 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:42.584 INFO:teuthology.orchestra.run.smithi031.stdout:(95/145): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2023-12-14T18:16:42.615 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:42.617 INFO:teuthology.orchestra.run.smithi176.stdout:(96/145): elfutils-libelf-devel-0.189-3.el8.x86 435 kB/s | 62 kB 00:00 2023-12-14T18:16:42.642 INFO:teuthology.orchestra.run.smithi176.stdout:(97/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 312 kB/s | 47 kB 00:00 2023-12-14T18:16:42.651 INFO:teuthology.orchestra.run.smithi031.stdout:(96/145): postgresql-contrib-10.23-2.module+el8 5.3 MB/s | 811 kB 00:00 2023-12-14T18:16:42.692 INFO:teuthology.orchestra.run.smithi176.stdout:(98/145): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.0 MB/s | 562 kB 00:00 2023-12-14T18:16:42.718 INFO:teuthology.orchestra.run.smithi176.stdout:(99/145): python3-prettytable-0.7.2-14.el8.noar 585 kB/s | 44 kB 00:00 2023-12-14T18:16:42.743 INFO:teuthology.orchestra.run.smithi176.stdout:(100/145): python3-jsonpointer-1.10-11.el8.noar 159 kB/s | 20 kB 00:00 2023-12-14T18:16:42.760 INFO:teuthology.orchestra.run.smithi031.stdout:(97/145): openblas-threads-0.3.15-6.el8.x86_64. 9.9 MB/s | 5.2 MB 00:00 2023-12-14T18:16:42.810 INFO:teuthology.orchestra.run.smithi176.stdout:(101/145): python3-jsonpatch-1.21-2.el8.noarch. 288 kB/s | 27 kB 00:00 2023-12-14T18:16:42.818 INFO:teuthology.orchestra.run.smithi031.stdout:(98/145): automake-1.16.1-8.el8.noarch.rpm 4.2 MB/s | 714 kB 00:00 2023-12-14T18:16:42.832 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2023-12-14T18:16:42.843 INFO:teuthology.orchestra.run.smithi176.stdout:(102/145): python3-pyasn1-modules-0.3.7-6.el8.n 730 kB/s | 110 kB 00:00 2023-12-14T18:16:42.910 INFO:teuthology.orchestra.run.smithi031.stdout:(99/145): postgresql-server-10.23-2.module+el8. 16 MB/s | 5.1 MB 00:00 2023-12-14T18:16:42.927 INFO:teuthology.orchestra.run.smithi176.stdout:(103/145): python3-werkzeug-0.12.2-4.el8.noarch 3.8 MB/s | 457 kB 00:00 2023-12-14T18:16:42.961 INFO:teuthology.orchestra.run.smithi176.stdout:(104/145): python3-pyasn1-0.3.7-6.el8.noarch.rp 581 kB/s | 126 kB 00:00 2023-12-14T18:16:42.969 INFO:teuthology.orchestra.run.smithi031.stdout:(100/145): libpq-13.11-1.el8.x86_64.rpm 948 kB/s | 198 kB 00:00 2023-12-14T18:16:43.011 INFO:teuthology.orchestra.run.smithi176.stdout:(105/145): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2023-12-14T18:16:43.061 INFO:teuthology.orchestra.run.smithi031.stdout:(101/145): python3-pytz-2017.2-11.el8.noarch.rp 361 kB/s | 54 kB 00:00 2023-12-14T18:16:43.068 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2023-12-14T18:16:43.086 INFO:teuthology.orchestra.run.smithi031.stdout:(102/145): libverto-devel-0.3.0-5.el8.x86_64.rp 153 kB/s | 18 kB 00:00 2023-12-14T18:16:43.111 INFO:teuthology.orchestra.run.smithi031.stdout:(103/145): jq-1.6-7.el8.x86_64.rpm 692 kB/s | 202 kB 00:00 2023-12-14T18:16:43.161 INFO:teuthology.orchestra.run.smithi176.stdout:(106/145): python3-markupsafe-0.23-19.el8.x86_6 258 kB/s | 39 kB 00:00 2023-12-14T18:16:43.195 INFO:teuthology.orchestra.run.smithi031.stdout:(104/145): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2023-12-14T18:16:43.203 INFO:teuthology.orchestra.run.smithi176.stdout:(107/145): flex-2.6.1-9.el8.x86_64.rpm 1.1 MB/s | 320 kB 00:00 2023-12-14T18:16:43.262 INFO:teuthology.orchestra.run.smithi031.stdout:(105/145): python3-oauthlib-2.1.0-1.el8.noarch. 883 kB/s | 155 kB 00:00 2023-12-14T18:16:43.270 INFO:teuthology.orchestra.run.smithi176.stdout:(108/145): libtool-2.4.6-25.el8.x86_64.rpm 2.2 MB/s | 709 kB 00:00 2023-12-14T18:16:43.285 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/161 2023-12-14T18:16:43.350 INFO:teuthology.orchestra.run.smithi176.stdout:(109/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.3 MB/s | 187 kB 00:00 2023-12-14T18:16:43.355 INFO:teuthology.orchestra.run.smithi031.stdout:(106/145): python3-pycparser-2.14-14.el8.noarch 449 kB/s | 109 kB 00:00 2023-12-14T18:16:43.379 INFO:teuthology.orchestra.run.smithi031.stdout:(107/145): python3-requests-oauthlib-1.0.0-1.el 232 kB/s | 43 kB 00:00 2023-12-14T18:16:43.421 INFO:teuthology.orchestra.run.smithi176.stdout:(110/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2023-12-14T18:16:43.432 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/161 2023-12-14T18:16:43.446 INFO:teuthology.orchestra.run.smithi176.stdout:(111/145): uuid-1.6.2-43.el8.x86_64.rpm 364 kB/s | 64 kB 00:00 2023-12-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi031.stdout:(108/145): libaio-devel-0.3.112-1.el8.x86_64.rp 97 kB/s | 19 kB 00:00 2023-12-14T18:16:43.471 INFO:teuthology.orchestra.run.smithi176.stdout:(112/145): python3-pyOpenSSL-19.0.0-1.el8.noarc 852 kB/s | 103 kB 00:00 2023-12-14T18:16:43.480 INFO:teuthology.orchestra.run.smithi031.stdout:(109/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 353 kB/s | 44 kB 00:00 2023-12-14T18:16:43.513 INFO:teuthology.orchestra.run.smithi031.stdout:(110/145): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-12-14T18:16:43.589 INFO:teuthology.orchestra.run.smithi031.stdout:(111/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.7 MB/s | 229 kB 00:00 2023-12-14T18:16:43.618 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:43.664 INFO:teuthology.orchestra.run.smithi031.stdout:(112/145): libselinux-devel-2.9-5.el8.x86_64.rp 1.3 MB/s | 200 kB 00:00 2023-12-14T18:16:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:(113/145): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-12-14T18:16:43.689 INFO:teuthology.orchestra.run.smithi031.stdout:(113/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2023-12-14T18:16:43.705 INFO:teuthology.orchestra.run.smithi176.stdout:(114/145): python3-jinja2-2.10.1-3.el8.noarch.r 2.3 MB/s | 538 kB 00:00 2023-12-14T18:16:43.823 INFO:teuthology.orchestra.run.smithi031.stdout:(114/145): libsepol-devel-2.9-3.el8.x86_64.rpm 549 kB/s | 87 kB 00:00 2023-12-14T18:16:43.840 INFO:teuthology.orchestra.run.smithi031.stdout:(115/145): keyutils-libs-devel-1.5.10-9.el8.x86 192 kB/s | 48 kB 00:00 2023-12-14T18:16:43.857 INFO:teuthology.orchestra.run.smithi031.stdout:(116/145): zlib-devel-1.2.11-18.el8_5.x86_64.rp 346 kB/s | 58 kB 00:00 2023-12-14T18:16:43.889 INFO:teuthology.orchestra.run.smithi176.stdout:(115/145): python3-scipy-1.0.0-21.module+el8.5. 29 MB/s | 14 MB 00:00 2023-12-14T18:16:43.922 INFO:teuthology.orchestra.run.smithi176.stdout:(116/145): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2023-12-14T18:16:43.957 INFO:teuthology.orchestra.run.smithi031.stdout:(117/145): libcom_err-devel-1.45.6-4.el8.x86_64 289 kB/s | 39 kB 00:00 2023-12-14T18:16:43.999 INFO:teuthology.orchestra.run.smithi031.stdout:(118/145): fuse-2.9.7-17.el8.x86_64.rpm 586 kB/s | 83 kB 00:00 2023-12-14T18:16:44.006 INFO:teuthology.orchestra.run.smithi176.stdout:(117/145): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2023-12-14T18:16:44.048 INFO:teuthology.orchestra.run.smithi176.stdout:(118/145): python3-numpy-f2py-1.14.3-10.el8.x86 1.4 MB/s | 225 kB 00:00 2023-12-14T18:16:44.053 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:44.057 INFO:teuthology.orchestra.run.smithi031.stdout:(119/145): openssl-devel-1.1.1k-9.el8_7.x86_64. 11 MB/s | 2.3 MB 00:00 2023-12-14T18:16:44.108 INFO:teuthology.orchestra.run.smithi031.stdout:(120/145): libgfortran-8.5.0-20.el8.x86_64.rpm 4.2 MB/s | 645 kB 00:00 2023-12-14T18:16:44.133 INFO:teuthology.orchestra.run.smithi031.stdout:(121/145): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-12-14T18:16:44.139 INFO:teuthology.orchestra.run.smithi176.stdout:(119/145): librdkafka-0.11.4-3.el8.x86_64.rpm 1.6 MB/s | 354 kB 00:00 2023-12-14T18:16:44.224 INFO:teuthology.orchestra.run.smithi031.stdout:(122/145): libuuid-devel-2.32.1-43.el8.x86_64.r 601 kB/s | 100 kB 00:00 2023-12-14T18:16:44.252 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:44.255 INFO:teuthology.orchestra.run.smithi176.stdout:(120/145): python3-mako-1.0.6-14.el8.noarch.rpm 757 kB/s | 157 kB 00:00 2023-12-14T18:16:44.268 INFO:teuthology.orchestra.run.smithi031.stdout:(123/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-12-14T18:16:44.325 INFO:teuthology.orchestra.run.smithi031.stdout:(124/145): python3-cryptography-3.2.1-6.el8.x86 2.8 MB/s | 559 kB 00:00 2023-12-14T18:16:44.351 INFO:teuthology.orchestra.run.smithi031.stdout:(125/145): xfsprogs-devel-5.0.0-12.el8.x86_64.r 573 kB/s | 73 kB 00:00 2023-12-14T18:16:44.382 INFO:teuthology.orchestra.run.smithi176.stdout:(121/145): openblas-threads-0.3.15-6.el8.x86_64 14 MB/s | 5.2 MB 00:00 2023-12-14T18:16:44.401 INFO:teuthology.orchestra.run.smithi031.stdout:(126/145): NetworkManager-initscripts-updown-1. 1.0 MB/s | 143 kB 00:00 2023-12-14T18:16:44.443 INFO:teuthology.orchestra.run.smithi031.stdout:(127/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 511 kB/s | 47 kB 00:00 2023-12-14T18:16:44.468 INFO:teuthology.orchestra.run.smithi031.stdout:(128/145): elfutils-libelf-devel-0.189-3.el8.x8 433 kB/s | 62 kB 00:00 2023-12-14T18:16:44.474 INFO:teuthology.orchestra.run.smithi176.stdout:(122/145): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2023-12-14T18:16:44.516 INFO:teuthology.orchestra.run.smithi176.stdout:(123/145): postgresql-10.23-2.module+el8.8.0+19 5.8 MB/s | 1.5 MB 00:00 2023-12-14T18:16:44.543 INFO:teuthology.orchestra.run.smithi031.stdout:(129/145): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.9 MB/s | 562 kB 00:00 2023-12-14T18:16:44.558 INFO:teuthology.orchestra.run.smithi176.stdout:(124/145): postgresql-contrib-10.23-2.module+el 4.5 MB/s | 811 kB 00:00 2023-12-14T18:16:44.666 INFO:teuthology.orchestra.run.smithi176.stdout:(125/145): postgresql-server-10.23-2.module+el8 26 MB/s | 5.1 MB 00:00 2023-12-14T18:16:44.675 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:44.694 INFO:teuthology.orchestra.run.smithi031.stdout:(130/145): openssl-1.1.1k-9.el8_7.x86_64.rpm 4.6 MB/s | 710 kB 00:00 2023-12-14T18:16:44.725 INFO:teuthology.orchestra.run.smithi176.stdout:(126/145): automake-1.16.1-8.el8.noarch.rpm 3.3 MB/s | 714 kB 00:00 2023-12-14T18:16:44.750 INFO:teuthology.orchestra.run.smithi176.stdout:(127/145): jq-1.6-7.el8.x86_64.rpm 2.4 MB/s | 202 kB 00:00 2023-12-14T18:16:44.809 INFO:teuthology.orchestra.run.smithi176.stdout:(128/145): libpq-13.11-1.el8.x86_64.rpm 789 kB/s | 198 kB 00:00 2023-12-14T18:16:44.826 INFO:teuthology.orchestra.run.smithi176.stdout:(129/145): python3-pytz-2017.2-11.el8.noarch.rp 538 kB/s | 54 kB 00:00 2023-12-14T18:16:44.851 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:44.877 INFO:teuthology.orchestra.run.smithi031.stdout:(131/145): openssl-libs-1.1.1k-9.el8_7.x86_64.r 8.0 MB/s | 1.5 MB 00:00 2023-12-14T18:16:44.967 INFO:teuthology.orchestra.run.smithi176.stdout:(130/145): openssl-1.1.1k-9.el8_7.x86_64.rpm 4.9 MB/s | 710 kB 00:00 2023-12-14T18:16:44.981 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:45.019 INFO:teuthology.orchestra.run.smithi031.stdout:(132/145): librbd1-18.0.0-3905.g1faa3533.el8.x8 6.5 MB/s | 3.6 MB 00:00 2023-12-14T18:16:45.044 INFO:teuthology.orchestra.run.smithi031.stdout:(133/145): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2023-12-14T18:16:45.136 INFO:teuthology.orchestra.run.smithi031.stdout:(134/145): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2023-12-14T18:16:45.144 INFO:teuthology.orchestra.run.smithi176.stdout:(131/145): openssl-libs-1.1.1k-9.el8_7.x86_64.r 8.3 MB/s | 1.5 MB 00:00 2023-12-14T18:16:45.201 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:45.278 INFO:teuthology.orchestra.run.smithi031.stdout:(135/145): libmount-2.32.1-43.el8.x86_64.rpm 1.0 MB/s | 237 kB 00:00 2023-12-14T18:16:45.293 INFO:teuthology.orchestra.run.smithi176.stdout:(132/145): elfutils-libelf-0.189-3.el8.x86_64.r 1.5 MB/s | 232 kB 00:00 2023-12-14T18:16:45.303 INFO:teuthology.orchestra.run.smithi031.stdout:(136/145): libsmartcols-2.32.1-43.el8.x86_64.rp 1.1 MB/s | 179 kB 00:00 2023-12-14T18:16:45.304 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:45.429 INFO:teuthology.orchestra.run.smithi031.stdout:(137/145): libuuid-2.32.1-43.el8.x86_64.rpm 659 kB/s | 99 kB 00:00 2023-12-14T18:16:45.479 INFO:teuthology.orchestra.run.smithi031.stdout:(138/145): librados2-18.0.0-3905.g1faa3533.el8. 3.5 MB/s | 3.7 MB 00:01 2023-12-14T18:16:45.494 INFO:teuthology.orchestra.run.smithi176.stdout:(133/145): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2023-12-14T18:16:45.554 INFO:teuthology.orchestra.run.smithi031.stdout:(139/145): util-linux-2.32.1-43.el8.x86_64.rpm 9.9 MB/s | 2.5 MB 00:00 2023-12-14T18:16:45.571 INFO:teuthology.orchestra.run.smithi031.stdout:(140/145): util-linux-user-2.32.1-43.el8.x86_64 722 kB/s | 103 kB 00:00 2023-12-14T18:16:45.605 INFO:teuthology.orchestra.run.smithi031.stdout:(141/145): xfsdump-3.1.8-6.el8.x86_64.rpm 2.6 MB/s | 333 kB 00:00 2023-12-14T18:16:45.660 INFO:teuthology.orchestra.run.smithi176.stdout:(134/145): librbd1-18.0.0-3905.g1faa3533.el8.x8 4.2 MB/s | 3.6 MB 00:00 2023-12-14T18:16:45.680 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:16:45.685 INFO:teuthology.orchestra.run.smithi176.stdout:(135/145): libmount-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 237 kB 00:00 2023-12-14T18:16:45.705 INFO:teuthology.orchestra.run.smithi031.stdout:(142/145): libblkid-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2023-12-14T18:16:45.736 INFO:teuthology.orchestra.run.smithi176.stdout:(136/145): libsmartcols-2.32.1-43.el8.x86_64.rp 2.3 MB/s | 179 kB 00:00 2023-12-14T18:16:45.739 INFO:teuthology.orchestra.run.smithi031.stdout:(143/145): iproute-6.2.0-5.el8_9.x86_64.rpm 4.5 MB/s | 854 kB 00:00 2023-12-14T18:16:45.805 INFO:teuthology.orchestra.run.smithi031.stdout:(144/145): libfdisk-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 254 kB 00:00 2023-12-14T18:16:45.844 INFO:teuthology.orchestra.run.smithi176.stdout:(137/145): libuuid-2.32.1-43.el8.x86_64.rpm 623 kB/s | 99 kB 00:00 2023-12-14T18:16:45.928 INFO:teuthology.orchestra.run.smithi176.stdout:(138/145): util-linux-2.32.1-43.el8.x86_64.rpm 13 MB/s | 2.5 MB 00:00 2023-12-14T18:16:45.959 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:45.979 INFO:teuthology.orchestra.run.smithi176.stdout:(139/145): util-linux-user-2.32.1-43.el8.x86_64 768 kB/s | 103 kB 00:00 2023-12-14T18:16:45.989 INFO:teuthology.orchestra.run.smithi031.stdout:(145/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 3.8 MB/s | 1.1 MB 00:00 2023-12-14T18:16:45.993 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:16:45.993 INFO:teuthology.orchestra.run.smithi031.stdout:Total 6.8 MB/s | 205 MB 00:30 2023-12-14T18:16:46.029 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:46.062 INFO:teuthology.orchestra.run.smithi176.stdout:(140/145): librados2-18.0.0-3905.g1faa3533.el8. 2.8 MB/s | 3.7 MB 00:01 2023-12-14T18:16:46.087 INFO:teuthology.orchestra.run.smithi176.stdout:(141/145): xfsdump-3.1.8-6.el8.x86_64.rpm 2.0 MB/s | 333 kB 00:00 2023-12-14T18:16:46.121 INFO:teuthology.orchestra.run.smithi176.stdout:(142/145): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2023-12-14T18:16:46.188 INFO:teuthology.orchestra.run.smithi176.stdout:(143/145): libblkid-2.32.1-43.el8.x86_64.rpm 1.7 MB/s | 220 kB 00:00 2023-12-14T18:16:46.221 INFO:teuthology.orchestra.run.smithi176.stdout:(144/145): libfdisk-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 254 kB 00:00 2023-12-14T18:16:46.265 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2023-12-14T18:16:46.272 INFO:teuthology.orchestra.run.smithi176.stdout:(145/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 7.1 MB/s | 1.1 MB 00:00 2023-12-14T18:16:46.276 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:16:46.276 INFO:teuthology.orchestra.run.smithi176.stdout:Total 6.1 MB/s | 205 MB 00:33 2023-12-14T18:16:46.531 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2023-12-14T18:16:46.705 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-12-14T18:16:47.111 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2023-12-14T18:16:47.116 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-12-14T18:16:47.343 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2023-12-14T18:16:47.363 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 18/161 2023-12-14T18:16:47.543 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:47.570 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-12-14T18:16:47.570 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-12-14T18:16:47.637 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:47.740 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2023-12-14T18:16:47.740 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2023-12-14T18:16:47.835 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:47.893 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:48.046 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/161 2023-12-14T18:16:48.321 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/161 2023-12-14T18:16:48.574 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/161 2023-12-14T18:16:48.743 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/161 2023-12-14T18:16:48.910 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/161 2023-12-14T18:16:49.051 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/161 2023-12-14T18:16:49.220 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-12-14T18:16:49.232 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-12-14T18:16:49.256 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2023-12-14T18:16:49.266 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2023-12-14T18:16:49.680 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/161 2023-12-14T18:16:49.893 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/161 2023-12-14T18:16:50.504 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/161 2023-12-14T18:16:50.684 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/161 2023-12-14T18:16:51.388 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/161 2023-12-14T18:16:51.649 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/161 2023-12-14T18:16:51.823 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/161 2023-12-14T18:16:52.093 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2023-12-14T18:16:52.282 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 31/161 2023-12-14T18:16:52.420 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 32/161 2023-12-14T18:16:52.688 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 33/161 2023-12-14T18:16:52.851 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2023-12-14T18:16:54.620 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2023-12-14T18:16:54.699 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2023-12-14T18:16:54.904 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-14T18:16:54.952 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 35/161 2023-12-14T18:16:55.164 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:55.244 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-12-14T18:16:55.416 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:55.517 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-14T18:16:55.651 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2023-12-14T18:16:55.835 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:55.901 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:55.909 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:55.914 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/161 2023-12-14T18:16:56.169 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:56.197 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:56.235 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:56.238 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/161 2023-12-14T18:16:56.438 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2023-12-14T18:16:56.482 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2023-12-14T18:16:56.492 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/161 2023-12-14T18:16:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:56.694 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2023-12-14T18:16:56.728 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-12-14T18:16:56.741 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:56.953 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/161 2023-12-14T18:16:56.971 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:56.977 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2023-12-14T18:16:57.045 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:57.070 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/161 2023-12-14T18:16:57.197 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 41/161 2023-12-14T18:16:57.276 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2023-12-14T18:16:57.403 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 42/161 2023-12-14T18:16:57.521 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/161 2023-12-14T18:16:57.588 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/161 2023-12-14T18:16:57.600 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:57.677 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:57.704 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:57.782 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/161 2023-12-14T18:16:57.908 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:57.918 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:57.998 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:58.205 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2023-12-14T18:16:58.316 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:58.428 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2023-12-14T18:16:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:58.518 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:58.600 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:58.642 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:58.876 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:58.964 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:59.140 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:59.280 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:16:59.327 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:59.495 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:59.551 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:59.634 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:59.662 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:59.717 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:59.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:59.812 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:59.882 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2023-12-14T18:16:59.911 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:17:00.001 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.1-8.el8.noarch 49/161 2023-12-14T18:17:00.163 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2023-12-14T18:17:00.199 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-43.el8.x86_64 50/161 2023-12-14T18:17:00.258 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:17:00.278 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 50/161 2023-12-14T18:17:00.337 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-12-14T18:17:00.424 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/161 2023-12-14T18:17:00.527 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/161 2023-12-14T18:17:00.545 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:17:00.648 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:17:00.691 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-7.el8.x86_64 52/161 2023-12-14T18:17:00.692 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2023-12-14T18:17:00.881 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/161 2023-12-14T18:17:00.894 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2023-12-14T18:17:00.939 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 18/161 2023-12-14T18:17:01.040 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/161 2023-12-14T18:17:01.118 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:01.185 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2023-12-14T18:17:01.223 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:01.267 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/161 2023-12-14T18:17:01.413 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:17:01.418 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-12-14T18:17:01.453 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/161 2023-12-14T18:17:01.512 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:17:01.663 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/161 2023-12-14T18:17:01.681 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/161 2023-12-14T18:17:01.838 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2023-12-14T18:17:01.888 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/161 2023-12-14T18:17:01.902 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/161 2023-12-14T18:17:02.077 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/161 2023-12-14T18:17:02.139 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/161 2023-12-14T18:17:02.147 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 18/161 2023-12-14T18:17:02.287 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/161 2023-12-14T18:17:02.301 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/161 2023-12-14T18:17:02.312 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:02.423 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:02.461 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/161 2023-12-14T18:17:02.570 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/161 2023-12-14T18:17:02.617 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/161 2023-12-14T18:17:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:17:02.810 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:17:02.837 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/161 2023-12-14T18:17:03.003 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/161 2023-12-14T18:17:03.062 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/161 2023-12-14T18:17:03.160 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/161 2023-12-14T18:17:03.239 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/161 2023-12-14T18:17:03.245 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/161 2023-12-14T18:17:03.320 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/161 2023-12-14T18:17:03.401 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/161 2023-12-14T18:17:03.443 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/161 2023-12-14T18:17:03.808 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/161 2023-12-14T18:17:04.004 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/161 2023-12-14T18:17:04.012 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/161 2023-12-14T18:17:04.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 67/161 2023-12-14T18:17:04.220 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/161 2023-12-14T18:17:04.250 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/161 2023-12-14T18:17:04.386 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/161 2023-12-14T18:17:04.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/161 2023-12-14T18:17:04.853 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 70/161 2023-12-14T18:17:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/161 2023-12-14T18:17:04.943 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/161 2023-12-14T18:17:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/161 2023-12-14T18:17:05.215 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/161 2023-12-14T18:17:05.215 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/161 2023-12-14T18:17:05.407 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/161 2023-12-14T18:17:05.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/161 2023-12-14T18:17:05.676 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2023-12-14T18:17:05.699 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/161 2023-12-14T18:17:05.823 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 31/161 2023-12-14T18:17:05.909 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/161 2023-12-14T18:17:05.952 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 32/161 2023-12-14T18:17:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/161 2023-12-14T18:17:06.123 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/161 2023-12-14T18:17:06.229 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 33/161 2023-12-14T18:17:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/161 2023-12-14T18:17:06.375 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/161 2023-12-14T18:17:06.420 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/161 2023-12-14T18:17:06.531 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2023-12-14T18:17:06.626 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/161 2023-12-14T18:17:06.692 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2023-12-14T18:17:06.727 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2023-12-14T18:17:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/161 2023-12-14T18:17:06.994 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2023-12-14T18:17:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/161 2023-12-14T18:17:07.222 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2023-12-14T18:17:07.437 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2023-12-14T18:17:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2023-12-14T18:17:07.593 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2023-12-14T18:17:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 31/161 2023-12-14T18:17:07.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2023-12-14T18:17:07.833 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 32/161 2023-12-14T18:17:08.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 33/161 2023-12-14T18:17:08.083 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2023-12-14T18:17:08.293 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2023-12-14T18:17:08.435 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2023-12-14T18:17:08.493 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2023-12-14T18:17:08.646 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2023-12-14T18:17:08.768 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 35/161 2023-12-14T18:17:08.894 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2023-12-14T18:17:09.152 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2023-12-14T18:17:09.172 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/161 2023-12-14T18:17:09.312 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2023-12-14T18:17:09.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2023-12-14T18:17:09.670 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/161 2023-12-14T18:17:09.734 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/161 2023-12-14T18:17:09.847 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2023-12-14T18:17:09.986 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/161 2023-12-14T18:17:10.031 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2023-12-14T18:17:10.046 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/161 2023-12-14T18:17:10.272 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2023-12-14T18:17:10.275 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/161 2023-12-14T18:17:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/161 2023-12-14T18:17:10.494 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-12-14T18:17:10.744 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2023-12-14T18:17:10.957 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 41/161 2023-12-14T18:17:11.183 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 42/161 2023-12-14T18:17:11.391 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:17:11.401 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2023-12-14T18:17:11.472 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:17:11.691 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:17:11.736 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2023-12-14T18:17:11.746 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:17:11.955 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-12-14T18:17:11.987 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2023-12-14T18:17:12.186 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2023-12-14T18:17:12.248 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2023-12-14T18:17:12.301 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/161 2023-12-14T18:17:12.408 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:17:12.494 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2023-12-14T18:17:12.723 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2023-12-14T18:17:12.919 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:17:12.952 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 38/161 2023-12-14T18:17:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-12-14T18:17:13.224 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:17:13.319 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:17:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2023-12-14T18:17:13.501 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2023-12-14T18:17:13.542 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : automake-1.16.1-8.el8.noarch 49/161 2023-12-14T18:17:13.706 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:17:13.707 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 41/161 2023-12-14T18:17:13.749 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : uuid-1.6.2-43.el8.x86_64 50/161 2023-12-14T18:17:13.777 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:17:13.829 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 50/161 2023-12-14T18:17:13.897 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 42/161 2023-12-14T18:17:13.982 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 103/161 2023-12-14T18:17:14.007 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/161 2023-12-14T18:17:14.087 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/161 2023-12-14T18:17:14.126 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:17:14.194 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:17:14.307 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : jq-1.6-7.el8.x86_64 52/161 2023-12-14T18:17:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:17:14.464 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/161 2023-12-14T18:17:14.494 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:17:14.615 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/161 2023-12-14T18:17:14.754 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2023-12-14T18:17:14.867 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/161 2023-12-14T18:17:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2023-12-14T18:17:15.078 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/161 2023-12-14T18:17:15.118 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:17:15.315 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/161 2023-12-14T18:17:15.496 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/161 2023-12-14T18:17:15.688 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:17:15.710 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/161 2023-12-14T18:17:15.903 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/161 2023-12-14T18:17:15.994 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:17:16.015 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:17:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/161 2023-12-14T18:17:16.245 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/161 2023-12-14T18:17:16.470 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/161 2023-12-14T18:17:16.578 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/161 2023-12-14T18:17:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/161 2023-12-14T18:17:16.803 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/161 2023-12-14T18:17:16.856 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/161 2023-12-14T18:17:16.994 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/161 2023-12-14T18:17:17.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/161 2023-12-14T18:17:17.217 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/161 2023-12-14T18:17:17.387 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/161 2023-12-14T18:17:17.597 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/161 2023-12-14T18:17:17.697 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/161 2023-12-14T18:17:17.900 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 67/161 2023-12-14T18:17:17.924 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/161 2023-12-14T18:17:18.077 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/161 2023-12-14T18:17:18.104 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/161 2023-12-14T18:17:18.245 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:18.318 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/161 2023-12-14T18:17:18.370 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/161 2023-12-14T18:17:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:18.570 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 70/161 2023-12-14T18:17:18.769 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/161 2023-12-14T18:17:18.938 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/161 2023-12-14T18:17:19.083 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/161 2023-12-14T18:17:19.156 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/161 2023-12-14T18:17:19.333 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/161 2023-12-14T18:17:19.373 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:17:19.401 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/161 2023-12-14T18:17:19.469 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:17:19.597 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/161 2023-12-14T18:17:19.617 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/161 2023-12-14T18:17:19.840 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/161 2023-12-14T18:17:19.894 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/161 2023-12-14T18:17:20.060 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/161 2023-12-14T18:17:20.256 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2023-12-14T18:17:20.271 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/161 2023-12-14T18:17:20.419 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2023-12-14T18:17:20.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/161 2023-12-14T18:17:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/161 2023-12-14T18:17:20.719 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2023-12-14T18:17:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/161 2023-12-14T18:17:20.938 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2023-12-14T18:17:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/161 2023-12-14T18:17:21.169 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2023-12-14T18:17:21.318 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2023-12-14T18:17:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/161 2023-12-14T18:17:21.508 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2023-12-14T18:17:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.1-8.el8.noarch 70/161 2023-12-14T18:17:21.733 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2023-12-14T18:17:21.858 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-43.el8.x86_64 71/161 2023-12-14T18:17:21.943 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2023-12-14T18:17:21.949 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 71/161 2023-12-14T18:17:22.151 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2023-12-14T18:17:22.158 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 72/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 72/161 2023-12-14T18:17:22.320 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2023-12-14T18:17:22.443 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-7.el8.x86_64 73/161 2023-12-14T18:17:22.569 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2023-12-14T18:17:22.658 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/161 2023-12-14T18:17:22.785 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2023-12-14T18:17:22.859 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/161 2023-12-14T18:17:23.003 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2023-12-14T18:17:23.094 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/161 2023-12-14T18:17:23.244 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2023-12-14T18:17:23.301 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2023-12-14T18:17:23.480 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2023-12-14T18:17:23.499 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2023-12-14T18:17:23.663 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2023-12-14T18:17:23.806 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2023-12-14T18:17:23.913 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2023-12-14T18:17:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2023-12-14T18:17:24.255 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2023-12-14T18:17:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2023-12-14T18:17:24.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2023-12-14T18:17:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2023-12-14T18:17:24.999 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2023-12-14T18:17:25.188 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2023-12-14T18:17:25.343 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2023-12-14T18:17:25.420 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2023-12-14T18:17:25.538 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-12-14T18:17:25.581 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2023-12-14T18:17:25.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2023-12-14T18:17:25.847 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2023-12-14T18:17:26.072 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2023-12-14T18:17:26.140 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2023-12-14T18:17:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2023-12-14T18:17:26.350 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2023-12-14T18:17:26.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2023-12-14T18:17:26.759 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2023-12-14T18:17:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2023-12-14T18:17:27.140 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2023-12-14T18:17:27.212 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2023-12-14T18:17:27.313 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:17:27.453 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:17:27.789 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 103/161 2023-12-14T18:17:28.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2023-12-14T18:17:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2023-12-14T18:17:29.146 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-12-14T18:17:29.514 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2023-12-14T18:17:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2023-12-14T18:17:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2023-12-14T18:17:30.842 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2023-12-14T18:17:31.048 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:17:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:17:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 103/161 2023-12-14T18:17:31.566 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:31.813 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:32.407 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:33.375 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:17:33.518 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:17:33.679 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:35.471 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:35.682 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:17:36.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:17:36.722 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:17:36.864 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:38.993 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:38.993 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2023-12-14T18:17:38.993 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2023-12-14T18:17:38.993 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /dev 2023-12-14T18:17:38.993 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /run 2023-12-14T18:17:38.993 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2023-12-14T18:17:38.994 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2023-12-14T18:17:38.994 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2023-12-14T18:17:38.994 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2023-12-14T18:17:38.994 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /dev 2023-12-14T18:17:38.994 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:17:39.353 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:40.926 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:41.144 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:41.364 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:41.959 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:42.119 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:42.137 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:43.951 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:44.138 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:44.258 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 112/161 2023-12-14T18:17:44.485 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 113/161 2023-12-14T18:17:44.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:44.943 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:45.252 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:46.221 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:46.368 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:46.461 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:46.822 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 117/161 2023-12-14T18:17:47.002 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:47.039 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:47.160 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:47.172 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:48.451 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 120/161 2023-12-14T18:17:48.694 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:48.728 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:48.927 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:48.969 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:49.143 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:49.491 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:49.685 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:52.786 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /sys 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /proc 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /dev 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /run 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /mnt 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /var/tmp 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /home 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /tmp 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /dev 2023-12-14T18:17:52.787 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:17:53.176 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:53.624 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:54.546 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 125/161 2023-12-14T18:17:54.752 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:54.855 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 126/161 2023-12-14T18:17:54.997 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:55.149 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2023-12-14T18:17:55.216 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:55.434 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:55.565 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 128/161 2023-12-14T18:17:55.890 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:55.904 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 129/161 2023-12-14T18:17:56.059 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:56.077 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:56.125 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-25.el8.x86_64 130/161 2023-12-14T18:17:56.139 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 130/161 2023-12-14T18:17:56.378 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2023-12-14T18:17:56.588 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2023-12-14T18:17:56.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 133/161 2023-12-14T18:17:57.782 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:57.923 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:58.040 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 112/161 2023-12-14T18:17:58.223 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 113/161 2023-12-14T18:17:58.433 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:58.653 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /run 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2023-12-14T18:17:58.829 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-12-14T18:17:58.830 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:17:58.954 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:59.177 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:59.509 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:59.550 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:59.921 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:18:00.027 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2023-12-14T18:18:00.117 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:18:00.210 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:18:00.414 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2023-12-14T18:18:00.599 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 117/161 2023-12-14T18:18:00.654 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 137/161 2023-12-14T18:18:00.796 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:18:00.832 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:18:00.850 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 138/161 2023-12-14T18:18:00.983 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:18:00.987 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:18:00.995 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:18:01.050 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 139/161 2023-12-14T18:18:01.165 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:18:01.356 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 140/161 2023-12-14T18:18:01.410 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:18:01.603 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:18:01.684 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 141/161 2023-12-14T18:18:01.924 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.1-9.el8.x86_64 142/161 2023-12-14T18:18:02.010 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 142/161 2023-12-14T18:18:02.111 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:18:02.225 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.0.4-10.el8.x86_64 143/161 2023-12-14T18:18:02.238 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 143/161 2023-12-14T18:18:02.243 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 120/161 2023-12-14T18:18:02.261 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:18:02.282 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:18:02.382 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 144/161 2023-12-14T18:18:02.392 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 144/161 2023-12-14T18:18:02.479 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:18:02.513 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:18:02.518 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/161 2023-12-14T18:18:02.697 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2023-12-14T18:18:02.708 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:18:02.751 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:18:02.902 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2023-12-14T18:18:02.908 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:18:03.063 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2023-12-14T18:18:03.063 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:18:03.254 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:18:03.328 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:18:03.328 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:18:03.433 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:18:03.578 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:18:03.578 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:18:03.831 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:18:04.060 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2023-12-14T18:18:04.221 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2023-12-14T18:18:04.306 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:18:04.460 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2023-12-14T18:18:04.460 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:18:04.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:18:04.626 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 112/161 2023-12-14T18:18:04.719 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:18:04.719 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:18:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 113/161 2023-12-14T18:18:04.977 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:18:04.977 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:18:05.211 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:05.211 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:18:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:18:05.477 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:18:05.477 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:18:05.693 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:05.844 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2023-12-14T18:18:05.844 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:18:06.721 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:18:06.751 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 1/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 3/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 4/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 5/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 6/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 7/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 8/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:18:06.752 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 13/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 14/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 15/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 16/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 17/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 18/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 20/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 21/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 22/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 23/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 24/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 25/161 2023-12-14T18:18:06.753 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 26/161 2023-12-14T18:18:06.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 27/161 2023-12-14T18:18:06.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 28/161 2023-12-14T18:18:06.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 29/161 2023-12-14T18:18:06.754 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 30/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 31/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 32/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 33/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 34/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 35/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 36/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 37/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 38/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2023-12-14T18:18:06.765 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2023-12-14T18:18:06.766 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2023-12-14T18:18:06.767 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/161 2023-12-14T18:18:06.768 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 87/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 93/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 94/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 95/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 96/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 98/161 2023-12-14T18:18:06.769 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 113/161 2023-12-14T18:18:06.770 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 114/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 115/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 116/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 117/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 118/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 119/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 120/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 121/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 122/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 123/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 124/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-10.23-2.module+el8.8.0+19493+5 125/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.1-8.el8.noarch 126/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpq-13.11-1.el8.x86_64 127/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-7.el8.x86_64 128/161 2023-12-14T18:18:06.771 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 129/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 132/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 134/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 135/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 136/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 138/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 139/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 140/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 141/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 143/161 2023-12-14T18:18:06.772 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 145/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 147/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 149/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 151/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 154/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 155/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 156/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 158/161 2023-12-14T18:18:06.773 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:06.774 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-12-14T18:18:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:18:07.011 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:18:07.337 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:18:07.408 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 117/161 2023-12-14T18:18:07.606 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:18:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:18:07.826 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:18:07.841 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:18:08.277 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 125/161 2023-12-14T18:18:08.578 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 126/161 2023-12-14T18:18:08.838 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2023-12-14T18:18:09.228 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 120/161 2023-12-14T18:18:09.257 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 128/161 2023-12-14T18:18:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:18:09.488 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:18:09.617 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 129/161 2023-12-14T18:18:09.684 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:18:09.735 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:18:09.840 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libtool-2.4.6-25.el8.x86_64 130/161 2023-12-14T18:18:09.853 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 130/161 2023-12-14T18:18:09.926 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:18:10.084 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2023-12-14T18:18:10.261 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2023-12-14T18:18:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:18:10.441 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 133/161 2023-12-14T18:18:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:18:12.722 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:18:13.006 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:18:13.057 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:18:13.506 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2023-12-14T18:18:13.838 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2023-12-14T18:18:14.102 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 137/161 2023-12-14T18:18:14.265 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 138/161 2023-12-14T18:18:14.489 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 139/161 2023-12-14T18:18:14.767 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 140/161 2023-12-14T18:18:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:18:15.115 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 141/161 2023-12-14T18:18:15.333 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flex-2.6.1-9.el8.x86_64 142/161 2023-12-14T18:18:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 125/161 2023-12-14T18:18:15.421 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 142/161 2023-12-14T18:18:15.622 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : bison-3.0.4-10.el8.x86_64 143/161 2023-12-14T18:18:15.636 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 143/161 2023-12-14T18:18:15.753 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 126/161 2023-12-14T18:18:15.830 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 144/161 2023-12-14T18:18:15.839 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 144/161 2023-12-14T18:18:15.983 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/161 2023-12-14T18:18:15.991 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-25.el8.x86_64 127/161 2023-12-14T18:18:16.189 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2023-12-14T18:18:16.407 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2023-12-14T18:18:16.576 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2023-12-14T18:18:16.577 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:18:16.817 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 127/161 2023-12-14T18:18:16.909 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:18:16.909 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:18:17.110 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 128/161 2023-12-14T18:18:17.193 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:18:17.193 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:18:17.390 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/161 2023-12-14T18:18:17.428 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:18:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:18:17.665 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2023-12-14T18:18:17.880 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2023-12-14T18:18:17.883 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2023-12-14T18:18:17.896 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-12-14T18:18:17.896 INFO:teuthology.orchestra.run.smithi204.stdout:Installed products updated. 2023-12-14T18:18:18.122 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2023-12-14T18:18:18.141 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2023-12-14T18:18:18.141 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:18:18.343 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 133/161 2023-12-14T18:18:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:18:18.383 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.484 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.1-8.el8.noarch 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.0.4-10.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.485 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.486 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.1-9.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-7.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.487 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libpq-13.11-1.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-14T18:18:18.488 INFO:teuthology.orchestra.run.smithi204.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-14T18:18:18.489 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-14T18:18:18.490 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-2.6-1.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-14T18:18:18.491 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-14T18:18:18.492 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-14T18:18:18.493 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-14T18:18:18.494 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-14T18:18:18.494 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:18:18.494 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2023-12-14T18:18:18.641 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:18:18.641 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:18.697 DEBUG:teuthology.parallel:result is None 2023-12-14T18:18:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:18.883 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:18:19.149 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:18:19.149 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:19.408 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:19.574 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2023-12-14T18:18:19.574 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:18:20.527 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:18:20.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 1/161 2023-12-14T18:18:20.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2/161 2023-12-14T18:18:20.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 3/161 2023-12-14T18:18:20.527 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 4/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 5/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 6/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 7/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 8/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 13/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 14/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 15/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 16/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 17/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 18/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:18:20.528 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 20/161 2023-12-14T18:18:20.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 21/161 2023-12-14T18:18:20.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 22/161 2023-12-14T18:18:20.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 23/161 2023-12-14T18:18:20.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 24/161 2023-12-14T18:18:20.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 25/161 2023-12-14T18:18:20.529 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 26/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 27/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 28/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 29/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 30/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 31/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 32/161 2023-12-14T18:18:20.540 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 33/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 34/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 35/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 36/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 37/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 38/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2023-12-14T18:18:20.541 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2023-12-14T18:18:20.542 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/161 2023-12-14T18:18:20.543 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 87/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/161 2023-12-14T18:18:20.544 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 93/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 94/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 95/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 96/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 98/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/161 2023-12-14T18:18:20.545 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 113/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 114/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 115/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 116/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 117/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 118/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 119/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 120/161 2023-12-14T18:18:20.546 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 121/161 2023-12-14T18:18:20.547 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 122/161 2023-12-14T18:18:20.548 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 123/161 2023-12-14T18:18:20.548 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 124/161 2023-12-14T18:18:20.548 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : postgresql-server-10.23-2.module+el8.8.0+19493+5 125/161 2023-12-14T18:18:20.548 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : automake-1.16.1-8.el8.noarch 126/161 2023-12-14T18:18:20.548 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libpq-13.11-1.el8.x86_64 127/161 2023-12-14T18:18:20.548 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : jq-1.6-7.el8.x86_64 128/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 129/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 132/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 134/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 135/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 136/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 138/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 139/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 140/161 2023-12-14T18:18:20.549 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 141/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 143/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 145/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 147/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 149/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 151/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 154/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 155/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 156/161 2023-12-14T18:18:20.550 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:20.551 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 158/161 2023-12-14T18:18:20.551 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:20.551 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-12-14T18:18:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:18:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:18:21.222 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:18:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2023-12-14T18:18:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2023-12-14T18:18:22.354 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 137/161 2023-12-14T18:18:22.558 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 138/161 2023-12-14T18:18:22.758 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 139/161 2023-12-14T18:18:23.064 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 140/161 2023-12-14T18:18:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 141/161 2023-12-14T18:18:23.539 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 142/161 2023-12-14T18:18:23.551 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 142/161 2023-12-14T18:18:23.784 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 143/161 2023-12-14T18:18:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/161 2023-12-14T18:18:24.119 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/161 2023-12-14T18:18:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/161 2023-12-14T18:18:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/161 2023-12-14T18:18:24.534 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2023-12-14T18:18:24.734 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2023-12-14T18:18:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2023-12-14T18:18:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:18:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:18:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:18:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:18:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:18:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:18:25.950 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2023-12-14T18:18:26.164 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2023-12-14T18:18:26.442 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2023-12-14T18:18:26.442 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:18:26.726 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:18:26.726 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:18:27.001 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:18:27.001 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:27.234 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:27.234 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:18:27.459 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:18:27.459 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:27.717 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:27.909 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2023-12-14T18:18:27.910 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:18:28.894 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:18:28.894 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 1/161 2023-12-14T18:18:28.894 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 3/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 4/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 5/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 6/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 7/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 8/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 13/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 14/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 15/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 16/161 2023-12-14T18:18:28.895 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 17/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 18/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 20/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 21/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 22/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 23/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 24/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 25/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 26/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 27/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 28/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 29/161 2023-12-14T18:18:28.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 30/161 2023-12-14T18:18:28.907 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 31/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 32/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 33/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 34/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 35/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 36/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 37/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 38/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2023-12-14T18:18:28.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2023-12-14T18:18:28.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/161 2023-12-14T18:18:28.910 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 86/161 2023-12-14T18:18:28.911 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 87/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 88/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 89/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 90/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 91/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 92/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 93/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 94/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 95/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 96/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 97/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-10.23-2.module+el8.8.0+19493+5 98/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.1-8.el8.noarch 99/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpq-13.11-1.el8.x86_64 100/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-7.el8.x86_64 101/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 102/161 2023-12-14T18:18:28.912 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/161 2023-12-14T18:18:28.913 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 118/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 124/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 125/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 126/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 128/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 129/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2023-12-14T18:18:28.914 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 132/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 134/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 135/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 136/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 138/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 139/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 140/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 141/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 143/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 145/161 2023-12-14T18:18:28.915 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 147/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 149/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 151/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 154/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 155/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 156/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 158/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:18:28.916 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-12-14T18:18:37.206 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-12-14T18:18:37.206 INFO:teuthology.orchestra.run.smithi176.stdout:Installed products updated. 2023-12-14T18:18:37.780 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.781 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: automake-1.16.1-8.el8.noarch 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: bison-3.0.4-10.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.782 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.783 INFO:teuthology.orchestra.run.smithi176.stdout: cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: flex-2.6.1-9.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: jq-1.6-7.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-14T18:18:37.784 INFO:teuthology.orchestra.run.smithi176.stdout: libpq-13.11-1.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-14T18:18:37.785 INFO:teuthology.orchestra.run.smithi176.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-contrib-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: postgresql-server-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-14T18:18:37.786 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-14T18:18:37.787 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend-2.6-1.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-14T18:18:37.788 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-14T18:18:37.789 INFO:teuthology.orchestra.run.smithi176.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-14T18:18:37.790 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-14T18:18:37.791 INFO:teuthology.orchestra.run.smithi176.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-14T18:18:37.791 INFO:teuthology.orchestra.run.smithi176.stdout: 2023-12-14T18:18:37.791 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2023-12-14T18:18:38.001 DEBUG:teuthology.parallel:result is None 2023-12-14T18:18:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-12-14T18:18:42.522 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.190 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.1-8.el8.noarch 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.0.4-10.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.191 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.192 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.1-9.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-7.el8.x86_64 2023-12-14T18:18:43.193 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libpq-13.11-1.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-14T18:18:43.194 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-14T18:18:43.195 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-14T18:18:43.196 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-14T18:18:43.197 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-2.6-1.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-14T18:18:43.198 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.199 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:18:43.200 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-12-14T18:18:43.455 DEBUG:teuthology.parallel:result is None 2023-12-14T18:18:43.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:18:43.595 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-14T18:18:43.623 INFO:teuthology.orchestra.run.smithi031.stdout:18.0.0-3905.g1faa3533.el8 2023-12-14T18:18:43.624 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g1faa3533.el8 2023-12-14T18:18:43.624 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g1faa3533 is installed. 2023-12-14T18:18:43.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:18:43.840 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-14T18:18:43.868 INFO:teuthology.orchestra.run.smithi176.stdout:18.0.0-3905.g1faa3533.el8 2023-12-14T18:18:43.868 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g1faa3533.el8 2023-12-14T18:18:43.868 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g1faa3533 is installed. 2023-12-14T18:18:43.870 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:18:44.083 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-14T18:18:44.111 INFO:teuthology.orchestra.run.smithi204.stdout:18.0.0-3905.g1faa3533.el8 2023-12-14T18:18:44.111 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g1faa3533.el8 2023-12-14T18:18:44.111 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g1faa3533 is installed. 2023-12-14T18:18:44.113 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-14T18:18:44.113 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:18:44.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-14T18:18:44.165 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:18:44.165 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-14T18:18:44.213 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:18:44.213 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-14T18:18:44.260 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-14T18:18:44.261 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:18:44.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2023-12-14T18:18:44.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-14T18:18:44.370 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:18:44.371 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2023-12-14T18:18:44.402 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-14T18:18:44.473 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:18:44.473 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2023-12-14T18:18:44.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-14T18:18:44.577 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-14T18:18:44.577 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:18:44.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-14T18:18:44.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-14T18:18:44.685 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:18:44.686 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-14T18:18:44.716 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-14T18:18:44.786 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:18:44.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-14T18:18:44.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-14T18:18:44.889 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-14T18:18:44.889 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:18:44.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2023-12-14T18:18:44.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-14T18:18:45.000 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:18:45.000 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2023-12-14T18:18:45.028 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-14T18:18:45.096 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:18:45.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2023-12-14T18:18:45.128 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-14T18:18:45.199 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-14T18:18:45.265 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-12-14T18:18:45.266 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:18:45.266 INFO:tasks.cephadm:Cluster fsid is 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:45.266 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-14T18:18:45.266 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.31', 'mon.b': '172.21.15.176', 'mon.c': '172.21.15.204'} 2023-12-14T18:18:45.266 INFO:tasks.cephadm:First mon is mon.a on smithi031 2023-12-14T18:18:45.266 INFO:tasks.cephadm:First mgr is x 2023-12-14T18:18:45.266 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-14T18:18:45.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2023-12-14T18:18:45.303 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname $(hostname -s) 2023-12-14T18:18:45.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2023-12-14T18:18:45.366 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-14T18:18:45.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:18:45.582 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74952/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3905-g1faa3533', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3905.g1faa3533'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-13 17:15:35.607942', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-12-14T18:18:45.720 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-choffman-62793, sha1 1faa353368ad4f5aeb56d7107b638336f58659ed from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:18:45.721 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm 2023-12-14T18:18:45.722 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:46.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Dec 14 18:18 /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:46.259 DEBUG:teuthology.orchestra.run.smithi176:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:46.898 INFO:teuthology.orchestra.run.smithi176.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Dec 14 18:18 /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:46.899 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:47.273 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Dec 14 18:18 /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:47.274 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:47.297 DEBUG:teuthology.orchestra.run.smithi176:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:47.319 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:47.354 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed on all hosts... 2023-12-14T18:18:47.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed pull 2023-12-14T18:18:47.357 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed pull 2023-12-14T18:18:47.361 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed pull 2023-12-14T18:18:48.722 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:48.966 INFO:teuthology.orchestra.run.smithi176.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:49.599 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:19:32.485 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2023-12-14T18:19:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:19:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "image_id": "0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0", 2023-12-14T18:19:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "repo_digests": [ 2023-12-14T18:19:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4f00bb7c6b97266b68e12e45f4e426e52b86010cb97d3e773903241deee4098b" 2023-12-14T18:19:32.485 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2023-12-14T18:19:32.486 INFO:teuthology.orchestra.run.smithi176.stdout:} 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0", 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4f00bb7c6b97266b68e12e45f4e426e52b86010cb97d3e773903241deee4098b" 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:19:32.674 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0", 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4f00bb7c6b97266b68e12e45f4e426e52b86010cb97d3e773903241deee4098b" 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2023-12-14T18:19:48.060 INFO:teuthology.orchestra.run.smithi204.stdout:} 2023-12-14T18:19:48.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2023-12-14T18:19:48.120 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph 2023-12-14T18:19:48.158 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2023-12-14T18:19:48.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2023-12-14T18:19:48.222 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /etc/ceph 2023-12-14T18:19:48.253 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2023-12-14T18:19:48.279 INFO:tasks.cephadm:Writing seed config... 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-12-14T18:19:48.280 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-14T18:19:48.281 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-14T18:19:48.282 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-14T18:19:48.283 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-14T18:19:48.283 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-14T18:19:48.283 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-14T18:19:48.283 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-14T18:19:48.283 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-14T18:19:48.283 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-12-14T18:19:48.283 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:19:48.284 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-14T18:19:48.305 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 37e25396-9aad-11ee-95a5-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-14T18:19:48.306 DEBUG:teuthology.orchestra.run.smithi031:mon.a> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a.service 2023-12-14T18:19:48.348 DEBUG:teuthology.orchestra.run.smithi031:mgr.x> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x.service 2023-12-14T18:19:48.382 INFO:journalctl@ceph.mon.a.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:19:48.383 INFO:tasks.cephadm:Bootstrapping... 2023-12-14T18:19:48.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed -v bootstrap --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:19:48.415 INFO:journalctl@ceph.mgr.x.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:19:48.670 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:19:48.671 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed', '-v', 'bootstrap', '--fsid', '37e25396-9aad-11ee-95a5-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2023-12-14T18:19:48.718 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.6.1 2023-12-14T18:19:48.718 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-14T18:19:48.719 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2023-12-14T18:19:48.758 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.6.1 2023-12-14T18:19:48.758 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2023-12-14T18:19:48.758 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2023-12-14T18:19:48.766 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-14T18:19:48.766 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-14T18:19:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-14T18:19:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-12-14T18:19:48.779 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2023-12-14T18:19:48.786 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2023-12-14T18:19:48.786 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2023-12-14T18:19:48.786 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2023-12-14T18:19:48.827 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.6.1 2023-12-14T18:19:48.827 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.6.1 is present 2023-12-14T18:19:48.827 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2023-12-14T18:19:48.827 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2023-12-14T18:19:48.834 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-14T18:19:48.834 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-14T18:19:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-14T18:19:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-12-14T18:19:48.846 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2023-12-14T18:19:48.853 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2023-12-14T18:19:48.853 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2023-12-14T18:19:48.853 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2023-12-14T18:19:48.853 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:48.854 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 140706342648296 on /run/cephadm/37e25396-9aad-11ee-95a5-87774f69a715.lock 2023-12-14T18:19:48.854 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 140706342648296 acquired on /run/cephadm/37e25396-9aad-11ee-95a5-87774f69a715.lock 2023-12-14T18:19:48.854 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2023-12-14T18:19:48.854 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2023-12-14T18:19:48.854 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2023-12-14T18:19:48.860 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-12-14T18:19:48.860 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2023-12-14T18:19:48.865 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1609sec hoplimit 64 pref medium 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-12-14T18:19:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-12-14T18:19:48.867 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-14T18:19:48.867 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2023-12-14T18:19:48.867 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2023-12-14T18:19:48.867 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-14T18:19:48.867 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-14T18:19:48.868 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:a83bc5fcf8de435b160e58e3bb5830aa8038f139ef00a293340a26641aa5a474 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0 2023-12-14T18:19:49.240 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-14T18:19:49.992 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev) 2023-12-14T18:19:49.992 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev) 2023-12-14T18:19:49.992 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2023-12-14T18:19:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2023-12-14T18:19:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2023-12-14T18:19:51.185 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBGR3tld/SsMhAAXEaQxikdSwmSOmDvIk7E/A== 2023-12-14T18:19:51.816 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBHR3tlilpOGxAAKS2RFTRI6hjUoSfiehmq1w== 2023-12-14T18:19:52.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQBIR3tl2PEJBBAAb9CT1JjucaBVSuUiPTVsDA== 2023-12-14T18:19:52.414 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for a [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:19:53.035 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2023-12-14T18:19:53.732 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.a on 2023-12-14T18:19:53.853 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-14T18:19:53.952 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-14T18:19:54.065 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-37e25396-9aad-11ee-95a5-87774f69a715.target → /etc/systemd/system/ceph-37e25396-9aad-11ee-95a5-87774f69a715.target. 2023-12-14T18:19:54.065 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-37e25396-9aad-11ee-95a5-87774f69a715.target → /etc/systemd/system/ceph-37e25396-9aad-11ee-95a5-87774f69a715.target. 2023-12-14T18:19:54.220 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a 2023-12-14T18:19:54.221 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a.service: Unit ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a.service not loaded. 2023-12-14T18:19:54.323 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-37e25396-9aad-11ee-95a5-87774f69a715.target.wants/ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a.service → /etc/systemd/system/ceph-37e25396-9aad-11ee-95a5-87774f69a715@.service. 2023-12-14T18:19:54.704 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:19:54.704 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-12-14T18:19:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:19:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-12-14T18:19:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-12-14T18:19:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2023-12-14T18:19:54.714 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2023-12-14T18:19:54.714 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2023-12-14T18:19:54.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:54 smithi031 ceph-mon[41602]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-14T18:19:55.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.45221s) 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-14T18:19:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.101+0000 7f3105923700 1 Processor -- start 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.101+0000 7f3105923700 1 -- start start 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.101+0000 7f3105923700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f31001088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.101+0000 7f3105923700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100079e00 con 0x7f31001064f0 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f31001088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f31001088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47246/0 (socket says 172.21.15.31:47246) 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30feffd700 1 -- 172.21.15.31:0/269199112 learned_addr learned my addr 172.21.15.31:0/269199112 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30feffd700 1 -- 172.21.15.31:0/269199112 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3100108e10 con 0x7f31001064f0 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30feffd700 1 --2- 172.21.15.31:0/269199112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f31001088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f30f00098a0 tx=0x7f30f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f932083ea4e347 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30fdffb700 1 -- 172.21.15.31:0/269199112 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30f0004160 con 0x7f31001064f0 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30fdffb700 1 -- 172.21.15.31:0/269199112 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f00042c0 con 0x7f31001064f0 2023-12-14T18:19:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.102+0000 7f30fdffb700 1 -- 172.21.15.31:0/269199112 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30f00045a0 con 0x7f31001064f0 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 -- 172.21.15.31:0/269199112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 msgr2=0x7f31001088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 --2- 172.21.15.31:0/269199112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f31001088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f30f00098a0 tx=0x7f30f0009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 -- 172.21.15.31:0/269199112 shutdown_connections 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 --2- 172.21.15.31:0/269199112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f31001088d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 -- 172.21.15.31:0/269199112 >> 172.21.15.31:0/269199112 conn(0x7f31000ffed0 msgr2=0x7f3100102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 -- 172.21.15.31:0/269199112 shutdown_connections 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.103+0000 7f3105923700 1 -- 172.21.15.31:0/269199112 wait complete. 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.104+0000 7f3105923700 1 Processor -- start 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.104+0000 7f3105923700 1 -- start start 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.104+0000 7f3105923700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f3100105e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.104+0000 7f3105923700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100106390 con 0x7f31001064f0 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.104+0000 7f30feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f3100105e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f30feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f3100105e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47262/0 (socket says 172.21.15.31:47262) 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f30feffd700 1 -- 172.21.15.31:0/2054989247 learned_addr learned my addr 172.21.15.31:0/2054989247 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f30feffd700 1 -- 172.21.15.31:0/2054989247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0009550 con 0x7f31001064f0 2023-12-14T18:19:55.507 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f30feffd700 1 --2- 172.21.15.31:0/2054989247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f3100105e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3100079240 tx=0x7f30f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30f0011630 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0011790 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30f0011a30 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31001091f0 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3100109060 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f30f0022070 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.105+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f30f001c790 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.106+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3100062120 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.109+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f30f001bcf0 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.187+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3100192c20 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.188+0000 7f3104921700 1 -- 172.21.15.31:0/2054989247 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7f30f001b450 con 0x7f31001064f0 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.188+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 msgr2=0x7f3100105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.188+0000 7f3105923700 1 --2- 172.21.15.31:0/2054989247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f3100105e50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3100079240 tx=0x7f30f0004750 comp rx=0 tx=0).stop 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.189+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 shutdown_connections 2023-12-14T18:19:55.508 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.189+0000 7f3105923700 1 --2- 172.21.15.31:0/2054989247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31001064f0 0x7f3100105e50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:55.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.189+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 >> 172.21.15.31:0/2054989247 conn(0x7f31000ffed0 msgr2=0x7f3100101870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:55.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.189+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 shutdown_connections 2023-12-14T18:19:55.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.189+0000 7f3105923700 1 -- 172.21.15.31:0/2054989247 wait complete. 2023-12-14T18:19:55.509 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2023-12-14T18:19:55.509 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2023-12-14T18:19:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:55 smithi031 ceph-mon[41602]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-14T18:19:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:55 smithi031 ceph-mon[41602]: monmap e1: 1 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} 2023-12-14T18:19:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:55 smithi031 ceph-mon[41602]: fsmap 2023-12-14T18:19:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:55 smithi031 ceph-mon[41602]: osdmap e1: 0 total, 0 up, 0 in 2023-12-14T18:19:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:55 smithi031 ceph-mon[41602]: mgrmap e1: no daemons active 2023-12-14T18:19:56.086 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:55 smithi031 ceph-mon[41602]: from='client.? 172.21.15.31:0/2054989247' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-14T18:19:56.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:56.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2023-12-14T18:19:56.444 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-14T18:19:56.445 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6d069e5700 1 Processor -- start 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6d069e5700 1 -- start start 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6d069e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d001088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6d069e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d00079de0 con 0x7f6d00106510 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6cfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d001088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6cfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d001088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47268/0 (socket says 172.21.15.31:47268) 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6cfffff700 1 -- 172.21.15.31:0/882744043 learned_addr learned my addr 172.21.15.31:0/882744043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.976+0000 7f6cfffff700 1 -- 172.21.15.31:0/882744043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d00108e30 con 0x7f6d00106510 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6cfffff700 1 --2- 172.21.15.31:0/882744043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d001088f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00098a0 tx=0x7f6cf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4733c7699403f0a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6cfeffd700 1 -- 172.21.15.31:0/882744043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cf0004160 con 0x7f6d00106510 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6cfeffd700 1 -- 172.21.15.31:0/882744043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf00042c0 con 0x7f6d00106510 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6cfeffd700 1 -- 172.21.15.31:0/882744043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cf00045a0 con 0x7f6d00106510 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 -- 172.21.15.31:0/882744043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 msgr2=0x7f6d001088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 --2- 172.21.15.31:0/882744043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d001088f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6cf00098a0 tx=0x7f6cf0009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 -- 172.21.15.31:0/882744043 shutdown_connections 2023-12-14T18:19:56.446 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 --2- 172.21.15.31:0/882744043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d001088f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 -- 172.21.15.31:0/882744043 >> 172.21.15.31:0/882744043 conn(0x7f6d000ffed0 msgr2=0x7f6d00102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 -- 172.21.15.31:0/882744043 shutdown_connections 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.977+0000 7f6d069e5700 1 -- 172.21.15.31:0/882744043 wait complete. 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.978+0000 7f6d069e5700 1 Processor -- start 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.978+0000 7f6d069e5700 1 -- start start 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.978+0000 7f6d069e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d00198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.978+0000 7f6d069e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d00199470 con 0x7f6d00106510 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d00198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d00198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47284/0 (socket says 172.21.15.31:47284) 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfffff700 1 -- 172.21.15.31:0/486004262 learned_addr learned my addr 172.21.15.31:0/486004262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfffff700 1 -- 172.21.15.31:0/486004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf0009550 con 0x7f6d00106510 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfffff700 1 --2- 172.21.15.31:0/486004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d00198f30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6d00079220 tx=0x7f6cf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cf0011630 con 0x7f6d00106510 2023-12-14T18:19:56.447 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf0011790 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6d069e5700 1 -- 172.21.15.31:0/486004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d00109210 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cf0011a30 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.979+0000 7f6d069e5700 1 -- 172.21.15.31:0/486004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d00109080 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.980+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6cf0022070 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.980+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6cf001c790 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.980+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce00052f0 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:55.983+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f6cf001bcf0 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.057+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6ce0002400 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.061+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cf001be90 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.061+0000 7f6cfd7fa700 1 -- 172.21.15.31:0/486004262 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f6cf00246e0 con 0x7f6d00106510 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 msgr2=0x7f6d00198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 --2- 172.21.15.31:0/486004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d00198f30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6d00079220 tx=0x7f6cf0004750 comp rx=0 tx=0).stop 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 shutdown_connections 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 --2- 172.21.15.31:0/486004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d00106510 0x7f6d00198f30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 >> 172.21.15.31:0/486004262 conn(0x7f6d000ffed0 msgr2=0x7f6d00101870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:56.448 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 shutdown_connections 2023-12-14T18:19:56.449 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.062+0000 7f6ce6ffd700 1 -- 172.21.15.31:0/486004262 wait complete. 2023-12-14T18:19:56.449 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2023-12-14T18:19:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:56 smithi031 ceph-mon[41602]: from='client.? 172.21.15.31:0/486004262' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-14T18:19:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:56 smithi031 ceph-mon[41602]: from='client.? 172.21.15.31:0/486004262' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-14T18:19:58.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:57 smithi031 ceph-mon[41602]: from='client.? 172.21.15.31:0/1907090812' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb103613700 1 Processor -- start 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb103613700 1 -- start start 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb103613700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc104120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb103613700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1046f0 con 0x7fb0fc103d10 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb1013af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb1013af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47288/0 (socket says 172.21.15.31:47288) 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.809+0000 7fb1013af700 1 -- 172.21.15.31:0/914828796 learned_addr learned my addr 172.21.15.31:0/914828796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:19:58.540 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.810+0000 7fb1013af700 1 -- 172.21.15.31:0/914828796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc106ca0 con 0x7fb0fc103d10 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.810+0000 7fb1013af700 1 --2- 172.21.15.31:0/914828796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc104120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80098a0 tx=0x7fb0f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607dd2fc475862 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.810+0000 7fb0f3fff700 1 -- 172.21.15.31:0/914828796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f8004160 con 0x7fb0fc103d10 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.810+0000 7fb0f3fff700 1 -- 172.21.15.31:0/914828796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0f80042c0 con 0x7fb0fc103d10 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.810+0000 7fb0f3fff700 1 -- 172.21.15.31:0/914828796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f8011a00 con 0x7fb0fc103d10 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 -- 172.21.15.31:0/914828796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 msgr2=0x7fb0fc104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 --2- 172.21.15.31:0/914828796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc104120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80098a0 tx=0x7fb0f8009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 -- 172.21.15.31:0/914828796 shutdown_connections 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 --2- 172.21.15.31:0/914828796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc104120 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 -- 172.21.15.31:0/914828796 >> 172.21.15.31:0/914828796 conn(0x7fb0fc0ff650 msgr2=0x7fb0fc101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 -- 172.21.15.31:0/914828796 shutdown_connections 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 -- 172.21.15.31:0/914828796 wait complete. 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.811+0000 7fb103613700 1 Processor -- start 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb103613700 1 -- start start 2023-12-14T18:19:58.541 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb103613700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb103613700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19c030 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb1013af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb1013af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47304/0 (socket says 172.21.15.31:47304) 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb1013af700 1 -- 172.21.15.31:0/1907090812 learned_addr learned my addr 172.21.15.31:0/1907090812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb1013af700 1 -- 172.21.15.31:0/1907090812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8009550 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb1013af700 1 --2- 172.21.15.31:0/1907090812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc19baf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009fb0 tx=0x7fb0f8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.812+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f8003930 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.813+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc109ed0 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.813+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0f8003a90 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.813+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb0f80195e0 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.813+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc10a060 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.813+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb0f8019bc0 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.813+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb0f80243f0 con 0x7fb0fc103d10 2023-12-14T18:19:58.542 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.814+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc107b10 con 0x7fb0fc103d10 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.816+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fb0fc107b10 con 0x7fb0fc103d10 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.892+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb0fc062120 con 0x7fb0fc103d10 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.892+0000 7fb0f27fc700 1 -- 172.21.15.31:0/1907090812 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fb0f8003c00 con 0x7fb0fc103d10 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 msgr2=0x7fb0fc19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 --2- 172.21.15.31:0/1907090812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc19baf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009fb0 tx=0x7fb0f8011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 shutdown_connections 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 --2- 172.21.15.31:0/1907090812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0fc103d10 0x7fb0fc19baf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 >> 172.21.15.31:0/1907090812 conn(0x7fb0fc0ff650 msgr2=0x7fb0fc1003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 shutdown_connections 2023-12-14T18:19:58.543 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:56.893+0000 7fb103613700 1 -- 172.21.15.31:0/1907090812 wait complete. 2023-12-14T18:19:58.544 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2023-12-14T18:19:58.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:58 smithi031 systemd[1]: Stopping Ceph mon.a for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:19:59.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a[41579]: 2023-12-14T18:19:59.016+0000 7fdab98d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:19:59.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a[41579]: 2023-12-14T18:19:59.016+0000 7fdab98d0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-14T18:20:00.000 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:19:59 smithi031 bash[41924]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a 2023-12-14T18:20:01.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:01 smithi031 systemd[1]: ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a.service: Succeeded. 2023-12-14T18:20:01.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:01 smithi031 systemd[1]: Stopped Ceph mon.a for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:01.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:01 smithi031 systemd[1]: Starting Ceph mon.a for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:20:02.585 INFO:teuthology.orchestra.run.smithi031.stdout:Setting mon public_network to 172.21.0.0/20 2023-12-14T18:20:02.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: set uid:gid to 167:167 (ceph:ceph) 2023-12-14T18:20:02.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev), process ceph-mon, pid 7 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: pidfile_write: ignore empty --pid-file 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: load: jerasure load: lrc 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: RocksDB version: 7.9.2 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Git sha 0 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Compile date 2023-12-13 16:19:48 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: DB SUMMARY 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: DB Session ID: LXAVDKSJTR7ZE9A4QZC4 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: CURRENT file: CURRENT 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: IDENTITY file: IDENTITY 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-14T18:20:02.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98713 ; 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.error_if_exists: 0 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.create_if_missing: 0 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.paranoid_checks: 1 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.env: 0x56053c073720 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.fs: PosixFileSystem 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.info_log: 0x56053ed833c0 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_file_opening_threads: 16 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.statistics: (nil) 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.use_fsync: 0 2023-12-14T18:20:02.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_log_file_size: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.keep_log_file_num: 1000 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.recycle_log_file_num: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.allow_fallocate: 1 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.allow_mmap_reads: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.allow_mmap_writes: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.use_direct_reads: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.create_missing_column_families: 0 2023-12-14T18:20:02.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.db_log_dir: 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.wal_dir: 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.advise_random_on_open: 1 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.db_write_buffer_size: 0 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.write_buffer_manager: 0x56053dff43c0 2023-12-14T18:20:02.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.rate_limiter: (nil) 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.wal_recovery_mode: 2 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.enable_thread_tracking: 0 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.enable_pipelined_write: 0 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.unordered_write: 0 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-14T18:20:02.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.row_cache: None 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.wal_filter: None 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.allow_ingest_behind: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.two_write_queues: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.manual_wal_flush: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.wal_compression: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.atomic_flush: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.log_readahead_size: 0 2023-12-14T18:20:02.971 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.best_efforts_recovery: 0 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.allow_data_in_errors: 0 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.db_host_id: __hostname__ 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_background_jobs: 2 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_background_compactions: -1 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_subcompactions: 1 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-14T18:20:02.972 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_total_wal_size: 0 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_open_files: -1 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bytes_per_sync: 0 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_readahead_size: 0 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_background_flushes: -1 2023-12-14T18:20:02.973 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Compression algorithms supported: 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kZSTD supported: 0 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kXpressCompression supported: 0 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kLZ4HCCompression supported: 1 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kZlibCompression supported: 1 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kSnappyCompression supported: 1 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kLZ4Compression supported: 1 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: kBZip2Compression supported: 0 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-14T18:20:02.974 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.merge_operator: 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_filter: None 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_filter_factory: None 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.sst_partitioner_factory: None 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56053ed834e0) 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks: 1 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout: pin_top_level_index_and_filter: 1 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_type: 0 2023-12-14T18:20:02.975 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_index_type: 0 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_shortening: 1 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: checksum: 4 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: no_block_cache: 0 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache: 0x56053e0951f0 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_name: BinnedLRUCache 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_options: 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: capacity : 536870912 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_shard_bits : 4 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: strict_capacity_limit : 0 2023-12-14T18:20:02.976 INFO:journalctl@ceph.mon.a.smithi031.stdout: high_pri_pool_ratio: 0.000 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_cache_compressed: (nil) 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: persistent_cache: (nil) 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size: 4096 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_size_deviation: 10 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_restart_interval: 16 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: index_block_restart_interval: 1 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: metadata_block_size: 4096 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: partition_filters: 0 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: use_delta_encoding: 1 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: filter_policy: bloomfilter 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: whole_key_filtering: 1 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: verify_compression: 0 2023-12-14T18:20:02.977 INFO:journalctl@ceph.mon.a.smithi031.stdout: read_amp_bytes_per_bit: 0 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: format_version: 5 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_index_compression: 1 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: block_align: 0 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: max_auto_readahead_size: 262144 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: prepopulate_block_cache: 0 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: initial_auto_readahead_size: 8192 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.write_buffer_size: 33554432 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_write_buffer_number: 2 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression: NoCompression 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression: Disabled 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.prefix_extractor: nullptr 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-14T18:20:02.978 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.num_levels: 7 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-14T18:20:02.979 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.level: 32767 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.strategy: 0 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.enabled: false 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.target_file_size_base: 67108864 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-14T18:20:02.980 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-14T18:20:02.981 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.arena_block_size: 1048576 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.disable_auto_compactions: 0 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-14T18:20:02.982 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.table_properties_collectors: 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.inplace_update_support: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.bloom_locality: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.max_successive_merges: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.paranoid_file_checks: 0 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.force_consistency_checks: 1 2023-12-14T18:20:02.983 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.report_bg_io_stats: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.ttl: 2592000 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.enable_blob_files: false 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.min_blob_size: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.blob_file_size: 268435456 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.blob_file_starting_level: 0 2023-12-14T18:20:02.984 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 25d7fa31-6b74-490b-afe7-a6fcefeaf06c 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578002501874, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578002503137, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91965, "index_size": 283, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13658, "raw_average_key_size": 50, "raw_value_size": 84506, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702578002, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "25d7fa31-6b74-490b-afe7-a6fcefeaf06c", "db_session_id": "LXAVDKSJTR7ZE9A4QZC4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578002503226, "job": 1, "event": "recovery_finished"} 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56053e132000 2023-12-14T18:20:02.985 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: DB pointer 0x56053e11e000 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** DB Stats ** 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2023-12-14T18:20:02.986 INFO:journalctl@ceph.mon.a.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: L0 2/0 93.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 111.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: Sum 2/0 93.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 111.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 111.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** Compaction Stats [default] ** 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 111.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-12-14T18:20:02.987 INFO:journalctl@ceph.mon.a.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: Cumulative compaction: 0.00 GB write, 14.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: Interval compaction: 0.00 GB write, 14.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache BinnedLRUCache@0x56053e0951f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: starting mon.a rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:02.988 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???) e1 preinit fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).mds e1 new map 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).mds e1 print_map 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout: e1 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout: legacy client fscid: -1 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout: 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout: No filesystems configured 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:02.989 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 bash[42048]: df226c72c46f5e0bd0ed64dc944dd465269330010794e598c6fa0a7832fe412f 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: monmap e1: 1 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: fsmap 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: osdmap e1: 0 total, 0 up, 0 in 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 ceph-mon[42085]: mgrmap e1: no daemons active 2023-12-14T18:20:02.990 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:02 smithi031 systemd[1]: Started Ceph mon.a for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:05.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:04 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4144327067' entity='client.admin' 2023-12-14T18:20:05.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ef21700 1 Processor -- start 2023-12-14T18:20:05.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ef21700 1 -- start start 2023-12-14T18:20:05.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ef21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:05.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ef21700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08079e00 con 0x7f0d081064f0 2023-12-14T18:20:05.607 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ccbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ccbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47318/0 (socket says 172.21.15.31:47318) 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.863+0000 7f0d0ccbd700 1 -- 172.21.15.31:0/2632228654 learned_addr learned my addr 172.21.15.31:0/2632228654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.864+0000 7f0d0ccbd700 1 -- 172.21.15.31:0/2632228654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08108e10 con 0x7f0d081064f0 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.864+0000 7f0d0ccbd700 1 --2- 172.21.15.31:0/2632228654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098a0 tx=0x7f0cfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97af69a39bc59ae5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.864+0000 7f0d077fe700 1 -- 172.21.15.31:0/2632228654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc004160 con 0x7f0d081064f0 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.864+0000 7f0d077fe700 1 -- 172.21.15.31:0/2632228654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0cfc0042c0 con 0x7f0d081064f0 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.864+0000 7f0d077fe700 1 -- 172.21.15.31:0/2632228654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc011a00 con 0x7f0d081064f0 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 -- 172.21.15.31:0/2632228654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 msgr2=0x7f0d081088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 --2- 172.21.15.31:0/2632228654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0098a0 tx=0x7f0cfc009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 -- 172.21.15.31:0/2632228654 shutdown_connections 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 --2- 172.21.15.31:0/2632228654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081088d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 -- 172.21.15.31:0/2632228654 >> 172.21.15.31:0/2632228654 conn(0x7f0d080ffed0 msgr2=0x7f0d08102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:05.608 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 -- 172.21.15.31:0/2632228654 shutdown_connections 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.865+0000 7f0d0ef21700 1 -- 172.21.15.31:0/2632228654 wait complete. 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ef21700 1 Processor -- start 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ef21700 1 -- start start 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ef21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ef21700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08199cf0 con 0x7f0d081064f0 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ccbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ccbd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47328/0 (socket says 172.21.15.31:47328) 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ccbd700 1 -- 172.21.15.31:0/4144327067 learned_addr learned my addr 172.21.15.31:0/4144327067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ccbd700 1 -- 172.21.15.31:0/4144327067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc009550 con 0x7f0d081064f0 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.866+0000 7f0d0ccbd700 1 --2- 172.21.15.31:0/4144327067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081997b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009fb0 tx=0x7f0cfc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc003930 con 0x7f0d081064f0 2023-12-14T18:20:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d08109d40 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0cfc003a90 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cfc019610 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d08109bb0 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0cfc019bd0 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.867+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0cfc0243f0 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.868+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08062120 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.871+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f0cfc003c00 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.944+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f0d0819a480 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.946+0000 7f0d05ffb700 1 -- 172.21.15.31:0/4144327067 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0cfc01a9d0 con 0x7f0d081064f0 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 msgr2=0x7f0d081997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 --2- 172.21.15.31:0/4144327067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081997b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009fb0 tx=0x7f0cfc011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:05.610 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 shutdown_connections 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 --2- 172.21.15.31:0/4144327067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d081064f0 0x7f0d081997b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 >> 172.21.15.31:0/4144327067 conn(0x7f0d080ffed0 msgr2=0x7f0d081015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 shutdown_connections 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:03.947+0000 7f0d0ef21700 1 -- 172.21.15.31:0/4144327067 wait complete. 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:05.611 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2023-12-14T18:20:05.612 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 9283 ... 2023-12-14T18:20:05.612 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 8765 ... 2023-12-14T18:20:05.764 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x 2023-12-14T18:20:05.764 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x.service: Unit ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x.service not loaded. 2023-12-14T18:20:05.870 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-37e25396-9aad-11ee-95a5-87774f69a715.target.wants/ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-37e25396-9aad-11ee-95a5-87774f69a715@.service. 2023-12-14T18:20:06.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:05 smithi031 systemd[1]: Starting Ceph mgr.x for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:20:06.265 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:20:06.265 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-12-14T18:20:06.275 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:20:06.275 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-12-14T18:20:06.275 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-12-14T18:20:06.275 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2023-12-14T18:20:06.284 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:20:06.284 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-12-14T18:20:06.293 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:20:06.293 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-12-14T18:20:06.293 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-12-14T18:20:06.293 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-14T18:20:06.294 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2023-12-14T18:20:06.294 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2023-12-14T18:20:06.641 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:06 smithi031 bash[42321]: cddee8c588203a6970d4754d8d725a2c2c8dab6ad6003eae767fb993d26e7ba2 2023-12-14T18:20:06.642 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:06 smithi031 systemd[1]: Started Ceph mgr.x for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:06.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:06 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/282884046' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:20:06.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:06.639+0000 7fa3ca654200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:20:06.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:06.759+0000 7fa3ca654200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:20:06.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:06.866+0000 7fa3ca654200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:20:07.117 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:07.120 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:07.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:20:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:20:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:19:54.752261+0000", 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd995734700 1 Processor -- start 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd995734700 1 -- start start 2023-12-14T18:20:07.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd995734700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990104120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd995734700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9901046f0 con 0x7fd990103d10 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd98effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd98effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990104120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54096/0 (socket says 172.21.15.31:54096) 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.673+0000 7fd98effd700 1 -- 172.21.15.31:0/1170800788 learned_addr learned my addr 172.21.15.31:0/1170800788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.674+0000 7fd98effd700 1 -- 172.21.15.31:0/1170800788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd990106ca0 con 0x7fd990103d10 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.674+0000 7fd98effd700 1 --2- 172.21.15.31:0/1170800788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990104120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9800098a0 tx=0x7fd980009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c786eb59b89127 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.674+0000 7fd98dffb700 1 -- 172.21.15.31:0/1170800788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980004160 con 0x7fd990103d10 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.674+0000 7fd98dffb700 1 -- 172.21.15.31:0/1170800788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9800042c0 con 0x7fd990103d10 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.674+0000 7fd98dffb700 1 -- 172.21.15.31:0/1170800788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980011a00 con 0x7fd990103d10 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 -- 172.21.15.31:0/1170800788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 msgr2=0x7fd990104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 --2- 172.21.15.31:0/1170800788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990104120 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9800098a0 tx=0x7fd980009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 -- 172.21.15.31:0/1170800788 shutdown_connections 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 --2- 172.21.15.31:0/1170800788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990104120 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 -- 172.21.15.31:0/1170800788 >> 172.21.15.31:0/1170800788 conn(0x7fd9900ff650 msgr2=0x7fd990101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:07.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 -- 172.21.15.31:0/1170800788 shutdown_connections 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.675+0000 7fd995734700 1 -- 172.21.15.31:0/1170800788 wait complete. 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd995734700 1 Processor -- start 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd995734700 1 -- start start 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd995734700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd995734700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd990199e40 con 0x7fd990103d10 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd98effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd98effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54106/0 (socket says 172.21.15.31:54106) 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd98effd700 1 -- 172.21.15.31:0/282884046 learned_addr learned my addr 172.21.15.31:0/282884046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.676+0000 7fd98effd700 1 -- 172.21.15.31:0/282884046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009550 con 0x7fd990103d10 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd98effd700 1 --2- 172.21.15.31:0/282884046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990199900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd980009fb0 tx=0x7fd980011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd980003930 con 0x7fd990103d10 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd980003a90 con 0x7fd990103d10 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd990109ed0 con 0x7fd990103d10 2023-12-14T18:20:07.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9800195e0 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99010a060 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd980019bc0 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.677+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd9800243f0 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.678+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd990107b10 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.682+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fd990107b10 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.761+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd99019a380 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.762+0000 7fd977fff700 1 -- 172.21.15.31:0/282884046 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd980003c00 con 0x7fd990103d10 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 msgr2=0x7fd990199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 --2- 172.21.15.31:0/282884046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990199900 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd980009fb0 tx=0x7fd980011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 shutdown_connections 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 --2- 172.21.15.31:0/282884046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd990103d10 0x7fd990199900 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 >> 172.21.15.31:0/282884046 conn(0x7fd9900ff650 msgr2=0x7fd990100410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 shutdown_connections 2023-12-14T18:20:07.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:06.763+0000 7fd995734700 1 -- 172.21.15.31:0/282884046 wait complete. 2023-12-14T18:20:07.135 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2023-12-14T18:20:07.463 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:06.963+0000 7fa3ca654200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:20:07.945 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:07 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:07.680+0000 7fa3ca654200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:08.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:07 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:07.944+0000 7fa3ca654200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:20:08.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:08.040+0000 7fa3ca654200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:20:08.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:08.162+0000 7fa3ca654200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:08.583 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:08.264+0000 7fa3ca654200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:20:08.583 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:08.487+0000 7fa3ca654200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:20:08.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:08.582+0000 7fa3ca654200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:20:09.264 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:09.009+0000 7fa3ca654200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:20:09.632 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:09.316+0000 7fa3ca654200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:20:09.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:09 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3698781234' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:20:09.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:09.642+0000 7fa3ca654200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:20:09.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:09.758+0000 7fa3ca654200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:20:10.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:10.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:10.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2023-12-14T18:20:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:10.005 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:20:10.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:19:54.752261+0000", 2023-12-14T18:20:10.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.512+0000 7efcc28dc700 1 Processor -- start 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.512+0000 7efcc28dc700 1 -- start start 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.513+0000 7efcc28dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.513+0000 7efcc28dc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc106410 con 0x7efcbc105ac0 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.513+0000 7efcbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.513+0000 7efcbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54120/0 (socket says 172.21.15.31:54120) 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.513+0000 7efcbbfff700 1 -- 172.21.15.31:0/323554774 learned_addr learned my addr 172.21.15.31:0/323554774 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.513+0000 7efcbbfff700 1 -- 172.21.15.31:0/323554774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc106550 con 0x7efcbc105ac0 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.514+0000 7efcbbfff700 1 --2- 172.21.15.31:0/323554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc105ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098a0 tx=0x7efcac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf1c37b74ec09b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.514+0000 7efcbaffd700 1 -- 172.21.15.31:0/323554774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcac004160 con 0x7efcbc105ac0 2023-12-14T18:20:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.514+0000 7efcbaffd700 1 -- 172.21.15.31:0/323554774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcac0042c0 con 0x7efcbc105ac0 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.514+0000 7efcbaffd700 1 -- 172.21.15.31:0/323554774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcac011a00 con 0x7efcbc105ac0 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.514+0000 7efcc28dc700 1 -- 172.21.15.31:0/323554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 msgr2=0x7efcbc105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.514+0000 7efcc28dc700 1 --2- 172.21.15.31:0/323554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc105ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098a0 tx=0x7efcac009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 -- 172.21.15.31:0/323554774 shutdown_connections 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 --2- 172.21.15.31:0/323554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc105ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 -- 172.21.15.31:0/323554774 >> 172.21.15.31:0/323554774 conn(0x7efcbc101170 msgr2=0x7efcbc103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 -- 172.21.15.31:0/323554774 shutdown_connections 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 -- 172.21.15.31:0/323554774 wait complete. 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 Processor -- start 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.515+0000 7efcc28dc700 1 -- start start 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcc28dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc07bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:10.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcc28dc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc07c140 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc07bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc07bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54128/0 (socket says 172.21.15.31:54128) 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcbbfff700 1 -- 172.21.15.31:0/3698781234 learned_addr learned my addr 172.21.15.31:0/3698781234 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcbbfff700 1 -- 172.21.15.31:0/3698781234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009550 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.516+0000 7efcbbfff700 1 --2- 172.21.15.31:0/3698781234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc07bc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcac006af0 tx=0x7efcac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcac003a40 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc107b10 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcac003ba0 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efcac01a6e0 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc107980 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efcac01ac50 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.517+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efcac019700 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.518+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc062120 con 0x7efcbc105ac0 2023-12-14T18:20:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.523+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7efcac02de30 con 0x7efcbc105ac0 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.601+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efcbc07b110 con 0x7efcbc105ac0 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.602+0000 7efcb97fa700 1 -- 172.21.15.31:0/3698781234 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7efcac024a60 con 0x7efcbc105ac0 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 msgr2=0x7efcbc07bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 --2- 172.21.15.31:0/3698781234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc07bc00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efcac006af0 tx=0x7efcac011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 shutdown_connections 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 --2- 172.21.15.31:0/3698781234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcbc105ac0 0x7efcbc07bc00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 >> 172.21.15.31:0/3698781234 conn(0x7efcbc101170 msgr2=0x7efcbc101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 shutdown_connections 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:09.603+0000 7efcc28dc700 1 -- 172.21.15.31:0/3698781234 wait complete. 2023-12-14T18:20:10.019 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2023-12-14T18:20:10.892 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:10.617+0000 7fa3ca654200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:20:10.892 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:10.723+0000 7fa3ca654200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:20:11.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:10.890+0000 7fa3ca654200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:20:11.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:11 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:11.609+0000 7fa3ca654200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:20:11.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:11 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:11.814+0000 7fa3ca654200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:20:12.463 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:12.033+0000 7fa3ca654200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:20:12.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:12.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:12.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:20:12.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:20:12.786 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:12.787 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:20:12.788 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:20:12.798 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:20:12.798 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:20:12.798 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:20:12.798 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.798 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:20:12.799 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:20:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:19:54.752261+0000", 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3cd9c2700 1 Processor -- start 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3cd9c2700 1 -- start start 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3cd9c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c8105e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3cd9c2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c8106400 con 0x7fd3c8105a20 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c8105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c8105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54142/0 (socket says 172.21.15.31:54142) 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.350+0000 7fd3c6ffd700 1 -- 172.21.15.31:0/1995883197 learned_addr learned my addr 172.21.15.31:0/1995883197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:12.801 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.351+0000 7fd3c6ffd700 1 -- 172.21.15.31:0/1995883197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c8106c20 con 0x7fd3c8105a20 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.351+0000 7fd3c6ffd700 1 --2- 172.21.15.31:0/1995883197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c8105e30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0098a0 tx=0x7fd3bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9390f623d879fc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.351+0000 7fd3c5ffb700 1 -- 172.21.15.31:0/1995883197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc004160 con 0x7fd3c8105a20 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.351+0000 7fd3c5ffb700 1 -- 172.21.15.31:0/1995883197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd3bc0042c0 con 0x7fd3c8105a20 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.351+0000 7fd3c5ffb700 1 -- 172.21.15.31:0/1995883197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc011a00 con 0x7fd3c8105a20 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1995883197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 msgr2=0x7fd3c8105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 --2- 172.21.15.31:0/1995883197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c8105e30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0098a0 tx=0x7fd3bc009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1995883197 shutdown_connections 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 --2- 172.21.15.31:0/1995883197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c8105e30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1995883197 >> 172.21.15.31:0/1995883197 conn(0x7fd3c8100fd0 msgr2=0x7fd3c8103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1995883197 shutdown_connections 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1995883197 wait complete. 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.352+0000 7fd3cd9c2700 1 Processor -- start 2023-12-14T18:20:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3cd9c2700 1 -- start start 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3cd9c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c807bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3cd9c2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c807c140 con 0x7fd3c8105a20 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c807bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c807bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54150/0 (socket says 172.21.15.31:54150) 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3c6ffd700 1 -- 172.21.15.31:0/1334934533 learned_addr learned my addr 172.21.15.31:0/1334934533 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.353+0000 7fd3c6ffd700 1 -- 172.21.15.31:0/1334934533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc009550 con 0x7fd3c8105a20 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.354+0000 7fd3c6ffd700 1 --2- 172.21.15.31:0/1334934533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c807bc00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009fb0 tx=0x7fd3bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.354+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc003930 con 0x7fd3c8105a20 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.354+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd3bc003a90 con 0x7fd3c8105a20 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.354+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c8109e50 con 0x7fd3c8105a20 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.354+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3bc0195e0 con 0x7fd3c8105a20 2023-12-14T18:20:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.354+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c8109fe0 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.355+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd3bc019870 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.355+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c8107a90 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.355+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd3bc003c00 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.359+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fd3c8107a90 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.437+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd3c807aa30 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.438+0000 7fd3cc9c0700 1 -- 172.21.15.31:0/1334934533 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd3bc0243f0 con 0x7fd3c8105a20 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 msgr2=0x7fd3c807bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 --2- 172.21.15.31:0/1334934533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c807bc00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009fb0 tx=0x7fd3bc011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 shutdown_connections 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 --2- 172.21.15.31:0/1334934533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3c8105a20 0x7fd3c807bc00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 >> 172.21.15.31:0/1334934533 conn(0x7fd3c8100fd0 msgr2=0x7fd3c8102640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 shutdown_connections 2023-12-14T18:20:12.804 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:12.439+0000 7fd3cd9c2700 1 -- 172.21.15.31:0/1334934533 wait complete. 2023-12-14T18:20:12.805 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (3/15)... 2023-12-14T18:20:12.869 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:12 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1334934533' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:20:13.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:12.868+0000 7fa3ca654200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:20:13.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:13.408+0000 7fa3ca654200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:13.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:13.508+0000 7fa3ca654200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:13.724+0000 7fa3ca654200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: Activating manager daemon x 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: mgrmap e2: x(active, starting, since 0.00330275s) 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: Manager daemon x is now available 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:20:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:13 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' 2023-12-14T18:20:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:14 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' 2023-12-14T18:20:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:14 smithi031 ceph-mon[42085]: from='mgr.14100 172.21.15.31:0/3430138603' entity='mgr.x' 2023-12-14T18:20:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:14 smithi031 ceph-mon[42085]: mgrmap e3: x(active, since 1.00663s) 2023-12-14T18:20:15.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:15.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:15.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:20:15.869 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:15.870 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:20:15.871 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:20:15.880 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:20:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:20:15.882 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:19:54.752261+0000", 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.197+0000 7f23a3c5c700 1 Processor -- start 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.198+0000 7f23a3c5c700 1 -- start start 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.198+0000 7f23a3c5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:15.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.198+0000 7f23a3c5c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c105300 con 0x7f239c104920 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.198+0000 7f23a19f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.198+0000 7f23a19f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43018/0 (socket says 172.21.15.31:43018) 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.198+0000 7f23a19f8700 1 -- 172.21.15.31:0/4103531093 learned_addr learned my addr 172.21.15.31:0/4103531093 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.199+0000 7f23a19f8700 1 -- 172.21.15.31:0/4103531093 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c105b20 con 0x7f239c104920 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.199+0000 7f23a19f8700 1 --2- 172.21.15.31:0/4103531093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c104d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f23980098a0 tx=0x7f2398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1f9e3ba31f834 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.199+0000 7f23a09f6700 1 -- 172.21.15.31:0/4103531093 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2398004160 con 0x7f239c104920 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.199+0000 7f23a09f6700 1 -- 172.21.15.31:0/4103531093 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f23980042c0 con 0x7f239c104920 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.199+0000 7f23a09f6700 1 -- 172.21.15.31:0/4103531093 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2398011a00 con 0x7f239c104920 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 -- 172.21.15.31:0/4103531093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 msgr2=0x7f239c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 --2- 172.21.15.31:0/4103531093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c104d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f23980098a0 tx=0x7f2398009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 -- 172.21.15.31:0/4103531093 shutdown_connections 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 --2- 172.21.15.31:0/4103531093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c104d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:15.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 -- 172.21.15.31:0/4103531093 >> 172.21.15.31:0/4103531093 conn(0x7f239c0ffed0 msgr2=0x7f239c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 -- 172.21.15.31:0/4103531093 shutdown_connections 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 -- 172.21.15.31:0/4103531093 wait complete. 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.200+0000 7f23a3c5c700 1 Processor -- start 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a3c5c700 1 -- start start 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a3c5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a3c5c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c19c080 con 0x7f239c104920 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a19f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a19f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c19bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43028/0 (socket says 172.21.15.31:43028) 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a19f8700 1 -- 172.21.15.31:0/1376924793 learned_addr learned my addr 172.21.15.31:0/1376924793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a19f8700 1 -- 172.21.15.31:0/1376924793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398009550 con 0x7f239c104920 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f23a19f8700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c19bb40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2398006af0 tx=0x7f2398011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:15.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.201+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2398003a40 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.202+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2398003ba0 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.202+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c108d50 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.202+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f239801a700 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.202+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c108ee0 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.202+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f239801a980 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.203+0000 7f2392ffd700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f2388038590 0x7f238803aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.203+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c106990 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.203+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f239804c640 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.203+0000 7f23a11f7700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f2388038590 0x7f238803aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.203+0000 7f23a11f7700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f2388038590 0x7f238803aa40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.208+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f239c106990 con 0x7f239c104920 2023-12-14T18:20:15.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.503+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f239c106990 con 0x7f239c104920 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.504+0000 7f2392ffd700 1 -- 172.21.15.31:0/1376924793 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f239c106990 con 0x7f239c104920 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f2388038590 msgr2=0x7f238803aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f2388038590 0x7f238803aa40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 msgr2=0x7f239c19bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c19bb40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2398006af0 tx=0x7f2398011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 shutdown_connections 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f2388038590 0x7f238803aa40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 --2- 172.21.15.31:0/1376924793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f239c104920 0x7f239c19bb40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 >> 172.21.15.31:0/1376924793 conn(0x7f239c0ffed0 msgr2=0x7f239c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 shutdown_connections 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:15.506+0000 7f23a3c5c700 1 -- 172.21.15.31:0/1376924793 wait complete. 2023-12-14T18:20:15.887 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2023-12-14T18:20:16.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:15 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1376924793' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-14T18:20:16.791 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f8944c2e700 1 Processor -- start 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f8944c2e700 1 -- start start 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f8944c2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f8940104730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f8944c2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940079dc0 con 0x7f8940104320 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f893e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f8940104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f893e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f8940104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43036/0 (socket says 172.21.15.31:43036) 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.206+0000 7f893e59c700 1 -- 172.21.15.31:0/81749135 learned_addr learned my addr 172.21.15.31:0/81749135 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.207+0000 7f893e59c700 1 -- 172.21.15.31:0/81749135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940079f00 con 0x7f8940104320 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.207+0000 7f893e59c700 1 --2- 172.21.15.31:0/81749135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f8940104730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f89300098a0 tx=0x7f8930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdcac2079e1d4e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.207+0000 7f893d59a700 1 -- 172.21.15.31:0/81749135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8930004160 con 0x7f8940104320 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.207+0000 7f893d59a700 1 -- 172.21.15.31:0/81749135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f89300042c0 con 0x7f8940104320 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.207+0000 7f893d59a700 1 -- 172.21.15.31:0/81749135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8930011a00 con 0x7f8940104320 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 -- 172.21.15.31:0/81749135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 msgr2=0x7f8940104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 --2- 172.21.15.31:0/81749135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f8940104730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f89300098a0 tx=0x7f8930009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 -- 172.21.15.31:0/81749135 shutdown_connections 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 --2- 172.21.15.31:0/81749135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f8940104730 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 -- 172.21.15.31:0/81749135 >> 172.21.15.31:0/81749135 conn(0x7f89400ffed0 msgr2=0x7f8940102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 -- 172.21.15.31:0/81749135 shutdown_connections 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.208+0000 7f8944c2e700 1 -- 172.21.15.31:0/81749135 wait complete. 2023-12-14T18:20:16.793 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f8944c2e700 1 Processor -- start 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f8944c2e700 1 -- start start 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f8944c2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f894019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f8944c2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894019be90 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f893e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f894019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f893e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f894019b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43042/0 (socket says 172.21.15.31:43042) 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f893e59c700 1 -- 172.21.15.31:0/464893196 learned_addr learned my addr 172.21.15.31:0/464893196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.209+0000 7f893e59c700 1 -- 172.21.15.31:0/464893196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930009550 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.210+0000 7f893e59c700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f894019b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8930009fb0 tx=0x7f8930011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.210+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8930003930 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.210+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8940105ae0 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.210+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8930003a90 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.210+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8930019610 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.210+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8940105950 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.211+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f893001a430 con 0x7f8940104320 2023-12-14T18:20:16.794 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.211+0000 7f892f7fe700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f8928038670 0x7f892803ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.211+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f893004c470 con 0x7f8940104320 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.211+0000 7f893dd9b700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f8928038670 0x7f892803ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.211+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8940105c70 con 0x7f8940104320 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.212+0000 7f893dd9b700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f8928038670 0x7f892803ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8934006fd0 tx=0x7f8934006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.217+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8930003c00 con 0x7f8940104320 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.425+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f894002cfe0 con 0x7f8940104320 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.426+0000 7f892f7fe700 1 -- 172.21.15.31:0/464893196 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f8930023b80 con 0x7f8940104320 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f8928038670 msgr2=0x7f892803ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f8928038670 0x7f892803ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8934006fd0 tx=0x7f8934006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 msgr2=0x7f894019b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f894019b950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8930009fb0 tx=0x7f8930011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 shutdown_connections 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f8928038670 0x7f892803ab20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 --2- 172.21.15.31:0/464893196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8940104320 0x7f894019b950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:16.795 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 >> 172.21.15.31:0/464893196 conn(0x7f89400ffed0 msgr2=0x7f894018fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:16.796 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.428+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 shutdown_connections 2023-12-14T18:20:16.796 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:16.429+0000 7f8944c2e700 1 -- 172.21.15.31:0/464893196 wait complete. 2023-12-14T18:20:16.796 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2023-12-14T18:20:17.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:16 smithi031 ceph-mon[42085]: mgrmap e4: x(active, since 2s) 2023-12-14T18:20:17.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:16 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/464893196' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-14T18:20:18.118 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:17 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/418154981' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-14T18:20:18.119 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: ignoring --setuser ceph since I am not root 2023-12-14T18:20:18.119 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: ignoring --setgroup ceph since I am not root 2023-12-14T18:20:18.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbea18ae700 1 Processor -- start 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbea18ae700 1 -- start start 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbea18ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbea18ae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c105300 con 0x7fbe9c104920 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbe9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbe9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43056/0 (socket says 172.21.15.31:43056) 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbe9affd700 1 -- 172.21.15.31:0/538160055 learned_addr learned my addr 172.21.15.31:0/538160055 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.146+0000 7fbe9affd700 1 -- 172.21.15.31:0/538160055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c105b20 con 0x7fbe9c104920 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbe9affd700 1 --2- 172.21.15.31:0/538160055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c104d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0098a0 tx=0x7fbe8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94c598661e523c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbe99ffb700 1 -- 172.21.15.31:0/538160055 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c004160 con 0x7fbe9c104920 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbe99ffb700 1 -- 172.21.15.31:0/538160055 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe8c0042c0 con 0x7fbe9c104920 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbe99ffb700 1 -- 172.21.15.31:0/538160055 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c011a00 con 0x7fbe9c104920 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbea18ae700 1 -- 172.21.15.31:0/538160055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 msgr2=0x7fbe9c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbea18ae700 1 --2- 172.21.15.31:0/538160055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c104d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0098a0 tx=0x7fbe8c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbea18ae700 1 -- 172.21.15.31:0/538160055 shutdown_connections 2023-12-14T18:20:18.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.147+0000 7fbea18ae700 1 --2- 172.21.15.31:0/538160055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c104d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.148+0000 7fbea18ae700 1 -- 172.21.15.31:0/538160055 >> 172.21.15.31:0/538160055 conn(0x7fbe9c0ffed0 msgr2=0x7fbe9c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.148+0000 7fbea18ae700 1 -- 172.21.15.31:0/538160055 shutdown_connections 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.148+0000 7fbea18ae700 1 -- 172.21.15.31:0/538160055 wait complete. 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.148+0000 7fbea18ae700 1 Processor -- start 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.148+0000 7fbea18ae700 1 -- start start 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbea18ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbea18ae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c199e80 con 0x7fbe9c104920 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbe9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbe9affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43068/0 (socket says 172.21.15.31:43068) 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbe9affd700 1 -- 172.21.15.31:0/418154981 learned_addr learned my addr 172.21.15.31:0/418154981 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbe9affd700 1 -- 172.21.15.31:0/418154981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c009550 con 0x7fbe9c104920 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbe9affd700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c199940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009fb0 tx=0x7fbe8c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c003930 con 0x7fbe9c104920 2023-12-14T18:20:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c108d50 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.149+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe8c003a90 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.150+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c0195e0 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.150+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c108ee0 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.150+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fbe8c0198a0 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.151+0000 7fbea08ac700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fbe84038670 0x7fbe8403ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.151+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbe8c04c6b0 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.151+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c106990 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.151+0000 7fbe9a7fc700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fbe84038670 0x7fbe8403ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.151+0000 7fbe9a7fc700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fbe84038670 0x7fbe8403ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe90006fd0 tx=0x7fbe90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.156+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe9c106990 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.424+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fbe9c106990 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.805+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbe8c019d30 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.806+0000 7fbea08ac700 1 -- 172.21.15.31:0/418154981 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fbe8c018d20 con 0x7fbe9c104920 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fbe84038670 msgr2=0x7fbe8403ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fbe84038670 0x7fbe8403ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe90006fd0 tx=0x7fbe90006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:18.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 msgr2=0x7fbe9c199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c199940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009fb0 tx=0x7fbe8c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 shutdown_connections 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fbe84038670 0x7fbe8403ab20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 --2- 172.21.15.31:0/418154981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe9c104920 0x7fbe9c199940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.809+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 >> 172.21.15.31:0/418154981 conn(0x7fbe9c0ffed0 msgr2=0x7fbe9c0798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.810+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 shutdown_connections 2023-12-14T18:20:18.199 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:17.810+0000 7fbea18ae700 1 -- 172.21.15.31:0/418154981 wait complete. 2023-12-14T18:20:18.456 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:18.211+0000 7fbb98305200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:20:18.456 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:18.342+0000 7fbb98305200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:20:18.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:18.455+0000 7fbb98305200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:20:18.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:18.551+0000 7fbb98305200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:20:19.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:18 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/418154981' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-14T18:20:19.067 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:18 smithi031 ceph-mon[42085]: mgrmap e5: x(active, since 4s) 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.550+0000 7fe9aa0fb700 1 Processor -- start 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.550+0000 7fe9aa0fb700 1 -- start start 2023-12-14T18:20:19.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9aa0fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a4104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9aa0fb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a4079de0 con 0x7fe9a4104300 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a4104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a4104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43082/0 (socket says 172.21.15.31:43082) 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9a37fe700 1 -- 172.21.15.31:0/2304610098 learned_addr learned my addr 172.21.15.31:0/2304610098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9a37fe700 1 -- 172.21.15.31:0/2304610098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a4104c50 con 0x7fe9a4104300 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.551+0000 7fe9a37fe700 1 --2- 172.21.15.31:0/2304610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a4104710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9940098a0 tx=0x7fe994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c1d01fdf8a5b93 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9a27fc700 1 -- 172.21.15.31:0/2304610098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe994004160 con 0x7fe9a4104300 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9a27fc700 1 -- 172.21.15.31:0/2304610098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe9940042c0 con 0x7fe9a4104300 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9a27fc700 1 -- 172.21.15.31:0/2304610098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe994011a00 con 0x7fe9a4104300 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2304610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 msgr2=0x7fe9a4104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9aa0fb700 1 --2- 172.21.15.31:0/2304610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a4104710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9940098a0 tx=0x7fe994009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2304610098 shutdown_connections 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9aa0fb700 1 --2- 172.21.15.31:0/2304610098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a4104710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.552+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2304610098 >> 172.21.15.31:0/2304610098 conn(0x7fe9a40ffed0 msgr2=0x7fe9a4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.553+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2304610098 shutdown_connections 2023-12-14T18:20:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.553+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2304610098 wait complete. 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.553+0000 7fe9aa0fb700 1 Processor -- start 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.553+0000 7fe9aa0fb700 1 -- start start 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9aa0fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9aa0fb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a419bee0 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43094/0 (socket says 172.21.15.31:43094) 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9a37fe700 1 -- 172.21.15.31:0/2265042474 learned_addr learned my addr 172.21.15.31:0/2265042474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9a37fe700 1 -- 172.21.15.31:0/2265042474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994009550 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.554+0000 7fe9a37fe700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a419b9a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe994006af0 tx=0x7fe994011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.555+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe994003a40 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.555+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe994003ba0 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.555+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a4105c50 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.555+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe99401a6e0 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.555+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a4105ac0 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.555+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe99401a950 con 0x7fe9a4104300 2023-12-14T18:20:19.223 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.556+0000 7fe9a0ff9700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 0x7fe98c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.556+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe99404d690 con 0x7fe9a4104300 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.556+0000 7fe9a2ffd700 1 -- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 msgr2=0x7fe98c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.556+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990005320 con 0x7fe9a4104300 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.556+0000 7fe9a2ffd700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 0x7fe98c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.561+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe99401ac00 con 0x7fe9a4104300 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.757+0000 7fe9a2ffd700 1 -- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 msgr2=0x7fe98c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.757+0000 7fe9a2ffd700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 0x7fe98c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.833+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe990006200 con 0x7fe9a4104300 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.833+0000 7fe9a0ff9700 1 -- 172.21.15.31:0/2265042474 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe994018350 con 0x7fe9a4104300 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 msgr2=0x7fe98c03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 0x7fe98c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 msgr2=0x7fe9a419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a419b9a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe994006af0 tx=0x7fe994011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 shutdown_connections 2023-12-14T18:20:19.224 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7fe98c038710 0x7fe98c03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 --2- 172.21.15.31:0/2265042474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9a4104300 0x7fe9a419b9a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 >> 172.21.15.31:0/2265042474 conn(0x7fe9a40ffed0 msgr2=0x7fe9a418fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 shutdown_connections 2023-12-14T18:20:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:18.836+0000 7fe9aa0fb700 1 -- 172.21.15.31:0/2265042474 wait complete. 2023-12-14T18:20:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2023-12-14T18:20:19.225 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2023-12-14T18:20:19.321 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:19.266+0000 7fbb98305200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:19.637 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:19.536+0000 7fbb98305200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:20:19.637 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:19.636+0000 7fbb98305200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:20:19.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:19 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2265042474' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-14T18:20:19.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:19.759+0000 7fbb98305200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:19.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:19.862+0000 7fbb98305200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:20:20.463 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:20.084+0000 7fbb98305200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:20:20.463 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:20.179+0000 7fbb98305200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:20:20.899 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:20.601+0000 7fbb98305200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:20:21.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:20.898+0000 7fbb98305200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:20:21.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:21.219+0000 7fbb98305200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:20:21.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:21.324+0000 7fbb98305200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:20:22.450 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:22.176+0000 7fbb98305200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:20:22.451 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:22.283+0000 7fbb98305200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:20:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:22.449+0000 7fbb98305200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:20:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:23.168+0000 7fbb98305200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:20:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:23.373+0000 7fbb98305200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:20:23.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:23.595+0000 7fbb98305200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:20:24.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:24.394+0000 7fbb98305200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:20:25.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:24.936+0000 7fbb98305200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:25.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:25.035+0000 7fbb98305200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:25.251+0000 7fbb98305200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: Active manager daemon x restarted 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: Activating manager daemon x 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: osdmap e2: 0 total, 0 up, 0 in 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: mgrmap e6: x(active, starting, since 0.00582116s) 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:20:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: Manager daemon x is now available 2023-12-14T18:20:25.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:25.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:25 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:26.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: Found migration_current of "None". Setting to last migration. 2023-12-14T18:20:26.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:26.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:26.576 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:26.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:20:26.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:20:26.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:26.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:26.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:26.577 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:26 smithi031 ceph-mon[42085]: mgrmap e7: x(active, since 1.00674s) 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.585+0000 7f9817b7e700 1 Processor -- start 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.585+0000 7f9817b7e700 1 -- start start 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f9817b7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f9810104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f9817b7e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810105300 con 0x7f9810104920 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f981591a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f9810104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:26.656 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f981591a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f9810104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43110/0 (socket says 172.21.15.31:43110) 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f981591a700 1 -- 172.21.15.31:0/4126155158 learned_addr learned my addr 172.21.15.31:0/4126155158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f981591a700 1 -- 172.21.15.31:0/4126155158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9810105b20 con 0x7f9810104920 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.586+0000 7f981591a700 1 --2- 172.21.15.31:0/4126155158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f9810104d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f980c0098a0 tx=0x7f980c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681a623717e43bed server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9814918700 1 -- 172.21.15.31:0/4126155158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f980c004160 con 0x7f9810104920 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9814918700 1 -- 172.21.15.31:0/4126155158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f980c0042c0 con 0x7f9810104920 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9814918700 1 -- 172.21.15.31:0/4126155158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f980c011a00 con 0x7f9810104920 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9817b7e700 1 -- 172.21.15.31:0/4126155158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 msgr2=0x7f9810104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9817b7e700 1 --2- 172.21.15.31:0/4126155158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f9810104d30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f980c0098a0 tx=0x7f980c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9817b7e700 1 -- 172.21.15.31:0/4126155158 shutdown_connections 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9817b7e700 1 --2- 172.21.15.31:0/4126155158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f9810104d30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.587+0000 7f9817b7e700 1 -- 172.21.15.31:0/4126155158 >> 172.21.15.31:0/4126155158 conn(0x7f98100ffed0 msgr2=0x7f9810102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.588+0000 7f9817b7e700 1 -- 172.21.15.31:0/4126155158 shutdown_connections 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.588+0000 7f9817b7e700 1 -- 172.21.15.31:0/4126155158 wait complete. 2023-12-14T18:20:26.657 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.588+0000 7f9817b7e700 1 Processor -- start 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.588+0000 7f9817b7e700 1 -- start start 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.588+0000 7f9817b7e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f981018fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.588+0000 7f9817b7e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810190260 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f981591a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f981018fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f981591a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f981018fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43126/0 (socket says 172.21.15.31:43126) 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f981591a700 1 -- 172.21.15.31:0/1164893179 learned_addr learned my addr 172.21.15.31:0/1164893179 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f981591a700 1 -- 172.21.15.31:0/1164893179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c009550 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f981591a700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f981018fd20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f980c006af0 tx=0x7f980c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f980c003a40 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f980c003ba0 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9810108d50 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f980c01a700 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.589+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9810108ee0 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.590+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f980c024ad0 con 0x7f9810104920 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.591+0000 7f9806ffd700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.591+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 --> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97fc03b2d0 con 0x7f97fc038710 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.591+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f980c04c720 con 0x7f9810104920 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.591+0000 7f9815119700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 msgr2=0x7f97fc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.591+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.792+0000 7f9815119700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 msgr2=0x7f97fc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:19.792+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:20.193+0000 7f9815119700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 msgr2=0x7f97fc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:20.193+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:20.994+0000 7f9815119700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 msgr2=0x7f97fc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:20.994+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:22.595+0000 7f9815119700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 msgr2=0x7f97fc03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2094677585 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:22.595+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:25.268+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f980c02d910 con 0x7f9810104920 2023-12-14T18:20:26.659 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:25.268+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 msgr2=0x7f97fc03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:25.268+0000 7f9806ffd700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.267+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f980c04e250 con 0x7f9810104920 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.268+0000 7f9806ffd700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f97fc03bfe0 0x7f97fc03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.268+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97fc03b2d0 con 0x7f97fc03bfe0 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.268+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f97fc03bfe0 0x7f97fc03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.269+0000 7f9815119700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f97fc03bfe0 0x7f97fc03e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9800003a10 tx=0x7f9800009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.270+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f97fc03b2d0 con 0x7f97fc03bfe0 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.277+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f981018f8c0 con 0x7f97fc03bfe0 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.277+0000 7f9806ffd700 1 -- 172.21.15.31:0/1164893179 <== mgr.14120 v2:172.21.15.31:6800/2330625991 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f981018f8c0 con 0x7f97fc03bfe0 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.277+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f97fc03bfe0 msgr2=0x7f97fc03e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.277+0000 7f9817b7e700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f97fc03bfe0 0x7f97fc03e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9800003a10 tx=0x7f9800009290 comp rx=0 tx=0).stop 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 msgr2=0x7f981018fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f981018fd20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f980c006af0 tx=0x7f980c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 shutdown_connections 2023-12-14T18:20:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f97fc03bfe0 0x7f97fc03e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9810104920 0x7f981018fd20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 --2- 172.21.15.31:0/1164893179 >> [v2:172.21.15.31:6800/2094677585,v1:172.21.15.31:6801/2094677585] conn(0x7f97fc038710 0x7f97fc03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 >> 172.21.15.31:0/1164893179 conn(0x7f98100ffed0 msgr2=0x7f98100798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 shutdown_connections 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.278+0000 7f9817b7e700 1 -- 172.21.15.31:0/1164893179 wait complete. 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2023-12-14T18:20:26.661 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2023-12-14T18:20:27.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:25] ENGINE Bus STARTING 2023-12-14T18:20:27.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:26] ENGINE Serving on http://172.21.15.31:8765 2023-12-14T18:20:27.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:26] ENGINE Serving on https://172.21.15.31:7150 2023-12-14T18:20:27.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:26] ENGINE Bus STARTED 2023-12-14T18:20:27.518 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-14T18:20:27.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-14T18:20:27.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:27.519 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:27 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f6992196700 1 Processor -- start 2023-12-14T18:20:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f6992196700 1 -- start start 2023-12-14T18:20:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f6992196700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f6992196700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c079e60 con 0x7f698c106490 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f698b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f698b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38940/0 (socket says 172.21.15.31:38940) 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f698b7fe700 1 -- 172.21.15.31:0/1510418279 learned_addr learned my addr 172.21.15.31:0/1510418279 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.996+0000 7f698b7fe700 1 -- 172.21.15.31:0/1510418279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c108db0 con 0x7f698c106490 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f698b7fe700 1 --2- 172.21.15.31:0/1510418279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c108870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f697c0098a0 tx=0x7f697c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7f541d22db861e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f698a7fc700 1 -- 172.21.15.31:0/1510418279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f697c004160 con 0x7f698c106490 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f698a7fc700 1 -- 172.21.15.31:0/1510418279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f697c0042c0 con 0x7f698c106490 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f698a7fc700 1 -- 172.21.15.31:0/1510418279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f697c011a00 con 0x7f698c106490 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f6992196700 1 -- 172.21.15.31:0/1510418279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 msgr2=0x7f698c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f6992196700 1 --2- 172.21.15.31:0/1510418279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c108870 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f697c0098a0 tx=0x7f697c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.997+0000 7f6992196700 1 -- 172.21.15.31:0/1510418279 shutdown_connections 2023-12-14T18:20:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.998+0000 7f6992196700 1 --2- 172.21.15.31:0/1510418279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c108870 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.998+0000 7f6992196700 1 -- 172.21.15.31:0/1510418279 >> 172.21.15.31:0/1510418279 conn(0x7f698c0ffed0 msgr2=0x7f698c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.998+0000 7f6992196700 1 -- 172.21.15.31:0/1510418279 shutdown_connections 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.998+0000 7f6992196700 1 -- 172.21.15.31:0/1510418279 wait complete. 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.998+0000 7f6992196700 1 Processor -- start 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.998+0000 7f6992196700 1 -- start start 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f6992196700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f6992196700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c199ca0 con 0x7f698c106490 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f698b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f698b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38954/0 (socket says 172.21.15.31:38954) 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f698b7fe700 1 -- 172.21.15.31:0/659075211 learned_addr learned my addr 172.21.15.31:0/659075211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f698b7fe700 1 -- 172.21.15.31:0/659075211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c009550 con 0x7f698c106490 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f698b7fe700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c199760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f697c006af0 tx=0x7f697c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f697c003a40 con 0x7f698c106490 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f697c003ba0 con 0x7f698c106490 2023-12-14T18:20:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c109d40 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:26.999+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f697c01a6e0 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.000+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c109bb0 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.000+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f697c024430 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.001+0000 7f6988ff9700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f69740385e0 0x7f697403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.001+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c062120 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.001+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f697c04c5b0 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.001+0000 7f698affd700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f69740385e0 0x7f697403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.001+0000 7f698affd700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f69740385e0 0x7f697403aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6980006fd0 tx=0x7f6980006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.006+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f697c0246e0 con 0x7f698c106490 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.234+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f698c1013d0 con 0x7f69740385e0 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.240+0000 7f6988ff9700 1 -- 172.21.15.31:0/659075211 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f698c1013d0 con 0x7f69740385e0 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.242+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f69740385e0 msgr2=0x7f697403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.242+0000 7f6992196700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f69740385e0 0x7f697403aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6980006fd0 tx=0x7f6980006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 msgr2=0x7f698c199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c199760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f697c006af0 tx=0x7f697c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 shutdown_connections 2023-12-14T18:20:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f69740385e0 0x7f697403aa90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 --2- 172.21.15.31:0/659075211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f698c106490 0x7f698c199760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 >> 172.21.15.31:0/659075211 conn(0x7f698c0ffed0 msgr2=0x7f698c100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 shutdown_connections 2023-12-14T18:20:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.243+0000 7f6992196700 1 -- 172.21.15.31:0/659075211 wait complete. 2023-12-14T18:20:28.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.969+0000 7f85b4f4b700 1 Processor -- start 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.969+0000 7f85b4f4b700 1 -- start start 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.969+0000 7f85b4f4b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b0105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.969+0000 7f85b4f4b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0105800 con 0x7f85b0104e20 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b0105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b0105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38958/0 (socket says 172.21.15.31:38958) 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85ae59c700 1 -- 172.21.15.31:0/2133980042 learned_addr learned my addr 172.21.15.31:0/2133980042 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85ae59c700 1 -- 172.21.15.31:0/2133980042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b0105b40 con 0x7f85b0104e20 2023-12-14T18:20:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85ae59c700 1 --2- 172.21.15.31:0/2133980042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b0105230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f85a00098a0 tx=0x7f85a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc85734368b2cfb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85add9b700 1 -- 172.21.15.31:0/2133980042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a0004160 con 0x7f85b0104e20 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85add9b700 1 -- 172.21.15.31:0/2133980042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85a00042c0 con 0x7f85b0104e20 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.970+0000 7f85add9b700 1 -- 172.21.15.31:0/2133980042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a0011a00 con 0x7f85b0104e20 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 -- 172.21.15.31:0/2133980042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 msgr2=0x7f85b0105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 --2- 172.21.15.31:0/2133980042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b0105230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f85a00098a0 tx=0x7f85a0009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 -- 172.21.15.31:0/2133980042 shutdown_connections 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 --2- 172.21.15.31:0/2133980042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b0105230 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 -- 172.21.15.31:0/2133980042 >> 172.21.15.31:0/2133980042 conn(0x7f85b00ff650 msgr2=0x7f85b0101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 -- 172.21.15.31:0/2133980042 shutdown_connections 2023-12-14T18:20:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.971+0000 7f85b4f4b700 1 -- 172.21.15.31:0/2133980042 wait complete. 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.972+0000 7f85b4f4b700 1 Processor -- start 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.972+0000 7f85b4f4b700 1 -- start start 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.972+0000 7f85b4f4b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.972+0000 7f85b4f4b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0199e20 con 0x7f85b0104e20 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85ae59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38960/0 (socket says 172.21.15.31:38960) 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85ae59c700 1 -- 172.21.15.31:0/1014560238 learned_addr learned my addr 172.21.15.31:0/1014560238 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85ae59c700 1 -- 172.21.15.31:0/1014560238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a0009550 con 0x7f85b0104e20 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85ae59c700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b01998e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fb0 tx=0x7f85a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:28.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a0003930 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f85a0003a90 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b0108d70 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a0019610 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.973+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b0108f00 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.974+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f85a0019880 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.974+0000 7f85a7fff700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f859c0385f0 0x7f859c03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.974+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b01069b0 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.974+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f85a004c660 con 0x7f85b0104e20 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.975+0000 7f85a5d9b700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f859c0385f0 0x7f859c03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.975+0000 7f85a5d9b700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f859c0385f0 0x7f859c03aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8598006fd0 tx=0x7f8598006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:27.980+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f85b01069b0 con 0x7f85b0104e20 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.191+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f85b01069b0 con 0x7f859c0385f0 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.192+0000 7f85a7fff700 1 -- 172.21.15.31:0/1014560238 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f85b01069b0 con 0x7f859c0385f0 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f859c0385f0 msgr2=0x7f859c03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f859c0385f0 0x7f859c03aaa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8598006fd0 tx=0x7f8598006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 msgr2=0x7f85b01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b01998e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fb0 tx=0x7f85a0011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 shutdown_connections 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f859c0385f0 0x7f859c03aaa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 --2- 172.21.15.31:0/1014560238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b0104e20 0x7f85b01998e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:28.578 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.194+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 >> 172.21.15.31:0/1014560238 conn(0x7f85b00ff650 msgr2=0x7f85b0104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:28.578 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.195+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 shutdown_connections 2023-12-14T18:20:28.578 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.195+0000 7f85b4f4b700 1 -- 172.21.15.31:0/1014560238 wait complete. 2023-12-14T18:20:28.578 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2023-12-14T18:20:28.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:28 smithi031 ceph-mon[42085]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:28.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:28 smithi031 ceph-mon[42085]: mgrmap e8: x(active, since 2s) 2023-12-14T18:20:29.767 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-mon[42085]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: Generating public/private rsa key pair. 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: Your identification has been saved in /tmp/tmp_5gp7kfk/key. 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: Your public key has been saved in /tmp/tmp_5gp7kfk/key.pub. 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: The key fingerprint is: 2023-12-14T18:20:29.768 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: SHA256:7/hmitRP5hrt2a0CHAiZz3OjIGk5BMRrqNMFK678MlM ceph-37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: The key's randomart image is: 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: +---[RSA 3072]----+ 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |+o o | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: | .o + | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |...= + . | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |ooB o = + | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |++ + . =So | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |o..E ..oo | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |o.. . o.= | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: |.= . . X+o . | 2023-12-14T18:20:29.769 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: | =. . +*B.o.. | 2023-12-14T18:20:29.770 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: +----[SHA256]-----+ 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.953+0000 7f1307622700 1 Processor -- start 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f1307622700 1 -- start start 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f1307622700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f1307622700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1300105300 con 0x7f1300104920 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f13053be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f13053be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38966/0 (socket says 172.21.15.31:38966) 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f13053be700 1 -- 172.21.15.31:0/2295472020 learned_addr learned my addr 172.21.15.31:0/2295472020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:29.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.954+0000 7f13053be700 1 -- 172.21.15.31:0/2295472020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1300105b20 con 0x7f1300104920 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f13053be700 1 --2- 172.21.15.31:0/2295472020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300104d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098a0 tx=0x7f12fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74869bca90a367ce server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f12f7fff700 1 -- 172.21.15.31:0/2295472020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc004160 con 0x7f1300104920 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f12f7fff700 1 -- 172.21.15.31:0/2295472020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12fc0042c0 con 0x7f1300104920 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f12f7fff700 1 -- 172.21.15.31:0/2295472020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc011a00 con 0x7f1300104920 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f1307622700 1 -- 172.21.15.31:0/2295472020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 msgr2=0x7f1300104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f1307622700 1 --2- 172.21.15.31:0/2295472020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300104d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0098a0 tx=0x7f12fc009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f1307622700 1 -- 172.21.15.31:0/2295472020 shutdown_connections 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f1307622700 1 --2- 172.21.15.31:0/2295472020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300104d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.955+0000 7f1307622700 1 -- 172.21.15.31:0/2295472020 >> 172.21.15.31:0/2295472020 conn(0x7f13000ffed0 msgr2=0x7f1300102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:29.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.956+0000 7f1307622700 1 -- 172.21.15.31:0/2295472020 shutdown_connections 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.956+0000 7f1307622700 1 -- 172.21.15.31:0/2295472020 wait complete. 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.956+0000 7f1307622700 1 Processor -- start 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.956+0000 7f1307622700 1 -- start start 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.956+0000 7f1307622700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300079b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.956+0000 7f1307622700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130019c620 con 0x7f1300104920 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f13053be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300079b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f13053be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300079b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38978/0 (socket says 172.21.15.31:38978) 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f13053be700 1 -- 172.21.15.31:0/2136677067 learned_addr learned my addr 172.21.15.31:0/2136677067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f13053be700 1 -- 172.21.15.31:0/2136677067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc009550 con 0x7f1300104920 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f13053be700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300079b40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009fb0 tx=0x7f12fc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc003930 con 0x7f1300104920 2023-12-14T18:20:29.840 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12fc003a90 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1300108d50 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12fc0195e0 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.957+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1300108ee0 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.958+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f12fc01a9d0 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.958+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1300106990 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.959+0000 7f12f67fc700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f12ec0386c0 0x7f12ec03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.959+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f12fc04c7b0 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.959+0000 7f1304bbd700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f12ec0386c0 0x7f12ec03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.959+0000 7f1304bbd700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f12ec0386c0 0x7f12ec03ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:28.964+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1300106990 con 0x7f1300104920 2023-12-14T18:20:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.175+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1300106990 con 0x7f12ec0386c0 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.481+0000 7f12f67fc700 1 -- 172.21.15.31:0/2136677067 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1300106990 con 0x7f12ec0386c0 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f12ec0386c0 msgr2=0x7f12ec03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f12ec0386c0 0x7f12ec03ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12f0006fd0 tx=0x7f12f0006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 msgr2=0x7f1300079b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300079b40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f12fc009fb0 tx=0x7f12fc011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 shutdown_connections 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f12ec0386c0 0x7f12ec03ab70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 --2- 172.21.15.31:0/2136677067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1300104920 0x7f1300079b40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.483+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 >> 172.21.15.31:0/2136677067 conn(0x7f13000ffed0 msgr2=0x7f1300101540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.484+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 shutdown_connections 2023-12-14T18:20:29.842 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:29.484+0000 7f1307622700 1 -- 172.21.15.31:0/2136677067 wait complete. 2023-12-14T18:20:30.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:30 smithi031 ceph-mon[42085]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:30.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:30 smithi031 ceph-mon[42085]: Generating ssh key... 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.203+0000 7f76c8416700 1 Processor -- start 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.203+0000 7f76c8416700 1 -- start start 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.204+0000 7f76c8416700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c0104730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.204+0000 7f76c8416700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c0079dc0 con 0x7f76c0104320 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.204+0000 7f76c61b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c0104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.204+0000 7f76c61b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c0104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38984/0 (socket says 172.21.15.31:38984) 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.204+0000 7f76c61b2700 1 -- 172.21.15.31:0/181966183 learned_addr learned my addr 172.21.15.31:0/181966183 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:30.806 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.204+0000 7f76c61b2700 1 -- 172.21.15.31:0/181966183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c0079f00 con 0x7f76c0104320 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c61b2700 1 --2- 172.21.15.31:0/181966183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c0104730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098a0 tx=0x7f76bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebb691d28f89755 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c51b0700 1 -- 172.21.15.31:0/181966183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76bc004160 con 0x7f76c0104320 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c51b0700 1 -- 172.21.15.31:0/181966183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76bc0042c0 con 0x7f76c0104320 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c51b0700 1 -- 172.21.15.31:0/181966183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76bc011a00 con 0x7f76c0104320 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c8416700 1 -- 172.21.15.31:0/181966183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 msgr2=0x7f76c0104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c8416700 1 --2- 172.21.15.31:0/181966183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c0104730 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098a0 tx=0x7f76bc009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c8416700 1 -- 172.21.15.31:0/181966183 shutdown_connections 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c8416700 1 --2- 172.21.15.31:0/181966183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c0104730 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.205+0000 7f76c8416700 1 -- 172.21.15.31:0/181966183 >> 172.21.15.31:0/181966183 conn(0x7f76c00ffed0 msgr2=0x7f76c0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.206+0000 7f76c8416700 1 -- 172.21.15.31:0/181966183 shutdown_connections 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.206+0000 7f76c8416700 1 -- 172.21.15.31:0/181966183 wait complete. 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.206+0000 7f76c8416700 1 Processor -- start 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.206+0000 7f76c8416700 1 -- start start 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.206+0000 7f76c8416700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.206+0000 7f76c8416700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c019be90 con 0x7f76c0104320 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c61b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c61b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c019b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38996/0 (socket says 172.21.15.31:38996) 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c61b2700 1 -- 172.21.15.31:0/1242770395 learned_addr learned my addr 172.21.15.31:0/1242770395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c61b2700 1 -- 172.21.15.31:0/1242770395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc009550 con 0x7f76c0104320 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c61b2700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c019b950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006af0 tx=0x7f76bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76bc003a40 con 0x7f76c0104320 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f76bc003ba0 con 0x7f76c0104320 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c0105ae0 con 0x7f76c0104320 2023-12-14T18:20:30.812 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76bc01a6e0 con 0x7f76c0104320 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.207+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c0105950 con 0x7f76c0104320 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.208+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f76bc024a60 con 0x7f76c0104320 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.208+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c0195260 con 0x7f76c0104320 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.208+0000 7f76b77fe700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f76ac038760 0x7f76ac03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.209+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f76bc04c7c0 con 0x7f76c0104320 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.209+0000 7f76c59b1700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f76ac038760 0x7f76ac03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.209+0000 7f76c59b1700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f76ac038760 0x7f76ac03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006fd0 tx=0x7f76b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.214+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f76bc024430 con 0x7f76c0104320 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.423+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f76c0060f30 con 0x7f76ac038760 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.424+0000 7f76b77fe700 1 -- 172.21.15.31:0/1242770395 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f76c0060f30 con 0x7f76ac038760 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f76ac038760 msgr2=0x7f76ac03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f76ac038760 0x7f76ac03ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76b0006fd0 tx=0x7f76b0006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 msgr2=0x7f76c019b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c019b950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006af0 tx=0x7f76bc011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 shutdown_connections 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f76ac038760 0x7f76ac03ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 --2- 172.21.15.31:0/1242770395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76c0104320 0x7f76c019b950 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 >> 172.21.15.31:0/1242770395 conn(0x7f76c00ffed0 msgr2=0x7f76c018fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 shutdown_connections 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:30.426+0000 7f76c8416700 1 -- 172.21.15.31:0/1242770395 wait complete. 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2023-12-14T18:20:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2023-12-14T18:20:31.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:31 smithi031 ceph-mon[42085]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:32.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:32 smithi031 ceph-mon[42085]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:33.805 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:33 smithi031 ceph-mon[42085]: Deploying cephadm binary to smithi031 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb7275e700 1 Processor -- start 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb7275e700 1 -- start start 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb7275e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb7275e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c079e20 con 0x7fbb6c1064d0 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39000/0 (socket says 172.21.15.31:39000) 2023-12-14T18:20:33.920 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.191+0000 7fbb6bfff700 1 -- 172.21.15.31:0/3970391074 learned_addr learned my addr 172.21.15.31:0/3970391074 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.192+0000 7fbb6bfff700 1 -- 172.21.15.31:0/3970391074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c108df0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.192+0000 7fbb6bfff700 1 --2- 172.21.15.31:0/3970391074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c1088b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098a0 tx=0x7fbb5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288ecfe24da96a3a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.192+0000 7fbb6affd700 1 -- 172.21.15.31:0/3970391074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c004160 con 0x7fbb6c1064d0 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.192+0000 7fbb6affd700 1 -- 172.21.15.31:0/3970391074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb5c0042c0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.192+0000 7fbb6affd700 1 -- 172.21.15.31:0/3970391074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c011a00 con 0x7fbb6c1064d0 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 -- 172.21.15.31:0/3970391074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 msgr2=0x7fbb6c1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 --2- 172.21.15.31:0/3970391074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c1088b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098a0 tx=0x7fbb5c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 -- 172.21.15.31:0/3970391074 shutdown_connections 2023-12-14T18:20:33.921 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 --2- 172.21.15.31:0/3970391074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c1088b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 -- 172.21.15.31:0/3970391074 >> 172.21.15.31:0/3970391074 conn(0x7fbb6c0ffed0 msgr2=0x7fbb6c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 -- 172.21.15.31:0/3970391074 shutdown_connections 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.193+0000 7fbb7275e700 1 -- 172.21.15.31:0/3970391074 wait complete. 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb7275e700 1 Processor -- start 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb7275e700 1 -- start start 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb7275e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb7275e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c199cc0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39012/0 (socket says 172.21.15.31:39012) 2023-12-14T18:20:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb6bfff700 1 -- 172.21.15.31:0/2954882063 learned_addr learned my addr 172.21.15.31:0/2954882063 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.194+0000 7fbb6bfff700 1 -- 172.21.15.31:0/2954882063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c009550 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.195+0000 7fbb6bfff700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c199780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009fb0 tx=0x7fbb5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.195+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c003930 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.195+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb5c003a90 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.195+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb5c019610 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.195+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c109d20 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.195+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c109b90 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.196+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbb5c0198d0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.196+0000 7fbb697fa700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fbb540386c0 0x7fbb5403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.196+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbb5c04c9f0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.196+0000 7fbb6b7fe700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fbb540386c0 0x7fbb5403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.196+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c109eb0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.197+0000 7fbb6b7fe700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fbb540386c0 0x7fbb5403ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb60006fd0 tx=0x7fbb60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.201+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbb5c019bd0 con 0x7fbb6c1064d0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:31.415+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb6c102190 con 0x7fbb540386c0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.607+0000 7fbb697fa700 1 -- 172.21.15.31:0/2954882063 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fbb6c102190 con 0x7fbb540386c0 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fbb540386c0 msgr2=0x7fbb5403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fbb540386c0 0x7fbb5403ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbb60006fd0 tx=0x7fbb60006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 msgr2=0x7fbb6c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:33.924 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c199780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c009fb0 tx=0x7fbb5c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 shutdown_connections 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fbb540386c0 0x7fbb5403ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 --2- 172.21.15.31:0/2954882063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c1064d0 0x7fbb6c199780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 >> 172.21.15.31:0/2954882063 conn(0x7fbb6c0ffed0 msgr2=0x7fbb6c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 shutdown_connections 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:33.610+0000 7fbb7275e700 1 -- 172.21.15.31:0/2954882063 wait complete. 2023-12-14T18:20:33.925 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mon service... 2023-12-14T18:20:34.933 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:34 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:34.934 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:34 smithi031 ceph-mon[42085]: Added host smithi031 2023-12-14T18:20:34.934 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:34 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:34.934 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:34 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:35.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.320+0000 7fd0fefff700 1 Processor -- start 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.320+0000 7fd0fefff700 1 -- start start 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.320+0000 7fd0fefff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.320+0000 7fd0fefff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8079e60 con 0x7fd0f8106490 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0fcd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0fcd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39016/0 (socket says 172.21.15.31:39016) 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0fcd9b700 1 -- 172.21.15.31:0/4188927507 learned_addr learned my addr 172.21.15.31:0/4188927507 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0fcd9b700 1 -- 172.21.15.31:0/4188927507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8108db0 con 0x7fd0f8106490 2023-12-14T18:20:35.012 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0fcd9b700 1 --2- 172.21.15.31:0/4188927507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8108870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0098a0 tx=0x7fd0ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db32b83450ef8e02 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0f77fe700 1 -- 172.21.15.31:0/4188927507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ec004160 con 0x7fd0f8106490 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0f77fe700 1 -- 172.21.15.31:0/4188927507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0ec0042c0 con 0x7fd0f8106490 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.321+0000 7fd0f77fe700 1 -- 172.21.15.31:0/4188927507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ec011a00 con 0x7fd0f8106490 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 -- 172.21.15.31:0/4188927507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 msgr2=0x7fd0f8108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 --2- 172.21.15.31:0/4188927507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8108870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0098a0 tx=0x7fd0ec009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 -- 172.21.15.31:0/4188927507 shutdown_connections 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 --2- 172.21.15.31:0/4188927507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8108870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 -- 172.21.15.31:0/4188927507 >> 172.21.15.31:0/4188927507 conn(0x7fd0f80ffed0 msgr2=0x7fd0f8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 -- 172.21.15.31:0/4188927507 shutdown_connections 2023-12-14T18:20:35.013 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.322+0000 7fd0fefff700 1 -- 172.21.15.31:0/4188927507 wait complete. 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fefff700 1 Processor -- start 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fefff700 1 -- start start 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fefff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fefff700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8199ca0 con 0x7fd0f8106490 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fcd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fcd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39022/0 (socket says 172.21.15.31:39022) 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fcd9b700 1 -- 172.21.15.31:0/3293230017 learned_addr learned my addr 172.21.15.31:0/3293230017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fcd9b700 1 -- 172.21.15.31:0/3293230017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec009550 con 0x7fd0f8106490 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.323+0000 7fd0fcd9b700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8199760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009fb0 tx=0x7fd0ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.324+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ec003930 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.324+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0ec003a90 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.324+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0ec019610 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.324+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f8109d40 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.324+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f8109bb0 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.325+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f8062120 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.325+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fd0ec01a9d0 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.326+0000 7fd0f5ffb700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd0e80386c0 0x7fd0e803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.326+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd0ec04cc40 con 0x7fd0f8106490 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.326+0000 7fd0f7fff700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd0e80386c0 0x7fd0e803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:35.015 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.326+0000 7fd0f7fff700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd0e80386c0 0x7fd0e803ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006fd0 tx=0x7fd0e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.330+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd0ec00bdb0 con 0x7fd0f8106490 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.563+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd0f81013d0 con 0x7fd0e80386c0 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.569+0000 7fd0f5ffb700 1 -- 172.21.15.31:0/3293230017 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd0f81013d0 con 0x7fd0e80386c0 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd0e80386c0 msgr2=0x7fd0e803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd0e80386c0 0x7fd0e803ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006fd0 tx=0x7fd0e4006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 msgr2=0x7fd0f8199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8199760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009fb0 tx=0x7fd0ec011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 shutdown_connections 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd0e80386c0 0x7fd0e803ab70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 --2- 172.21.15.31:0/3293230017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0f8106490 0x7fd0f8199760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 >> 172.21.15.31:0/3293230017 conn(0x7fd0f80ffed0 msgr2=0x7fd0f8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:35.016 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 shutdown_connections 2023-12-14T18:20:35.017 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:34.571+0000 7fd0fefff700 1 -- 172.21.15.31:0/3293230017 wait complete. 2023-12-14T18:20:35.017 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying unmanaged mgr service... 2023-12-14T18:20:35.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:35 smithi031 ceph-mon[42085]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:35.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:35 smithi031 ceph-mon[42085]: Saving service mon spec with placement count:5 2023-12-14T18:20:35.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:35 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.413+0000 7f8ca4129700 1 Processor -- start 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.413+0000 7f8ca4129700 1 -- start start 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.413+0000 7f8ca4129700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.413+0000 7f8ca4129700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c079e60 con 0x7f8c9c106490 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca1ec5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca1ec5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46590/0 (socket says 172.21.15.31:46590) 2023-12-14T18:20:36.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca1ec5700 1 -- 172.21.15.31:0/2513765159 learned_addr learned my addr 172.21.15.31:0/2513765159 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca1ec5700 1 -- 172.21.15.31:0/2513765159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c108db0 con 0x7f8c9c106490 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca1ec5700 1 --2- 172.21.15.31:0/2513765159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c108870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098a0 tx=0x7f8c98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a527bf33191a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca0ec3700 1 -- 172.21.15.31:0/2513765159 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c98004160 con 0x7f8c9c106490 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca0ec3700 1 -- 172.21.15.31:0/2513765159 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c980042c0 con 0x7f8c9c106490 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.414+0000 7f8ca0ec3700 1 -- 172.21.15.31:0/2513765159 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c98011a00 con 0x7f8c9c106490 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 -- 172.21.15.31:0/2513765159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 msgr2=0x7f8c9c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 --2- 172.21.15.31:0/2513765159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c108870 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c980098a0 tx=0x7f8c98009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 -- 172.21.15.31:0/2513765159 shutdown_connections 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 --2- 172.21.15.31:0/2513765159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c108870 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 -- 172.21.15.31:0/2513765159 >> 172.21.15.31:0/2513765159 conn(0x7f8c9c0ffed0 msgr2=0x7f8c9c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 -- 172.21.15.31:0/2513765159 shutdown_connections 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.415+0000 7f8ca4129700 1 -- 172.21.15.31:0/2513765159 wait complete. 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca4129700 1 Processor -- start 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca4129700 1 -- start start 2023-12-14T18:20:36.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca4129700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca4129700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c106330 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca1ec5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca1ec5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46606/0 (socket says 172.21.15.31:46606) 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca1ec5700 1 -- 172.21.15.31:0/1764329765 learned_addr learned my addr 172.21.15.31:0/1764329765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.416+0000 7f8ca1ec5700 1 -- 172.21.15.31:0/1764329765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98009550 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.417+0000 7f8ca1ec5700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c105df0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006af0 tx=0x7f8c98011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.417+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c98003a40 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.417+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c98003ba0 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.417+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c109d40 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.417+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c9801a6e0 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.417+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c109bb0 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.418+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8c9801a840 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.418+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c062120 con 0x7f8c9c106490 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.418+0000 7f8c92ffd700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f8c88038760 0x7f8c8803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:36.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.418+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8c9804c940 con 0x7f8c9c106490 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.418+0000 7f8ca16c4700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f8c88038760 0x7f8c8803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.419+0000 7f8ca16c4700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f8c88038760 0x7f8c8803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c006fd0 tx=0x7f8c8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.423+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8c9801aaf0 con 0x7f8c9c106490 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.493+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c9804c350 con 0x7f8c9c106490 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.653+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8c9c102190 con 0x7f8c88038760 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.658+0000 7f8c92ffd700 1 -- 172.21.15.31:0/1764329765 <== mgr.14120 v2:172.21.15.31:6800/2330625991 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f8c9c102190 con 0x7f8c88038760 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f8c88038760 msgr2=0x7f8c8803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f8c88038760 0x7f8c8803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c006fd0 tx=0x7f8c8c006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 msgr2=0x7f8c9c105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c105df0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c98006af0 tx=0x7f8c98011ad0 comp rx=0 tx=0).stop 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 shutdown_connections 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f8c88038760 0x7f8c8803ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 --2- 172.21.15.31:0/1764329765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c9c106490 0x7f8c9c105df0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:36.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 >> 172.21.15.31:0/1764329765 conn(0x7f8c9c0ffed0 msgr2=0x7f8c9c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:36.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 shutdown_connections 2023-12-14T18:20:36.054 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:35.660+0000 7f8ca4129700 1 -- 172.21.15.31:0/1764329765 wait complete. 2023-12-14T18:20:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:36 smithi031 ceph-mon[42085]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:36 smithi031 ceph-mon[42085]: Saving service mgr spec with placement count:2 2023-12-14T18:20:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:36 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:36 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:36 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3268886109' entity='client.admin' 2023-12-14T18:20:37.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.375+0000 7f96649a5700 1 Processor -- start 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.375+0000 7f96649a5700 1 -- start start 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f96649a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f9660108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f96649a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660079e60 con 0x7f9660106490 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f9660108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f9660108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46612/0 (socket says 172.21.15.31:46612) 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965e59c700 1 -- 172.21.15.31:0/2211857664 learned_addr learned my addr 172.21.15.31:0/2211857664 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965e59c700 1 -- 172.21.15.31:0/2211857664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9660108db0 con 0x7f9660106490 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965e59c700 1 --2- 172.21.15.31:0/2211857664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f9660108870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96480098a0 tx=0x7f9648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985679a64941c56f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965d59a700 1 -- 172.21.15.31:0/2211857664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648004160 con 0x7f9660106490 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.376+0000 7f965d59a700 1 -- 172.21.15.31:0/2211857664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96480042c0 con 0x7f9660106490 2023-12-14T18:20:37.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f965d59a700 1 -- 172.21.15.31:0/2211857664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96480119b0 con 0x7f9660106490 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 -- 172.21.15.31:0/2211857664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 msgr2=0x7f9660108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 --2- 172.21.15.31:0/2211857664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f9660108870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96480098a0 tx=0x7f9648009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 -- 172.21.15.31:0/2211857664 shutdown_connections 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 --2- 172.21.15.31:0/2211857664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f9660108870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 -- 172.21.15.31:0/2211857664 >> 172.21.15.31:0/2211857664 conn(0x7f96600ffed0 msgr2=0x7f9660102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 -- 172.21.15.31:0/2211857664 shutdown_connections 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.377+0000 7f96649a5700 1 -- 172.21.15.31:0/2211857664 wait complete. 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f96649a5700 1 Processor -- start 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f96649a5700 1 -- start start 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f96649a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f966019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f96649a5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966019be70 con 0x7f9660106490 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f965e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f966019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f965e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f966019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46628/0 (socket says 172.21.15.31:46628) 2023-12-14T18:20:37.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.378+0000 7f965e59c700 1 -- 172.21.15.31:0/3268886109 learned_addr learned my addr 172.21.15.31:0/3268886109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f965e59c700 1 -- 172.21.15.31:0/3268886109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009550 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f965e59c700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f966019b930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9648003e60 tx=0x7f9648011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9648003970 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9648003ad0 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9660109d80 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96480195e0 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.379+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9660109bf0 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.380+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f9648019850 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.380+0000 7f96577fe700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f964c038710 0x7f964c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.380+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f964804c8e0 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.380+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9660109f10 con 0x7f9660106490 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.380+0000 7f965dd9b700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f964c038710 0x7f964c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:37.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.381+0000 7f965dd9b700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f964c038710 0x7f964c03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9650006fd0 tx=0x7f9650006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.385+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9648019b50 con 0x7f9660106490 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.602+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f966019c3b0 con 0x7f9660106490 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.606+0000 7f96577fe700 1 -- 172.21.15.31:0/3268886109 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f966019c3b0 con 0x7f9660106490 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f964c038710 msgr2=0x7f964c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f964c038710 0x7f964c03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9650006fd0 tx=0x7f9650006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 msgr2=0x7f966019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f966019b930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9648003e60 tx=0x7f9648011aa0 comp rx=0 tx=0).stop 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 shutdown_connections 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f964c038710 0x7f964c03abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 --2- 172.21.15.31:0/3268886109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9660106490 0x7f966019b930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 >> 172.21.15.31:0/3268886109 conn(0x7f96600ffed0 msgr2=0x7f9660101560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 shutdown_connections 2023-12-14T18:20:37.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:36.609+0000 7f96649a5700 1 -- 172.21.15.31:0/3268886109 wait complete. 2023-12-14T18:20:38.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:38 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:38.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2718463642' entity='client.admin' 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.215+0000 7f4c88746700 1 Processor -- start 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.215+0000 7f4c88746700 1 -- start start 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.216+0000 7f4c88746700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8007a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.216+0000 7f4c88746700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8007a7b0 con 0x7f4c8007bd70 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.216+0000 7f4c864e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8007a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.216+0000 7f4c864e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8007a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46634/0 (socket says 172.21.15.31:46634) 2023-12-14T18:20:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.216+0000 7f4c864e2700 1 -- 172.21.15.31:0/4148410836 learned_addr learned my addr 172.21.15.31:0/4148410836 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.216+0000 7f4c864e2700 1 -- 172.21.15.31:0/4148410836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8007a8f0 con 0x7f4c8007bd70 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.217+0000 7f4c864e2700 1 --2- 172.21.15.31:0/4148410836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8007a270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0098a0 tx=0x7f4c7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5737ef5ed875a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.217+0000 7f4c854e0700 1 -- 172.21.15.31:0/4148410836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c7c004160 con 0x7f4c8007bd70 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.217+0000 7f4c854e0700 1 -- 172.21.15.31:0/4148410836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c7c0042c0 con 0x7f4c8007bd70 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.217+0000 7f4c854e0700 1 -- 172.21.15.31:0/4148410836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c7c0119b0 con 0x7f4c8007bd70 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.217+0000 7f4c88746700 1 -- 172.21.15.31:0/4148410836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 msgr2=0x7f4c8007a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.217+0000 7f4c88746700 1 --2- 172.21.15.31:0/4148410836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8007a270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0098a0 tx=0x7f4c7c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:38.829 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 -- 172.21.15.31:0/4148410836 shutdown_connections 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 --2- 172.21.15.31:0/4148410836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8007a270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 -- 172.21.15.31:0/4148410836 >> 172.21.15.31:0/4148410836 conn(0x7f4c80101170 msgr2=0x7f4c80103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 -- 172.21.15.31:0/4148410836 shutdown_connections 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 -- 172.21.15.31:0/4148410836 wait complete. 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 Processor -- start 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.218+0000 7f4c88746700 1 -- start start 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c88746700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c88746700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8019bfb0 con 0x7f4c8007bd70 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c864e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c864e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8019ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46644/0 (socket says 172.21.15.31:46644) 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c864e2700 1 -- 172.21.15.31:0/2718463642 learned_addr learned my addr 172.21.15.31:0/2718463642 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c864e2700 1 -- 172.21.15.31:0/2718463642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c009550 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c864e2700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8019ba70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000c00 tx=0x7f4c7c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c7c0039f0 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c7c003b50 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.219+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8010bfc0 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.220+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c7c01a780 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.220+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8010be30 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.221+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8010c150 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.221+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4c7c01aa40 con 0x7f4c8007bd70 2023-12-14T18:20:38.831 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.221+0000 7f4c777fe700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f4c6c038710 0x7f4c6c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.221+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4c7c04c970 con 0x7f4c8007bd70 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.221+0000 7f4c85ce1700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f4c6c038710 0x7f4c6c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.222+0000 7f4c85ce1700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f4c6c038710 0x7f4c6c03abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006fd0 tx=0x7f4c70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.226+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c7c023990 con 0x7f4c8007bd70 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.439+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f4c80062120 con 0x7f4c8007bd70 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.444+0000 7f4c777fe700 1 -- 172.21.15.31:0/2718463642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f4c7c0233a0 con 0x7f4c8007bd70 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.446+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f4c6c038710 msgr2=0x7f4c6c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.446+0000 7f4c88746700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f4c6c038710 0x7f4c6c03abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4c70006fd0 tx=0x7f4c70006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.446+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 msgr2=0x7f4c8019ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.446+0000 7f4c88746700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8019ba70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000c00 tx=0x7f4c7c011a30 comp rx=0 tx=0).stop 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.447+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 shutdown_connections 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.447+0000 7f4c88746700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f4c6c038710 0x7f4c6c03abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.447+0000 7f4c88746700 1 --2- 172.21.15.31:0/2718463642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c8007bd70 0x7f4c8019ba70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.447+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 >> 172.21.15.31:0/2718463642 conn(0x7f4c80101170 msgr2=0x7f4c80102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.447+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 shutdown_connections 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:38.447+0000 7f4c88746700 1 -- 172.21.15.31:0/2718463642 wait complete. 2023-12-14T18:20:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2023-12-14T18:20:39.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:39 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: ignoring --setuser ceph since I am not root 2023-12-14T18:20:39.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:39 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: ignoring --setgroup ceph since I am not root 2023-12-14T18:20:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:39 smithi031 ceph-mon[42085]: from='mgr.14120 172.21.15.31:0/376641446' entity='mgr.x' 2023-12-14T18:20:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:39 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3062962182' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.159+0000 7f10d44e4700 1 Processor -- start 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.159+0000 7f10d44e4700 1 -- start start 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.159+0000 7f10d44e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc1088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.159+0000 7f10d44e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc079e00 con 0x7f10cc1064f0 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d2280700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc1088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d2280700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc1088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46650/0 (socket says 172.21.15.31:46650) 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d2280700 1 -- 172.21.15.31:0/684303216 learned_addr learned my addr 172.21.15.31:0/684303216 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d2280700 1 -- 172.21.15.31:0/684303216 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc108e10 con 0x7f10cc1064f0 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d2280700 1 --2- 172.21.15.31:0/684303216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc1088d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098a0 tx=0x7f10c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7291ebe9c9e8ad68 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:40.293 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d127e700 1 -- 172.21.15.31:0/684303216 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10c8004160 con 0x7f10cc1064f0 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d127e700 1 -- 172.21.15.31:0/684303216 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c80042c0 con 0x7f10cc1064f0 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.160+0000 7f10d127e700 1 -- 172.21.15.31:0/684303216 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10c80119b0 con 0x7f10cc1064f0 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 -- 172.21.15.31:0/684303216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 msgr2=0x7f10cc1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 --2- 172.21.15.31:0/684303216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc1088d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f10c80098a0 tx=0x7f10c8009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 -- 172.21.15.31:0/684303216 shutdown_connections 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 --2- 172.21.15.31:0/684303216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc1088d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 -- 172.21.15.31:0/684303216 >> 172.21.15.31:0/684303216 conn(0x7f10cc0ffed0 msgr2=0x7f10cc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 -- 172.21.15.31:0/684303216 shutdown_connections 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.161+0000 7f10d44e4700 1 -- 172.21.15.31:0/684303216 wait complete. 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d44e4700 1 Processor -- start 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d44e4700 1 -- start start 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d44e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc105e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:40.294 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d44e4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc106390 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d2280700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc105e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d2280700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc105e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46658/0 (socket says 172.21.15.31:46658) 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.162+0000 7f10d2280700 1 -- 172.21.15.31:0/3062962182 learned_addr learned my addr 172.21.15.31:0/3062962182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10d2280700 1 -- 172.21.15.31:0/3062962182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c8009550 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10d2280700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc105e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10c8011980 tx=0x7f10c8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10c8003a60 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10c8003bc0 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10c801a670 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10cc109d80 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.163+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10cc109bf0 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.164+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10cc109f10 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.164+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f10c8024a00 con 0x7f10cc1064f0 2023-12-14T18:20:40.295 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.165+0000 7f10c37fe700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f10b8038710 0x7f10b803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.165+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f10c804c880 con 0x7f10cc1064f0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.165+0000 7f10d1a7f700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f10b8038710 0x7f10b803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.165+0000 7f10d1a7f700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f10b8038710 0x7f10b803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10bc006fd0 tx=0x7f10bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.170+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f10c8024440 con 0x7f10cc1064f0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.455+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f10cc105280 con 0x7f10cc1064f0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.879+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f10c80246d0 con 0x7f10cc1064f0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.881+0000 7f10c37fe700 1 -- 172.21.15.31:0/3062962182 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f10c802d910 con 0x7f10cc1064f0 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f10b8038710 msgr2=0x7f10b803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f10b8038710 0x7f10b803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10bc006fd0 tx=0x7f10bc006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 msgr2=0x7f10cc105e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc105e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10c8011980 tx=0x7f10c8011a60 comp rx=0 tx=0).stop 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 shutdown_connections 2023-12-14T18:20:40.296 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f10b8038710 0x7f10b803abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:40.297 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 --2- 172.21.15.31:0/3062962182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10cc1064f0 0x7f10cc105e50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:40.297 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 >> 172.21.15.31:0/3062962182 conn(0x7f10cc0ffed0 msgr2=0x7f10cc101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:40.297 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 shutdown_connections 2023-12-14T18:20:40.297 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:39.885+0000 7f10d44e4700 1 -- 172.21.15.31:0/3062962182 wait complete. 2023-12-14T18:20:40.428 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:40 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:40.307+0000 7f2c51639200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:20:40.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:40 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:40.426+0000 7f2c51639200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:20:40.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:40 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:40.550+0000 7f2c51639200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:20:40.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:40 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:40.651+0000 7f2c51639200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:20:41.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:40 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3062962182' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-14T18:20:41.178 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:40 smithi031 ceph-mon[42085]: mgrmap e9: x(active, since 14s) 2023-12-14T18:20:41.346 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.709+0000 7f19b163f700 1 Processor -- start 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.709+0000 7f19b163f700 1 -- start start 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19b163f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac105db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19b163f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac106380 con 0x7f19ac1059a0 2023-12-14T18:20:41.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac105db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac105db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46676/0 (socket says 172.21.15.31:46676) 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19aaffd700 1 -- 172.21.15.31:0/2087773451 learned_addr learned my addr 172.21.15.31:0/2087773451 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19aaffd700 1 -- 172.21.15.31:0/2087773451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac106ba0 con 0x7f19ac1059a0 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.710+0000 7f19aaffd700 1 --2- 172.21.15.31:0/2087773451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac105db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f199c0098a0 tx=0x7f199c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59449841e7bf2b3e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.711+0000 7f19a9ffb700 1 -- 172.21.15.31:0/2087773451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f199c004160 con 0x7f19ac1059a0 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.711+0000 7f19a9ffb700 1 -- 172.21.15.31:0/2087773451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f199c0042c0 con 0x7f19ac1059a0 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.711+0000 7f19a9ffb700 1 -- 172.21.15.31:0/2087773451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f199c0119b0 con 0x7f19ac1059a0 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.711+0000 7f19b163f700 1 -- 172.21.15.31:0/2087773451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 msgr2=0x7f19ac105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.711+0000 7f19b163f700 1 --2- 172.21.15.31:0/2087773451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac105db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f199c0098a0 tx=0x7f199c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.712+0000 7f19b163f700 1 -- 172.21.15.31:0/2087773451 shutdown_connections 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.712+0000 7f19b163f700 1 --2- 172.21.15.31:0/2087773451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac105db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:41.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.712+0000 7f19b163f700 1 -- 172.21.15.31:0/2087773451 >> 172.21.15.31:0/2087773451 conn(0x7f19ac100f90 msgr2=0x7f19ac1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.712+0000 7f19b163f700 1 -- 172.21.15.31:0/2087773451 shutdown_connections 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.712+0000 7f19b163f700 1 -- 172.21.15.31:0/2087773451 wait complete. 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19b163f700 1 Processor -- start 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19b163f700 1 -- start start 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19b163f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19b163f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac199c60 con 0x7f19ac1059a0 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46678/0 (socket says 172.21.15.31:46678) 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.713+0000 7f19aaffd700 1 -- 172.21.15.31:0/3964846351 learned_addr learned my addr 172.21.15.31:0/3964846351 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f19aaffd700 1 -- 172.21.15.31:0/3964846351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c009550 con 0x7f19ac1059a0 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f19aaffd700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac199720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f199c003e60 tx=0x7f199c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f199c003970 con 0x7f19ac1059a0 2023-12-14T18:20:41.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac109f60 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f199c003ad0 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f199c0195e0 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.714+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac109dd0 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.715+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac062190 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.716+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f199c0198a0 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.716+0000 7f1993fff700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 0x7f199403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.716+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f199c04caa0 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.716+0000 7f19aa7fc700 1 -- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 msgr2=0x7f199403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.716+0000 7f19aa7fc700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 0x7f199403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.725+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f199c00bc40 con 0x7f19ac1059a0 2023-12-14T18:20:41.357 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.917+0000 7f19aa7fc700 1 -- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 msgr2=0x7f199403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.917+0000 7f19aa7fc700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 0x7f199403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.992+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f19ac062120 con 0x7f19ac1059a0 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.993+0000 7f1993fff700 1 -- 172.21.15.31:0/3964846351 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f199c024e60 con 0x7f19ac1059a0 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 msgr2=0x7f199403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 0x7f199403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 msgr2=0x7f19ac199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac199720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f199c003e60 tx=0x7f199c011aa0 comp rx=0 tx=0).stop 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 shutdown_connections 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7f1994038710 0x7f199403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 --2- 172.21.15.31:0/3964846351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19ac1059a0 0x7f19ac199720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.995+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 >> 172.21.15.31:0/3964846351 conn(0x7f19ac100f90 msgr2=0x7f19ac102620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:41.358 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.996+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 shutdown_connections 2023-12-14T18:20:41.359 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:40.996+0000 7f19b163f700 1 -- 172.21.15.31:0/3964846351 wait complete. 2023-12-14T18:20:41.359 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2023-12-14T18:20:41.359 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2023-12-14T18:20:41.443 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:41 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:41.368+0000 7f2c51639200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:41.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:41 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:41.648+0000 7f2c51639200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:20:42.200 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:41 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3964846351' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-14T18:20:42.200 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:41 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:41.748+0000 7f2c51639200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:20:42.200 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:41 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:41.871+0000 7f2c51639200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:42.200 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:41 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:41.975+0000 7f2c51639200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:20:42.200 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:42 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:42.198+0000 7f2c51639200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:20:42.463 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:42 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:42.294+0000 7f2c51639200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:20:43.021 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:42 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:42.718+0000 7f2c51639200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:20:43.343 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:43.019+0000 7f2c51639200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:20:43.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:43.341+0000 7f2c51639200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:20:43.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:43.446+0000 7f2c51639200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:20:44.582 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:44.302+0000 7f2c51639200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:20:44.582 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:44.412+0000 7f2c51639200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:20:44.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:44.580+0000 7f2c51639200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:20:45.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:45.301+0000 7f2c51639200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:20:45.714 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:45.508+0000 7f2c51639200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:20:46.213 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:45.728+0000 7f2c51639200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:20:46.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:46.533+0000 7f2c51639200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:20:47.391 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:47.074+0000 7f2c51639200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:47.391 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:47.174+0000 7f2c51639200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:47.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:20:47.389+0000 7f2c51639200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: Active manager daemon x restarted 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: Activating manager daemon x 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: osdmap e3: 0 total, 0 up, 0 in 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: mgrmap e10: x(active, starting, since 0.00355164s) 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: Manager daemon x is now available 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:48.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:20:48.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:20:48.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:48 smithi031 ceph-mon[42085]: mgrmap e11: x(active, since 1.006s) 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd69a7d9700 1 Processor -- start 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd69a7d9700 1 -- start start 2023-12-14T18:20:48.768 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd69a7d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd69a7d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694079e60 con 0x7fd694106490 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd693fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd693fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46692/0 (socket says 172.21.15.31:46692) 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.713+0000 7fd693fff700 1 -- 172.21.15.31:0/297200203 learned_addr learned my addr 172.21.15.31:0/297200203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.714+0000 7fd693fff700 1 -- 172.21.15.31:0/297200203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd694108db0 con 0x7fd694106490 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.714+0000 7fd693fff700 1 --2- 172.21.15.31:0/297200203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694108870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6840098a0 tx=0x7fd684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd543a02143cfdf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.714+0000 7fd692ffd700 1 -- 172.21.15.31:0/297200203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd684004160 con 0x7fd694106490 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.714+0000 7fd692ffd700 1 -- 172.21.15.31:0/297200203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6840042c0 con 0x7fd694106490 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.714+0000 7fd692ffd700 1 -- 172.21.15.31:0/297200203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6840119b0 con 0x7fd694106490 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 -- 172.21.15.31:0/297200203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 msgr2=0x7fd694108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/297200203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694108870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6840098a0 tx=0x7fd684009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 -- 172.21.15.31:0/297200203 shutdown_connections 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/297200203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694108870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.769 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 -- 172.21.15.31:0/297200203 >> 172.21.15.31:0/297200203 conn(0x7fd6940ffed0 msgr2=0x7fd694102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 -- 172.21.15.31:0/297200203 shutdown_connections 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.715+0000 7fd69a7d9700 1 -- 172.21.15.31:0/297200203 wait complete. 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd69a7d9700 1 Processor -- start 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd69a7d9700 1 -- start start 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd69a7d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd69a7d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694106330 con 0x7fd694106490 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd693fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd693fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46698/0 (socket says 172.21.15.31:46698) 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.716+0000 7fd693fff700 1 -- 172.21.15.31:0/3410614330 learned_addr learned my addr 172.21.15.31:0/3410614330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd693fff700 1 -- 172.21.15.31:0/3410614330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd684009550 con 0x7fd694106490 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd693fff700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694105df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd684011980 tx=0x7fd684011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd684003a60 con 0x7fd694106490 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd694109d80 con 0x7fd694106490 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd684003bc0 con 0x7fd694106490 2023-12-14T18:20:48.770 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd68401a670 con 0x7fd694106490 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.717+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd694109bf0 con 0x7fd694106490 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.718+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd68401a930 con 0x7fd694106490 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.718+0000 7fd6917fa700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.718+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 --> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd67c03b320 con 0x7fd67c038760 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.718+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd68404c840 con 0x7fd694106490 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.718+0000 7fd6937fe700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 msgr2=0x7fd67c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.718+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.919+0000 7fd6937fe700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 msgr2=0x7fd67c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:41.919+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:42.320+0000 7fd6937fe700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 msgr2=0x7fd67c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:42.320+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:43.121+0000 7fd6937fe700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 msgr2=0x7fd67c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:43.121+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:44.722+0000 7fd6937fe700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 msgr2=0x7fd67c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2330625991 2023-12-14T18:20:48.771 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:44.722+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:47.404+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fd68402d960 con 0x7fd694106490 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:47.404+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 msgr2=0x7fd67c03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:47.404+0000 7fd6917fa700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.406+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fd68404dde0 con 0x7fd694106490 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.406+0000 7fd6917fa700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd67c03bff0 0x7fd67c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.406+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd67c03b320 con 0x7fd67c03bff0 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.406+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd67c03bff0 0x7fd67c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.407+0000 7fd6937fe700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd67c03bff0 0x7fd67c03e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd688003a10 tx=0x7fd6880092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.408+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fd67c03b320 con 0x7fd67c03bff0 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.414+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd694105990 con 0x7fd67c03bff0 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd6917fa700 1 -- 172.21.15.31:0/3410614330 <== mgr.14152 v2:172.21.15.31:6800/2688246057 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd694105990 con 0x7fd67c03bff0 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd67c03bff0 msgr2=0x7fd67c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd67c03bff0 0x7fd67c03e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd688003a10 tx=0x7fd6880092b0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 msgr2=0x7fd694105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:48.772 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694105df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd684011980 tx=0x7fd684011a60 comp rx=0 tx=0).stop 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 shutdown_connections 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd67c03bff0 0x7fd67c03e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd694106490 0x7fd694105df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 --2- 172.21.15.31:0/3410614330 >> [v2:172.21.15.31:6800/2330625991,v1:172.21.15.31:6801/2330625991] conn(0x7fd67c038760 0x7fd67c03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 >> 172.21.15.31:0/3410614330 conn(0x7fd6940ffed0 msgr2=0x7fd694101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 shutdown_connections 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:48.415+0000 7fd69a7d9700 1 -- 172.21.15.31:0/3410614330 wait complete. 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2023-12-14T18:20:48.773 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2023-12-14T18:20:49.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:48] ENGINE Bus STARTING 2023-12-14T18:20:49.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:48] ENGINE Serving on http://172.21.15.31:8765 2023-12-14T18:20:49.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-14T18:20:49.830 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-14T18:20:49.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:48] ENGINE Serving on https://172.21.15.31:7150 2023-12-14T18:20:49.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: [14/Dec/2023:18:20:48] ENGINE Bus STARTED 2023-12-14T18:20:49.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:49.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:49.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:49.831 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:49.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.160+0000 7f22cb46b700 1 Processor -- start 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.160+0000 7f22cb46b700 1 -- start start 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.160+0000 7f22cb46b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.160+0000 7f22cb46b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4105800 con 0x7f22c4104e20 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.161+0000 7f22c9207700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.161+0000 7f22c9207700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48114/0 (socket says 172.21.15.31:48114) 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.161+0000 7f22c9207700 1 -- 172.21.15.31:0/1219589025 learned_addr learned my addr 172.21.15.31:0/1219589025 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.161+0000 7f22c9207700 1 -- 172.21.15.31:0/1219589025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4105b40 con 0x7f22c4104e20 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.161+0000 7f22c9207700 1 --2- 172.21.15.31:0/1219589025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4105230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22c00098a0 tx=0x7f22c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9190a7ac99a01e38 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.161+0000 7f22bbfff700 1 -- 172.21.15.31:0/1219589025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22c0004160 con 0x7f22c4104e20 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22bbfff700 1 -- 172.21.15.31:0/1219589025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c00042c0 con 0x7f22c4104e20 2023-12-14T18:20:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22bbfff700 1 -- 172.21.15.31:0/1219589025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22c00119b0 con 0x7f22c4104e20 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22cb46b700 1 -- 172.21.15.31:0/1219589025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 msgr2=0x7f22c4105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22cb46b700 1 --2- 172.21.15.31:0/1219589025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4105230 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22c00098a0 tx=0x7f22c0009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22cb46b700 1 -- 172.21.15.31:0/1219589025 shutdown_connections 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22cb46b700 1 --2- 172.21.15.31:0/1219589025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4105230 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22cb46b700 1 -- 172.21.15.31:0/1219589025 >> 172.21.15.31:0/1219589025 conn(0x7f22c40ff650 msgr2=0x7f22c4101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.162+0000 7f22cb46b700 1 -- 172.21.15.31:0/1219589025 shutdown_connections 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.163+0000 7f22cb46b700 1 -- 172.21.15.31:0/1219589025 wait complete. 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.163+0000 7f22cb46b700 1 Processor -- start 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.163+0000 7f22cb46b700 1 -- start start 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.163+0000 7f22cb46b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.163+0000 7f22cb46b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4199cc0 con 0x7f22c4104e20 2023-12-14T18:20:49.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22c9207700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22c9207700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48126/0 (socket says 172.21.15.31:48126) 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22c9207700 1 -- 172.21.15.31:0/768331527 learned_addr learned my addr 172.21.15.31:0/768331527 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22c9207700 1 -- 172.21.15.31:0/768331527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009550 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22c9207700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4199780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f22c0011980 tx=0x7f22c0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22c0003a60 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22c0003bc0 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22cb46b700 1 -- 172.21.15.31:0/768331527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c4108f00 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22c001a670 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.164+0000 7f22cb46b700 1 -- 172.21.15.31:0/768331527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c4108d70 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.166+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a40052f0 con 0x7f22c4104e20 2023-12-14T18:20:49.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.166+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f22c0024440 con 0x7f22c4104e20 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.166+0000 7f22ba7fc700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f22b0038630 0x7f22b003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.166+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f22c004c740 con 0x7f22c4104e20 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.166+0000 7f22c8a06700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f22b0038630 0x7f22b003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.167+0000 7f22c8a06700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f22b0038630 0x7f22b003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22b4006fd0 tx=0x7f22b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.175+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f22c00189a0 con 0x7f22c4104e20 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.393+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f22a4000bc0 con 0x7f22b0038630 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.519+0000 7f22ba7fc700 1 -- 172.21.15.31:0/768331527 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f22a4000bc0 con 0x7f22b0038630 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f22b0038630 msgr2=0x7f22b003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f22b0038630 0x7f22b003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22b4006fd0 tx=0x7f22b4006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 msgr2=0x7f22c4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4199780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f22c0011980 tx=0x7f22c0011a60 comp rx=0 tx=0).stop 2023-12-14T18:20:49.887 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 shutdown_connections 2023-12-14T18:20:49.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f22b0038630 0x7f22b003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:49.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 --2- 172.21.15.31:0/768331527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22c4104e20 0x7f22c4199780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:49.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 >> 172.21.15.31:0/768331527 conn(0x7f22c40ff650 msgr2=0x7f22c4104250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:49.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 shutdown_connections 2023-12-14T18:20:49.888 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:49.521+0000 7f22affff700 1 -- 172.21.15.31:0/768331527 wait complete. 2023-12-14T18:20:49.888 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2023-12-14T18:20:50.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:50 smithi031 ceph-mon[42085]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:50.818 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:50 smithi031 ceph-mon[42085]: mgrmap e12: x(active, since 2s) 2023-12-14T18:20:51.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$GUwfx3F/CiEo5FB09TSd7e.RFo8jZVTHeurO7e2Y192ycfPnBZLjq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1702578050, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-14T18:20:51.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.291+0000 7f24c34d4700 1 Processor -- start 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.292+0000 7f24c34d4700 1 -- start start 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.292+0000 7f24c34d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.292+0000 7f24c34d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc079e20 con 0x7f24bc1064d0 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.292+0000 7f24c1270700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.292+0000 7f24c1270700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48130/0 (socket says 172.21.15.31:48130) 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.292+0000 7f24c1270700 1 -- 172.21.15.31:0/839901967 learned_addr learned my addr 172.21.15.31:0/839901967 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.293+0000 7f24c1270700 1 -- 172.21.15.31:0/839901967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24bc108df0 con 0x7f24bc1064d0 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.293+0000 7f24c1270700 1 --2- 172.21.15.31:0/839901967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1088b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009e80 tx=0x7f24b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a96627863e17d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.293+0000 7f24b3fff700 1 -- 172.21.15.31:0/839901967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24b800e070 con 0x7f24bc1064d0 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.293+0000 7f24b3fff700 1 -- 172.21.15.31:0/839901967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b800b8c0 con 0x7f24bc1064d0 2023-12-14T18:20:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.293+0000 7f24b3fff700 1 -- 172.21.15.31:0/839901967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24b80043e0 con 0x7f24bc1064d0 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 -- 172.21.15.31:0/839901967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 msgr2=0x7f24bc1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 --2- 172.21.15.31:0/839901967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1088b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009e80 tx=0x7f24b800b330 comp rx=0 tx=0).stop 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 -- 172.21.15.31:0/839901967 shutdown_connections 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 --2- 172.21.15.31:0/839901967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1088b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 -- 172.21.15.31:0/839901967 >> 172.21.15.31:0/839901967 conn(0x7f24bc0ffed0 msgr2=0x7f24bc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 -- 172.21.15.31:0/839901967 shutdown_connections 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.294+0000 7f24c34d4700 1 -- 172.21.15.31:0/839901967 wait complete. 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c34d4700 1 Processor -- start 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c34d4700 1 -- start start 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c34d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c34d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24bc199c00 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c1270700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c1270700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48138/0 (socket says 172.21.15.31:48138) 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c1270700 1 -- 172.21.15.31:0/1785039839 learned_addr learned my addr 172.21.15.31:0/1785039839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c1270700 1 -- 172.21.15.31:0/1785039839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009b30 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.295+0000 7f24c1270700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1996c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f24b8004000 tx=0x7f24b8003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.296+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24b800e040 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.296+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b800be30 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.296+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc109d60 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.296+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24b801c700 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.296+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24bc109bd0 con 0x7f24bc1064d0 2023-12-14T18:20:51.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.297+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f24b801c980 con 0x7f24bc1064d0 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.297+0000 7f24b27fc700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f24a8038760 0x7f24a803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.297+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f24b804c670 con 0x7f24bc1064d0 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.297+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24bc109ef0 con 0x7f24bc1064d0 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.297+0000 7f24c0a6f700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f24a8038760 0x7f24a803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.297+0000 7f24c0a6f700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f24a8038760 0x7f24a803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006fd0 tx=0x7f24ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.302+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f24b8017070 con 0x7f24bc1064d0 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.539+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f24bc061f40 con 0x7f24a8038760 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.792+0000 7f24b27fc700 1 -- 172.21.15.31:0/1785039839 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f24bc061f40 con 0x7f24a8038760 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f24a8038760 msgr2=0x7f24a803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:51.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f24a8038760 0x7f24a803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f24ac006fd0 tx=0x7f24ac006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 msgr2=0x7f24bc1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1996c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f24b8004000 tx=0x7f24b8003d00 comp rx=0 tx=0).stop 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 shutdown_connections 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f24a8038760 0x7f24a803ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 --2- 172.21.15.31:0/1785039839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24bc1064d0 0x7f24bc1996c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 >> 172.21.15.31:0/1785039839 conn(0x7f24bc0ffed0 msgr2=0x7f24bc100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 shutdown_connections 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:50.795+0000 7f24c34d4700 1 -- 172.21.15.31:0/1785039839 wait complete. 2023-12-14T18:20:51.198 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2023-12-14T18:20:52.114 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:51 smithi031 ceph-mon[42085]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:52.114 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:52.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2023-12-14T18:20:52.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73c263d700 1 Processor -- start 2023-12-14T18:20:52.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73c263d700 1 -- start start 2023-12-14T18:20:52.192 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73c263d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73c263d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc079e60 con 0x7f73bc106490 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48154/0 (socket says 172.21.15.31:48154) 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73bbfff700 1 -- 172.21.15.31:0/3791359307 learned_addr learned my addr 172.21.15.31:0/3791359307 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.602+0000 7f73bbfff700 1 -- 172.21.15.31:0/3791359307 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc108db0 con 0x7f73bc106490 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73bbfff700 1 --2- 172.21.15.31:0/3791359307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc108870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098a0 tx=0x7f73ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee727bdb82a2576 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73baffd700 1 -- 172.21.15.31:0/3791359307 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73ac004160 con 0x7f73bc106490 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73baffd700 1 -- 172.21.15.31:0/3791359307 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73ac0042c0 con 0x7f73bc106490 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73baffd700 1 -- 172.21.15.31:0/3791359307 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73ac0119b0 con 0x7f73bc106490 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73c263d700 1 -- 172.21.15.31:0/3791359307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 msgr2=0x7f73bc108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:52.193 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73c263d700 1 --2- 172.21.15.31:0/3791359307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc108870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098a0 tx=0x7f73ac009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73c263d700 1 -- 172.21.15.31:0/3791359307 shutdown_connections 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73c263d700 1 --2- 172.21.15.31:0/3791359307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc108870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73c263d700 1 -- 172.21.15.31:0/3791359307 >> 172.21.15.31:0/3791359307 conn(0x7f73bc0ffed0 msgr2=0x7f73bc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.603+0000 7f73c263d700 1 -- 172.21.15.31:0/3791359307 shutdown_connections 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.604+0000 7f73c263d700 1 -- 172.21.15.31:0/3791359307 wait complete. 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.604+0000 7f73c263d700 1 Processor -- start 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.604+0000 7f73c263d700 1 -- start start 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73c263d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73c263d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc199cd0 con 0x7f73bc106490 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73bbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48164/0 (socket says 172.21.15.31:48164) 2023-12-14T18:20:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73bbfff700 1 -- 172.21.15.31:0/3507243972 learned_addr learned my addr 172.21.15.31:0/3507243972 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73bbfff700 1 -- 172.21.15.31:0/3507243972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ac009550 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73bbfff700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc199790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73ac011980 tx=0x7f73ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73ac003a60 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.605+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73ac003bc0 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.606+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc109d80 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.606+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73ac01a670 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.606+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc109bf0 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.606+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f73ac024440 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.607+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc109f10 con 0x7f73bc106490 2023-12-14T18:20:52.195 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.607+0000 7f73b97fa700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73a40387b0 0x7f73a403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.607+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73ac04c930 con 0x7f73bc106490 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.607+0000 7f73bb7fe700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73a40387b0 0x7f73a403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.607+0000 7f73bb7fe700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73a40387b0 0x7f73a403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73b0006fd0 tx=0x7f73b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.612+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73ac0189a0 con 0x7f73bc106490 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.821+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f73bc19d3a0 con 0x7f73bc106490 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.821+0000 7f73b97fa700 1 -- 172.21.15.31:0/3507243972 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f73ac018350 con 0x7f73bc106490 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73a40387b0 msgr2=0x7f73a403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73a40387b0 0x7f73a403ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f73b0006fd0 tx=0x7f73b0006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 msgr2=0x7f73bc199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc199790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f73ac011980 tx=0x7f73ac011a60 comp rx=0 tx=0).stop 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 shutdown_connections 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73a40387b0 0x7f73a403ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:52.196 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 --2- 172.21.15.31:0/3507243972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73bc106490 0x7f73bc199790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:52.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 >> 172.21.15.31:0/3507243972 conn(0x7f73bc0ffed0 msgr2=0x7f73bc101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:52.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.823+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 shutdown_connections 2023-12-14T18:20:52.197 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:51.824+0000 7f73c263d700 1 -- 172.21.15.31:0/3507243972 wait complete. 2023-12-14T18:20:52.203 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:20:52.203 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2023-12-14T18:20:52.212 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:20:52.212 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2023-12-14T18:20:52.212 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2023-12-14T18:20:52.213 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-14T18:20:52.214 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2023-12-14T18:20:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031:8443/ 2023-12-14T18:20:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2023-12-14T18:20:52.214 INFO:teuthology.orchestra.run.smithi031.stdout: Password: s68e5xfic3 2023-12-14T18:20:52.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:52.215 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config directory 2023-12-14T18:20:52.215 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2023-12-14T18:20:53.050 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:52 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3507243972' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-14T18:20:53.249 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.623+0000 7f73f049a700 1 Processor -- start 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.623+0000 7f73f049a700 1 -- start start 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73f049a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e8104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73f049a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8105300 con 0x7f73e8104920 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73ee236700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e8104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73ee236700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e8104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48166/0 (socket says 172.21.15.31:48166) 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73ee236700 1 -- 172.21.15.31:0/2595825620 learned_addr learned my addr 172.21.15.31:0/2595825620 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73ee236700 1 -- 172.21.15.31:0/2595825620 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8105b20 con 0x7f73e8104920 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.624+0000 7f73ee236700 1 --2- 172.21.15.31:0/2595825620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e8104d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e40098a0 tx=0x7f73e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283985090fb7d00 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:53.250 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73ed234700 1 -- 172.21.15.31:0/2595825620 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73e4004160 con 0x7f73e8104920 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73ed234700 1 -- 172.21.15.31:0/2595825620 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e40042c0 con 0x7f73e8104920 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73ed234700 1 -- 172.21.15.31:0/2595825620 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73e40119b0 con 0x7f73e8104920 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73f049a700 1 -- 172.21.15.31:0/2595825620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 msgr2=0x7f73e8104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73f049a700 1 --2- 172.21.15.31:0/2595825620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e8104d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73e40098a0 tx=0x7f73e4009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73f049a700 1 -- 172.21.15.31:0/2595825620 shutdown_connections 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73f049a700 1 --2- 172.21.15.31:0/2595825620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e8104d30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.625+0000 7f73f049a700 1 -- 172.21.15.31:0/2595825620 >> 172.21.15.31:0/2595825620 conn(0x7f73e80ffed0 msgr2=0x7f73e8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.626+0000 7f73f049a700 1 -- 172.21.15.31:0/2595825620 shutdown_connections 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.626+0000 7f73f049a700 1 -- 172.21.15.31:0/2595825620 wait complete. 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.626+0000 7f73f049a700 1 Processor -- start 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73f049a700 1 -- start start 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73f049a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e81975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73f049a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8197b10 con 0x7f73e8104920 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73ee236700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e81975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:53.251 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73ee236700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e81975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48168/0 (socket says 172.21.15.31:48168) 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73ee236700 1 -- 172.21.15.31:0/3544557804 learned_addr learned my addr 172.21.15.31:0/3544557804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73ee236700 1 -- 172.21.15.31:0/3544557804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4009550 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.627+0000 7f73ee236700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e81975d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f73e4000c00 tx=0x7f73e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.628+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73e40039f0 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.628+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e4003b50 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.628+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e8078f80 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.628+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73e401a780 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.628+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e81068e0 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.629+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f73e4024440 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.629+0000 7f73df7fe700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73d403a9f0 0x7f73d403cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.629+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8079110 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.629+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73e404c870 con 0x7f73e8104920 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.629+0000 7f73eda35700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73d403a9f0 0x7f73d403cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.630+0000 7f73eda35700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73d403a9f0 0x7f73d403cea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006fd0 tx=0x7f73d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:53.252 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.634+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73e40189a0 con 0x7f73e8104920 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.843+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f73e802cfe0 con 0x7f73e8104920 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.844+0000 7f73df7fe700 1 -- 172.21.15.31:0/3544557804 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f73e4018350 con 0x7f73e8104920 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73d403a9f0 msgr2=0x7f73d403cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73d403a9f0 0x7f73d403cea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73d8006fd0 tx=0x7f73d8006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 msgr2=0x7f73e81975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e81975d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f73e4000c00 tx=0x7f73e4011a30 comp rx=0 tx=0).stop 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 shutdown_connections 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73d403a9f0 0x7f73d403cea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 --2- 172.21.15.31:0/3544557804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73e8104920 0x7f73e81975d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 >> 172.21.15.31:0/3544557804 conn(0x7f73e80ffed0 msgr2=0x7f73e818e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 shutdown_connections 2023-12-14T18:20:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:52.846+0000 7f73f049a700 1 -- 172.21.15.31:0/3544557804 wait complete. 2023-12-14T18:20:54.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f51377700 1 Processor -- start 2023-12-14T18:20:54.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f51377700 1 -- start start 2023-12-14T18:20:54.384 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f51377700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f51377700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c079e60 con 0x7f1f4c106490 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f4affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f4affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48182/0 (socket says 172.21.15.31:48182) 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.652+0000 7f1f4affd700 1 -- 172.21.15.31:0/4114626276 learned_addr learned my addr 172.21.15.31:0/4114626276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.653+0000 7f1f4affd700 1 -- 172.21.15.31:0/4114626276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c108db0 con 0x7f1f4c106490 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.653+0000 7f1f4affd700 1 --2- 172.21.15.31:0/4114626276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c108870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f340098a0 tx=0x7f1f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b9a55b2748ab6c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.653+0000 7f1f49ffb700 1 -- 172.21.15.31:0/4114626276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f34004160 con 0x7f1f4c106490 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.653+0000 7f1f49ffb700 1 -- 172.21.15.31:0/4114626276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f340042c0 con 0x7f1f4c106490 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.653+0000 7f1f49ffb700 1 -- 172.21.15.31:0/4114626276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f340119b0 con 0x7f1f4c106490 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 -- 172.21.15.31:0/4114626276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 msgr2=0x7f1f4c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 --2- 172.21.15.31:0/4114626276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c108870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f340098a0 tx=0x7f1f34009bb0 comp rx=0 tx=0).stop 2023-12-14T18:20:54.385 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 -- 172.21.15.31:0/4114626276 shutdown_connections 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 --2- 172.21.15.31:0/4114626276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c108870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 -- 172.21.15.31:0/4114626276 >> 172.21.15.31:0/4114626276 conn(0x7f1f4c0ffed0 msgr2=0x7f1f4c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 -- 172.21.15.31:0/4114626276 shutdown_connections 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 -- 172.21.15.31:0/4114626276 wait complete. 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.654+0000 7f1f51377700 1 Processor -- start 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f51377700 1 -- start start 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f51377700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f51377700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c199cc0 con 0x7f1f4c106490 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f4affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f4affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48184/0 (socket says 172.21.15.31:48184) 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f4affd700 1 -- 172.21.15.31:0/3602341911 learned_addr learned my addr 172.21.15.31:0/3602341911 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:54.386 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f4affd700 1 -- 172.21.15.31:0/3602341911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34009550 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.655+0000 7f1f4affd700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c199780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011980 tx=0x7f1f34011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.656+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f34003a60 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.656+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f34003bc0 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.656+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c109d80 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.656+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f3401a670 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.656+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c109bf0 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.657+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1f3401a8e0 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.657+0000 7f1f43fff700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1f380387b0 0x7f1f3803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.657+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f3404c8c0 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.657+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c109f10 con 0x7f1f4c106490 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.657+0000 7f1f4a7fc700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1f380387b0 0x7f1f3803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.657+0000 7f1f4a7fc700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1f380387b0 0x7f1f3803ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006fd0 tx=0x7f1f3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.662+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1f3401abe0 con 0x7f1f4c106490 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.794+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1f3402d910 con 0x7f1f4c106490 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.986+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f1f4c062120 con 0x7f1f4c106490 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.989+0000 7f1f43fff700 1 -- 172.21.15.31:0/3602341911 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f1f3402a2a0 con 0x7f1f4c106490 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1f380387b0 msgr2=0x7f1f3803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1f380387b0 0x7f1f3803ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006fd0 tx=0x7f1f3c006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 msgr2=0x7f1f4c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c199780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011980 tx=0x7f1f34011a60 comp rx=0 tx=0).stop 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 shutdown_connections 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1f380387b0 0x7f1f3803ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 --2- 172.21.15.31:0/3602341911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c106490 0x7f1f4c199780 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 >> 172.21.15.31:0/3602341911 conn(0x7f1f4c0ffed0 msgr2=0x7f1f4c100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 shutdown_connections 2023-12-14T18:20:54.388 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2023-12-14T18:20:53.991+0000 7f1f51377700 1 -- 172.21.15.31:0/3602341911 wait complete. 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2023-12-14T18:20:54.389 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.390 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2023-12-14T18:20:54.390 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.390 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-14T18:20:54.390 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:20:54.390 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2023-12-14T18:20:54.431 INFO:tasks.cephadm:Fetching config... 2023-12-14T18:20:54.432 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:20:54.432 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-14T18:20:54.451 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-14T18:20:54.451 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:20:54.452 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-14T18:20:54.512 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-14T18:20:54.512 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:20:54.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-14T18:20:54.588 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-14T18:20:54.588 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:20:54.588 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-14T18:20:54.651 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-14T18:20:54.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-14T18:20:54.736 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:54.749 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-14T18:20:54.797 INFO:teuthology.orchestra.run.smithi176.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:54.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-14T18:20:54.860 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5JIe+O3wWZek1TCckhmT2dZ/Yh3fij+10Lnk1zyg1ltCRkglSxIEPfgu12O7uWnJeUL7KENFXucIOxOAxxxqLOMZLUnnyygWdADegk4bqdE1N+Jfsk1IGfZCAc5u8NVcllP3o8UFIncBhMsy0QKeAV9eHvQwtQsNLefdcQb3CGe+vxQOHwmf7qCzd/j4JHtX77po5mPiS3d1/iwv+mwleee2jswHa/Z1BOr3LtEFLG8Wa70F3iX/t/4S4v7Qy9cCcJ7xlw4CpAQge3dPlQZkZ1MAyEZTVzIFp3FJ6Qdq2MAk2tbZofpWh77wZuyFGBhSBXihLPqSxUWtiW3EcnfjqS/7pBv3Er+dKQAYyK314AgaLV5GPo9zgUu5mm/hMaq7ASSNbmUv0ViTRTQdxVulRQj2EQGncfLfAbpvSHHPcgfxfZH7XREetLiEKZaUsk1DQa/tY/+ks57O85OZp8kFRTWF+4SaVvjqs0LNp7neCEbNdx9lsSL6W/5tBjTEwWcs= ceph-37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:54.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-14T18:20:54.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:54 smithi031 ceph-mon[42085]: mgrmap e13: x(active, since 6s) 2023-12-14T18:20:54.902 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:54 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3602341911' entity='client.admin' 2023-12-14T18:20:55.230 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.608+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/3408969552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 msgr2=0x7f8e180ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.608+0000 7f8e1fc4c700 1 --2- 172.21.15.31:0/3408969552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e180ff000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009910 tx=0x7f8e14009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.608+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/3408969552 shutdown_connections 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.608+0000 7f8e1fc4c700 1 --2- 172.21.15.31:0/3408969552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e180ff000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.608+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/3408969552 >> 172.21.15.31:0/3408969552 conn(0x7f8e180fa240 msgr2=0x7f8e180fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.609+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/3408969552 shutdown_connections 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.609+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/3408969552 wait complete. 2023-12-14T18:20:56.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.609+0000 7f8e1fc4c700 1 Processor -- start 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1fc4c700 1 -- start start 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1fc4c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e18068850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1fc4c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18197b10 con 0x7f8e180febf0 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1d9e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e18068850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1d9e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e18068850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38026/0 (socket says 172.21.15.31:38026) 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1d9e8700 1 -- 172.21.15.31:0/532647466 learned_addr learned my addr 172.21.15.31:0/532647466 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:56.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1d9e8700 1 -- 172.21.15.31:0/532647466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140095f0 con 0x7f8e180febf0 2023-12-14T18:20:56.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.610+0000 7f8e1d9e8700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e18068850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005f50 tx=0x7f8e140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.611+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e1401d070 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.611+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e14022470 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.611+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e18104a40 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.611+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e1400f5f0 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.611+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e181048b0 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.612+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8e1400f8b0 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.612+0000 7f8e0effd700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e04038870 0x7f8e0403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.612+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e1404caf0 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.612+0000 7f8e1d1e7700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e04038870 0x7f8e0403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.612+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1818d770 con 0x7f8e180febf0 2023-12-14T18:20:56.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.613+0000 7f8e1d1e7700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e04038870 0x7f8e0403ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e08006fd0 tx=0x7f8e08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:56.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.617+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8e14027030 con 0x7f8e180febf0 2023-12-14T18:20:56.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.836+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8e1802cfe0 con 0x7f8e180febf0 2023-12-14T18:20:56.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.839+0000 7f8e0effd700 1 -- 172.21.15.31:0/532647466 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8e1402a360 con 0x7f8e180febf0 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e04038870 msgr2=0x7f8e0403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e04038870 0x7f8e0403ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8e08006fd0 tx=0x7f8e08006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 msgr2=0x7f8e18068850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e18068850 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005f50 tx=0x7f8e140057c0 comp rx=0 tx=0).stop 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 shutdown_connections 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e04038870 0x7f8e0403ad20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 --2- 172.21.15.31:0/532647466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e180febf0 0x7f8e18068850 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 >> 172.21.15.31:0/532647466 conn(0x7f8e180fa240 msgr2=0x7f8e180faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 shutdown_connections 2023-12-14T18:20:56.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:56.842+0000 7f8e1fc4c700 1 -- 172.21.15.31:0/532647466 wait complete. 2023-12-14T18:20:56.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:56.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:56.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:20:56.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:56.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:56.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:57.360 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-14T18:20:57.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-14T18:20:57.750 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:20:58.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:57 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/532647466' entity='client.admin' 2023-12-14T18:20:58.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:58.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:20:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 -- 172.21.15.31:0/793794149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 msgr2=0x7fbb8c0ff020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 --2- 172.21.15.31:0/793794149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c0ff020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009910 tx=0x7fbb7c009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 -- 172.21.15.31:0/793794149 shutdown_connections 2023-12-14T18:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 --2- 172.21.15.31:0/793794149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c0ff020 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 -- 172.21.15.31:0/793794149 >> 172.21.15.31:0/793794149 conn(0x7fbb8c0fa1c0 msgr2=0x7fbb8c0fc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 -- 172.21.15.31:0/793794149 shutdown_connections 2023-12-14T18:20:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.662+0000 7fbb90ca4700 1 -- 172.21.15.31:0/793794149 wait complete. 2023-12-14T18:20:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.663+0000 7fbb90ca4700 1 Processor -- start 2023-12-14T18:20:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.663+0000 7fbb90ca4700 1 -- start start 2023-12-14T18:20:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.663+0000 7fbb90ca4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c193d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.663+0000 7fbb90ca4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c194250 con 0x7fbb8c0fec10 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb8a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c193d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb8a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c193d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38044/0 (socket says 172.21.15.31:38044) 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb8a59c700 1 -- 172.21.15.31:0/2004999405 learned_addr learned my addr 172.21.15.31:0/2004999405 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb8a59c700 1 -- 172.21.15.31:0/2004999405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0095f0 con 0x7fbb8c0fec10 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb8a59c700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c193d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006010 tx=0x7fbb7c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb7c01d070 con 0x7fbb8c0fec10 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7c0044e0 con 0x7fbb8c0fec10 2023-12-14T18:20:58.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c104ae0 con 0x7fbb8c0fec10 2023-12-14T18:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb7c0039b0 con 0x7fbb8c0fec10 2023-12-14T18:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.664+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c104950 con 0x7fbb8c0fec10 2023-12-14T18:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.665+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbb7c003c70 con 0x7fbb8c0fec10 2023-12-14T18:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.665+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c104c70 con 0x7fbb8c0fec10 2023-12-14T18:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.665+0000 7fbb7b7fe700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbb74038870 0x7fbb7403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.665+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb7c04cb50 con 0x7fbb8c0fec10 2023-12-14T18:20:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.666+0000 7fbb89d9b700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbb74038870 0x7fbb7403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.666+0000 7fbb89d9b700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbb74038870 0x7fbb7403ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb80006fd0 tx=0x7fbb80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.671+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbb7c02a950 con 0x7fbb8c0fec10 2023-12-14T18:20:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.885+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb8c060f30 con 0x7fbb74038870 2023-12-14T18:20:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.888+0000 7fbb7b7fe700 1 -- 172.21.15.31:0/2004999405 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbb8c060f30 con 0x7fbb74038870 2023-12-14T18:20:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbb74038870 msgr2=0x7fbb7403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbb74038870 0x7fbb7403ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb80006fd0 tx=0x7fbb80006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 msgr2=0x7fbb8c193d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:58.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c193d10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006010 tx=0x7fbb7c0057c0 comp rx=0 tx=0).stop 2023-12-14T18:20:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 shutdown_connections 2023-12-14T18:20:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbb74038870 0x7fbb7403ad20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 --2- 172.21.15.31:0/2004999405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb8c0fec10 0x7fbb8c193d10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 >> 172.21.15.31:0/2004999405 conn(0x7fbb8c0fa1c0 msgr2=0x7fbb8c0fae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 shutdown_connections 2023-12-14T18:20:58.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:20:58.891+0000 7fbb90ca4700 1 -- 172.21.15.31:0/2004999405 wait complete. 2023-12-14T18:20:59.305 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi176 2023-12-14T18:20:59.306 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:20:59.306 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.conf 2023-12-14T18:20:59.328 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:20:59.328 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:59.388 INFO:tasks.cephadm:Adding host smithi176 to orchestrator... 2023-12-14T18:20:59.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch host add smithi176 2023-12-14T18:20:59.743 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:00.158 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:20:59 smithi031 ceph-mon[42085]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:21:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1879145171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 msgr2=0x7f1bc0100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 --2- 172.21.15.31:0/1879145171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0100700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009910 tx=0x7f1bb0009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1879145171 shutdown_connections 2023-12-14T18:21:00.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 --2- 172.21.15.31:0/1879145171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0100700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1879145171 >> 172.21.15.31:0/1879145171 conn(0x7f1bc0074350 msgr2=0x7f1bc0074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1879145171 shutdown_connections 2023-12-14T18:21:00.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.786+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1879145171 wait complete. 2023-12-14T18:21:00.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bc5c14700 1 Processor -- start 2023-12-14T18:21:00.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bc5c14700 1 -- start start 2023-12-14T18:21:00.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bc5c14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0194090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:00.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bc5c14700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc01945d0 con 0x7f1bc0100330 2023-12-14T18:21:00.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bbf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0194090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:00.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bbf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0194090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38060/0 (socket says 172.21.15.31:38060) 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.787+0000 7f1bbf7fe700 1 -- 172.21.15.31:0/1827645759 learned_addr learned my addr 172.21.15.31:0/1827645759 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bbf7fe700 1 -- 172.21.15.31:0/1827645759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00095f0 con 0x7f1bc0100330 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bbf7fe700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0194090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0004f40 tx=0x7f1bb0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bb001d070 con 0x7f1bc0100330 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc01088b0 con 0x7f1bc0100330 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bb0004370 con 0x7f1bc0100330 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1bb00039b0 con 0x7f1bc0100330 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.788+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc0108720 con 0x7f1bc0100330 2023-12-14T18:21:00.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.789+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1bb0003c70 con 0x7f1bc0100330 2023-12-14T18:21:00.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.789+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc0108a40 con 0x7f1bc0100330 2023-12-14T18:21:00.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.789+0000 7f1bbcff9700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ba8038870 0x7f1ba803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:00.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.789+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1bb004c560 con 0x7f1bc0100330 2023-12-14T18:21:00.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.789+0000 7f1bbeffd700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ba8038870 0x7f1ba803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:00.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.790+0000 7f1bbeffd700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ba8038870 0x7f1ba803ad20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4006fd0 tx=0x7f1bb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:00.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:00.794+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1bb0029950 con 0x7f1bc0100330 2023-12-14T18:21:01.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:01.009+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bc01953f0 con 0x7f1ba8038870 2023-12-14T18:21:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:00 smithi031 ceph-mon[42085]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:00 smithi031 ceph-mon[42085]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:00 smithi031 ceph-mon[42085]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:21:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:02.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:02 smithi031 ceph-mon[42085]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi176", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:02.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:02 smithi031 ceph-mon[42085]: Deploying cephadm binary to smithi176 2023-12-14T18:21:03.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.062+0000 7f1bbcff9700 1 -- 172.21.15.31:0/1827645759 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1bc01953f0 con 0x7f1ba8038870 2023-12-14T18:21:03.065 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi176' with addr '172.21.15.176' 2023-12-14T18:21:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ba8038870 msgr2=0x7f1ba803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ba8038870 0x7f1ba803ad20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4006fd0 tx=0x7f1bb4006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 msgr2=0x7f1bc0194090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0194090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0004f40 tx=0x7f1bb0005020 comp rx=0 tx=0).stop 2023-12-14T18:21:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 shutdown_connections 2023-12-14T18:21:03.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ba8038870 0x7f1ba803ad20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:03.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 --2- 172.21.15.31:0/1827645759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bc0100330 0x7f1bc0194090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:03.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 >> 172.21.15.31:0/1827645759 conn(0x7f1bc0074350 msgr2=0x7f1bc00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:03.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 shutdown_connections 2023-12-14T18:21:03.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:03.066+0000 7f1bc5c14700 1 -- 172.21.15.31:0/1827645759 wait complete. 2023-12-14T18:21:03.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch host ls --format=json 2023-12-14T18:21:03.823 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:04 smithi031 ceph-mon[42085]: Added host smithi176 2023-12-14T18:21:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 -- 172.21.15.31:0/1352797630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 msgr2=0x7fd52c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 --2- 172.21.15.31:0/1352797630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c0731e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009910 tx=0x7fd51c009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 -- 172.21.15.31:0/1352797630 shutdown_connections 2023-12-14T18:21:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 --2- 172.21.15.31:0/1352797630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c0731e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 -- 172.21.15.31:0/1352797630 >> 172.21.15.31:0/1352797630 conn(0x7fd52c0fb870 msgr2=0x7fd52c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 -- 172.21.15.31:0/1352797630 shutdown_connections 2023-12-14T18:21:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.864+0000 7fd53106c700 1 -- 172.21.15.31:0/1352797630 wait complete. 2023-12-14T18:21:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.865+0000 7fd53106c700 1 Processor -- start 2023-12-14T18:21:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd53106c700 1 -- start start 2023-12-14T18:21:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd53106c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd53106c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c1979c0 con 0x7fd52c072e10 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd52ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd52ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34466/0 (socket says 172.21.15.31:34466) 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd52ad9d700 1 -- 172.21.15.31:0/3751111733 learned_addr learned my addr 172.21.15.31:0/3751111733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.866+0000 7fd52ad9d700 1 -- 172.21.15.31:0/3751111733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0095f0 con 0x7fd52c072e10 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.867+0000 7fd52ad9d700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c197480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd51c006010 tx=0x7fd51c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.867+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c01d070 con 0x7fd52c072e10 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.867+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd51c00bc10 con 0x7fd52c072e10 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.867+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c10c370 con 0x7fd52c072e10 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.867+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd51c003910 con 0x7fd52c072e10 2023-12-14T18:21:04.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.867+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c10c1e0 con 0x7fd52c072e10 2023-12-14T18:21:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.868+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd51c003bd0 con 0x7fd52c072e10 2023-12-14T18:21:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.868+0000 7fd528d99700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd518038870 0x7fd51803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.868+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c10c500 con 0x7fd52c072e10 2023-12-14T18:21:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.868+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd51c04c570 con 0x7fd52c072e10 2023-12-14T18:21:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.868+0000 7fd52259c700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd518038870 0x7fd51803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.869+0000 7fd52259c700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd518038870 0x7fd51803ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd514006fd0 tx=0x7fd514006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:04.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:04.873+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd51c026020 con 0x7fd52c072e10 2023-12-14T18:21:05.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.092+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd52c1095b0 con 0x7fd518038870 2023-12-14T18:21:05.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.093+0000 7fd528d99700 1 -- 172.21.15.31:0/3751111733 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fd52c1095b0 con 0x7fd518038870 2023-12-14T18:21:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:21:05.094 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}] 2023-12-14T18:21:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd518038870 msgr2=0x7fd51803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:05.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd518038870 0x7fd51803ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd514006fd0 tx=0x7fd514006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 msgr2=0x7fd52c197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c197480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd51c006010 tx=0x7fd51c004ff0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 shutdown_connections 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd518038870 0x7fd51803ad20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 --2- 172.21.15.31:0/3751111733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd52c072e10 0x7fd52c197480 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 >> 172.21.15.31:0/3751111733 conn(0x7fd52c0fb870 msgr2=0x7fd52c1068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 shutdown_connections 2023-12-14T18:21:05.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:05.095+0000 7fd53106c700 1 -- 172.21.15.31:0/3751111733 wait complete. 2023-12-14T18:21:05.502 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2023-12-14T18:21:05.502 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:21:05.502 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2023-12-14T18:21:05.523 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:21:05.524 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:05.584 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2023-12-14T18:21:05.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch host add smithi204 2023-12-14T18:21:05.936 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:05.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:05 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:06.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 -- 172.21.15.31:0/3126333853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 msgr2=0x7f0b380fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:06.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 --2- 172.21.15.31:0/3126333853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b380fef20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800b0a0 tx=0x7f0b2800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:21:06.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 -- 172.21.15.31:0/3126333853 shutdown_connections 2023-12-14T18:21:06.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 --2- 172.21.15.31:0/3126333853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b380fef20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:06.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 -- 172.21.15.31:0/3126333853 >> 172.21.15.31:0/3126333853 conn(0x7f0b38074930 msgr2=0x7f0b38074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:06.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 -- 172.21.15.31:0/3126333853 shutdown_connections 2023-12-14T18:21:06.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.950+0000 7f0b3d260700 1 -- 172.21.15.31:0/3126333853 wait complete. 2023-12-14T18:21:06.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.951+0000 7f0b3d260700 1 Processor -- start 2023-12-14T18:21:06.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.951+0000 7f0b3d260700 1 -- start start 2023-12-14T18:21:06.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b3d260700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b38197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:06.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b3d260700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b38197960 con 0x7f0b380feb50 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b36d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b38197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b36d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b38197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34490/0 (socket says 172.21.15.31:34490) 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b36d9d700 1 -- 172.21.15.31:0/4238372737 learned_addr learned my addr 172.21.15.31:0/4238372737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b36d9d700 1 -- 172.21.15.31:0/4238372737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28009d00 con 0x7f0b380feb50 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b36d9d700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b38197420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b280062a0 tx=0x7f0b2800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b2800e040 con 0x7f0b380feb50 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b28003d40 con 0x7f0b380feb50 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3810c3a0 con 0x7f0b380feb50 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b2801d600 con 0x7f0b380feb50 2023-12-14T18:21:06.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.952+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3810c210 con 0x7f0b380feb50 2023-12-14T18:21:06.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.953+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f0b28019070 con 0x7f0b380feb50 2023-12-14T18:21:06.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.954+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3810c530 con 0x7f0b380feb50 2023-12-14T18:21:06.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.954+0000 7f0b1ffff700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b200388f0 0x7f0b2003ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:06.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.954+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0b2804c7c0 con 0x7f0b380feb50 2023-12-14T18:21:06.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.954+0000 7f0b3659c700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b200388f0 0x7f0b2003ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:06.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.954+0000 7f0b3659c700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b200388f0 0x7f0b2003ada0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:06.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:06.959+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0b28026990 con 0x7f0b380feb50 2023-12-14T18:21:06.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:06 smithi031 ceph-mon[42085]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:21:07.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:07.208+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b381097a0 con 0x7f0b200388f0 2023-12-14T18:21:08.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:08.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:08.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:08.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:08.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:08.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:08.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:08.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:08.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:07 smithi031 ceph-mon[42085]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: Deploying cephadm binary to smithi204 2023-12-14T18:21:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:09.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:09.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.306+0000 7f0b1ffff700 1 -- 172.21.15.31:0/4238372737 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0b381097a0 con 0x7f0b200388f0 2023-12-14T18:21:09.308 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi204' with addr '172.21.15.204' 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.309+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b200388f0 msgr2=0x7f0b2003ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.309+0000 7f0b3d260700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b200388f0 0x7f0b2003ada0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c006fd0 tx=0x7f0b2c006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.309+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 msgr2=0x7f0b38197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.309+0000 7f0b3d260700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b38197420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b280062a0 tx=0x7f0b2800bd40 comp rx=0 tx=0).stop 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.310+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 shutdown_connections 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.310+0000 7f0b3d260700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b200388f0 0x7f0b2003ada0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.310+0000 7f0b3d260700 1 --2- 172.21.15.31:0/4238372737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380feb50 0x7f0b38197420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.310+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 >> 172.21.15.31:0/4238372737 conn(0x7f0b38074930 msgr2=0x7f0b380fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:09.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.310+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 shutdown_connections 2023-12-14T18:21:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:09.310+0000 7f0b3d260700 1 -- 172.21.15.31:0/4238372737 wait complete. 2023-12-14T18:21:09.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch host ls --format=json 2023-12-14T18:21:10.053 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:10 smithi031 ceph-mon[42085]: Added host smithi204 2023-12-14T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:10 smithi031 ceph-mon[42085]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:11.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 -- 172.21.15.31:0/636700321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 msgr2=0x7f5688100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:11.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 --2- 172.21.15.31:0/636700321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688100360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f568000b0a0 tx=0x7f568000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:21:11.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 -- 172.21.15.31:0/636700321 shutdown_connections 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 --2- 172.21.15.31:0/636700321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688100360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 -- 172.21.15.31:0/636700321 >> 172.21.15.31:0/636700321 conn(0x7f56880fb870 msgr2=0x7f56880fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 -- 172.21.15.31:0/636700321 shutdown_connections 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.079+0000 7f568de0b700 1 -- 172.21.15.31:0/636700321 wait complete. 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.080+0000 7f568de0b700 1 Processor -- start 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.080+0000 7f568de0b700 1 -- start start 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f568de0b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f568de0b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56881956c0 con 0x7f56880fff90 2023-12-14T18:21:11.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f56877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:11.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f56877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34502/0 (socket says 172.21.15.31:34502) 2023-12-14T18:21:11.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f56877fe700 1 -- 172.21.15.31:0/3038206543 learned_addr learned my addr 172.21.15.31:0/3038206543 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f56877fe700 1 -- 172.21.15.31:0/3038206543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680009d00 con 0x7f56880fff90 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f56877fe700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688195180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56800062a0 tx=0x7f568000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f568000e040 con 0x7f56880fff90 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5680003d40 con 0x7f56880fff90 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568810a170 con 0x7f56880fff90 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f568001d600 con 0x7f56880fff90 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.081+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5688109fe0 con 0x7f56880fff90 2023-12-14T18:21:11.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.082+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568810a300 con 0x7f56880fff90 2023-12-14T18:21:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.083+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5680019070 con 0x7f56880fff90 2023-12-14T18:21:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.083+0000 7f5684ff9700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f56740388a0 0x7f567403ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.083+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f568004d2d0 con 0x7f56880fff90 2023-12-14T18:21:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.083+0000 7f5686ffd700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f56740388a0 0x7f567403ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:11.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.084+0000 7f5686ffd700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f56740388a0 0x7f567403ad50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5670006fd0 tx=0x7f5670006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.088+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f568002f380 con 0x7f56880fff90 2023-12-14T18:21:11.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.304+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56881080b0 con 0x7f56740388a0 2023-12-14T18:21:11.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.305+0000 7f5684ff9700 1 -- 172.21.15.31:0/3038206543 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f56881080b0 con 0x7f56740388a0 2023-12-14T18:21:11.307 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:21:11.307 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2023-12-14T18:21:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f56740388a0 msgr2=0x7f567403ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f56740388a0 0x7f567403ad50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5670006fd0 tx=0x7f5670006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 msgr2=0x7f5688195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688195180 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56800062a0 tx=0x7f568000bd40 comp rx=0 tx=0).stop 2023-12-14T18:21:11.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 shutdown_connections 2023-12-14T18:21:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f56740388a0 0x7f567403ad50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 --2- 172.21.15.31:0/3038206543 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56880fff90 0x7f5688195180 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 >> 172.21.15.31:0/3038206543 conn(0x7f56880fb870 msgr2=0x7f5688104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 shutdown_connections 2023-12-14T18:21:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:11.308+0000 7f568de0b700 1 -- 172.21.15.31:0/3038206543 wait complete. 2023-12-14T18:21:11.720 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-14T18:21:11.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd crush tunables default 2023-12-14T18:21:12.069 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:12.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:13.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 -- 172.21.15.31:0/1295040158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 msgr2=0x7f42780ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:13.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 --2- 172.21.15.31:0/1295040158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42780ff2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4268009910 tx=0x7f4268009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:13.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 -- 172.21.15.31:0/1295040158 shutdown_connections 2023-12-14T18:21:13.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 --2- 172.21.15.31:0/1295040158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42780ff2d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:13.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 -- 172.21.15.31:0/1295040158 >> 172.21.15.31:0/1295040158 conn(0x7f4278074930 msgr2=0x7f4278074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:13.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 -- 172.21.15.31:0/1295040158 shutdown_connections 2023-12-14T18:21:13.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.023+0000 7f427e7f7700 1 -- 172.21.15.31:0/1295040158 wait complete. 2023-12-14T18:21:13.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.024+0000 7f427e7f7700 1 Processor -- start 2023-12-14T18:21:13.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.024+0000 7f427e7f7700 1 -- start start 2023-12-14T18:21:13.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.024+0000 7f427e7f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42781099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.024+0000 7f427e7f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427810b950 con 0x7f42780fef00 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4277fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42781099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4277fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42781099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34516/0 (socket says 172.21.15.31:34516) 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4277fff700 1 -- 172.21.15.31:0/4151493448 learned_addr learned my addr 172.21.15.31:0/4151493448 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4277fff700 1 -- 172.21.15.31:0/4151493448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680095f0 con 0x7f42780fef00 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4277fff700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42781099d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4268004f40 tx=0x7f4268005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f426801d070 con 0x7f42780fef00 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4268004370 con 0x7f42780fef00 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4278109090 con 0x7f42780fef00 2023-12-14T18:21:13.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42680039b0 con 0x7f42780fef00 2023-12-14T18:21:13.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.025+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4278108f00 con 0x7f42780fef00 2023-12-14T18:21:13.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.026+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4278109220 con 0x7f42780fef00 2023-12-14T18:21:13.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.028+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f4268003c70 con 0x7f42780fef00 2023-12-14T18:21:13.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.029+0000 7f4275ffb700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4264038870 0x7f426403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:13.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.029+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f426804c9b0 con 0x7f42780fef00 2023-12-14T18:21:13.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.029+0000 7f426ffff700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4264038870 0x7f426403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:13.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.029+0000 7f426ffff700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4264038870 0x7f426403ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4260006fd0 tx=0x7f4260006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:13.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.033+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f426802a980 con 0x7f42780fef00 2023-12-14T18:21:13.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:12 smithi031 ceph-mon[42085]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:21:13.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:12 smithi031 ceph-mon[42085]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:13.255+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4278106120 con 0x7f42780fef00 2023-12-14T18:21:14.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.030+0000 7f4275ffb700 1 -- 172.21.15.31:0/4151493448 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4268026020 con 0x7f42780fef00 2023-12-14T18:21:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4264038870 msgr2=0x7f426403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4264038870 0x7f426403ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4260006fd0 tx=0x7f4260006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 msgr2=0x7f42781099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42781099d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4268004f40 tx=0x7f4268005020 comp rx=0 tx=0).stop 2023-12-14T18:21:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 shutdown_connections 2023-12-14T18:21:14.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4264038870 0x7f426403ad20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:14.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 --2- 172.21.15.31:0/4151493448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42780fef00 0x7f42781099d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:14.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.033+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 >> 172.21.15.31:0/4151493448 conn(0x7f4278074930 msgr2=0x7f42780fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:14.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.034+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 shutdown_connections 2023-12-14T18:21:14.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:14.034+0000 7f427e7f7700 1 -- 172.21.15.31:0/4151493448 wait complete. 2023-12-14T18:21:14.035 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:14.333 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4151493448' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.334 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:14.427 INFO:tasks.cephadm:Adding mon.a on smithi031 2023-12-14T18:21:14.428 INFO:tasks.cephadm:Adding mon.b on smithi176 2023-12-14T18:21:14.428 INFO:tasks.cephadm:Adding mon.c on smithi204 2023-12-14T18:21:14.428 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch apply mon '3;smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c' 2023-12-14T18:21:14.771 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:15.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:15 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4151493448' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-14T18:21:15.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:15 smithi031 ceph-mon[42085]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:15.784 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 -- 172.21.15.204:0/3625672071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 msgr2=0x7fda880ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:15.784 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 --2- 172.21.15.204:0/3625672071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda880ff260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fda78009910 tx=0x7fda78009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 -- 172.21.15.204:0/3625672071 shutdown_connections 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 --2- 172.21.15.204:0/3625672071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda880ff260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 -- 172.21.15.204:0/3625672071 >> 172.21.15.204:0/3625672071 conn(0x7fda880fa750 msgr2=0x7fda880fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 -- 172.21.15.204:0/3625672071 shutdown_connections 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.781+0000 7fda8d690700 1 -- 172.21.15.204:0/3625672071 wait complete. 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda8d690700 1 Processor -- start 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda8d690700 1 -- start start 2023-12-14T18:21:15.785 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda8d690700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda881008a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda8d690700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda88102870 con 0x7fda880fee90 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda86ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda881008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda86ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda881008a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.204:52738/0 (socket says 172.21.15.204:52738) 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda86ffd700 1 -- 172.21.15.204:0/2393740809 learned_addr learned my addr 172.21.15.204:0/2393740809 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.782+0000 7fda86ffd700 1 -- 172.21.15.204:0/2393740809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda780095f0 con 0x7fda880fee90 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.783+0000 7fda86ffd700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda881008a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda78004f40 tx=0x7fda78005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.783+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda7801d070 con 0x7fda880fee90 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.783+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda78004370 con 0x7fda880fee90 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.783+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda88109070 con 0x7fda880fee90 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.783+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda780039b0 con 0x7fda880fee90 2023-12-14T18:21:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.783+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda88108ee0 con 0x7fda880fee90 2023-12-14T18:21:15.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.784+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fda78003c70 con 0x7fda880fee90 2023-12-14T18:21:15.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.784+0000 7fda6ffff700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fda70038870 0x7fda7003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:15.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.784+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda88109200 con 0x7fda880fee90 2023-12-14T18:21:15.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.784+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fda7804c7a0 con 0x7fda880fee90 2023-12-14T18:21:15.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.784+0000 7fda867fc700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fda70038870 0x7fda7003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:15.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.785+0000 7fda867fc700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fda70038870 0x7fda7003ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fda7c006fd0 tx=0x7fda7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:15.791 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.789+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fda78026070 con 0x7fda880fee90 2023-12-14T18:21:15.986 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.984+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fda881015b0 con 0x7fda70038870 2023-12-14T18:21:15.993 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.992+0000 7fda6ffff700 1 -- 172.21.15.204:0/2393740809 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fda881015b0 con 0x7fda70038870 2023-12-14T18:21:15.993 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2023-12-14T18:21:15.995 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fda70038870 msgr2=0x7fda7003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:15.995 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fda70038870 0x7fda7003ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fda7c006fd0 tx=0x7fda7c006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:15.995 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 msgr2=0x7fda881008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:15.995 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda881008a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fda78004f40 tx=0x7fda78005020 comp rx=0 tx=0).stop 2023-12-14T18:21:15.995 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 shutdown_connections 2023-12-14T18:21:15.996 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fda70038870 0x7fda7003ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:15.996 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 --2- 172.21.15.204:0/2393740809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda880fee90 0x7fda881008a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:15.996 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 >> 172.21.15.204:0/2393740809 conn(0x7fda880fa750 msgr2=0x7fda88068760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:15.996 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 shutdown_connections 2023-12-14T18:21:15.996 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:15.993+0000 7fda8d690700 1 -- 172.21.15.204:0/2393740809 wait complete. 2023-12-14T18:21:16.382 DEBUG:teuthology.orchestra.run.smithi176:mon.b> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.b.service 2023-12-14T18:21:16.386 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.c.service 2023-12-14T18:21:16.388 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-14T18:21:16.389 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-14T18:21:16.418 INFO:journalctl@ceph.mon.c.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:21:16.421 INFO:journalctl@ceph.mon.b.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:21:16.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:16.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:17.026 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:17.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:17 smithi031 ceph-mon[42085]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:17.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:17 smithi031 ceph-mon[42085]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c;count:3 2023-12-14T18:21:17.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:17 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:17.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:17 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:17.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:17 smithi031 ceph-mon[42085]: Deploying daemon mon.c on smithi204 2023-12-14T18:21:18.546 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 -- 172.21.15.204:0/1346458607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 msgr2=0x7f9a440ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 --2- 172.21.15.204:0/1346458607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a440ff2d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009910 tx=0x7f9a34009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 -- 172.21.15.204:0/1346458607 shutdown_connections 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 --2- 172.21.15.204:0/1346458607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a440ff2d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 -- 172.21.15.204:0/1346458607 >> 172.21.15.204:0/1346458607 conn(0x7f9a44074930 msgr2=0x7f9a44074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 -- 172.21.15.204:0/1346458607 shutdown_connections 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.543+0000 7f9a497f7700 1 -- 172.21.15.204:0/1346458607 wait complete. 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.544+0000 7f9a497f7700 1 Processor -- start 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.544+0000 7f9a497f7700 1 -- start start 2023-12-14T18:21:18.547 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.545+0000 7f9a497f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a44195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:18.548 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.545+0000 7f9a497f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a44196510 con 0x7f9a440fef00 2023-12-14T18:21:18.548 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.545+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a44195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:18.548 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.545+0000 7f9a42ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a44195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.204:42190/0 (socket says 172.21.15.204:42190) 2023-12-14T18:21:18.548 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.545+0000 7f9a42ffd700 1 -- 172.21.15.204:0/1592662863 learned_addr learned my addr 172.21.15.204:0/1592662863 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:21:18.550 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.545+0000 7f9a42ffd700 1 -- 172.21.15.204:0/1592662863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a340095f0 con 0x7f9a440fef00 2023-12-14T18:21:18.550 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.546+0000 7f9a42ffd700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a44195fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a34004f40 tx=0x7f9a34005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:18.550 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.546+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a3401d070 con 0x7f9a440fef00 2023-12-14T18:21:18.550 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.546+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a34004370 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.546+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a44109090 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.546+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a340039b0 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.546+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a44108f00 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.547+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f9a34003c20 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.547+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a44109220 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.547+0000 7f9a2bfff700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a2c0388c0 0x7f9a2c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.547+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9a3404c560 con 0x7f9a440fef00 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.547+0000 7f9a427fc700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a2c0388c0 0x7f9a2c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:18.551 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.548+0000 7f9a427fc700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a2c0388c0 0x7f9a2c03ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a38006fd0 tx=0x7f9a38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:18.559 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.553+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9a34029950 con 0x7f9a440fef00 2023-12-14T18:21:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 systemd[1]: Starting Ceph mon.c for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:21:18.860 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.857+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9a44068340 con 0x7f9a440fef00 2023-12-14T18:21:18.860 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.858+0000 7f9a2bfff700 1 -- 172.21.15.204:0/1592662863 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f9a34026030 con 0x7f9a440fef00 2023-12-14T18:21:18.861 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:21:18.861 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","modified":"2023-12-14T18:19:52.681816Z","created":"2023-12-14T18:19:52.681816Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-14T18:21:18.863 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a2c0388c0 msgr2=0x7f9a2c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:18.863 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a2c0388c0 0x7f9a2c03ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a38006fd0 tx=0x7f9a38006e40 comp rx=0 tx=0).stop 2023-12-14T18:21:18.863 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 msgr2=0x7f9a44195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:18.864 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a44195fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9a34004f40 tx=0x7f9a34005020 comp rx=0 tx=0).stop 2023-12-14T18:21:18.864 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 shutdown_connections 2023-12-14T18:21:18.864 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a2c0388c0 0x7f9a2c03ad70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:18.864 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 --2- 172.21.15.204:0/1592662863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a440fef00 0x7f9a44195fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:18.864 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 >> 172.21.15.204:0/1592662863 conn(0x7f9a44074930 msgr2=0x7f9a44103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:18.865 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 shutdown_connections 2023-12-14T18:21:18.865 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:18.860+0000 7f9a497f7700 1 -- 172.21.15.204:0/1592662863 wait complete. 2023-12-14T18:21:18.865 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2023-12-14T18:21:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:18 smithi031 ceph-mon[42085]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:19.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: set uid:gid to 167:167 (ceph:ceph) 2023-12-14T18:21:19.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev), process ceph-mon, pid 7 2023-12-14T18:21:19.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: pidfile_write: ignore empty --pid-file 2023-12-14T18:21:19.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: load: jerasure load: lrc 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: RocksDB version: 7.9.2 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Git sha 0 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Compile date 2023-12-13 16:19:48 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: DB SUMMARY 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: DB Session ID: ECBVC8S5Z0QBOX7T19YL 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: CURRENT file: CURRENT 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: IDENTITY file: IDENTITY 2023-12-14T18:21:19.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.error_if_exists: 0 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.create_if_missing: 0 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.paranoid_checks: 1 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-14T18:21:19.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.env: 0x557bca79c720 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.fs: PosixFileSystem 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.info_log: 0x557bcd7fd3c0 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_file_opening_threads: 16 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.statistics: (nil) 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.use_fsync: 0 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_log_file_size: 0 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-14T18:21:19.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.keep_log_file_num: 1000 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.recycle_log_file_num: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.allow_fallocate: 1 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.allow_mmap_reads: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.allow_mmap_writes: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.use_direct_reads: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.create_missing_column_families: 0 2023-12-14T18:21:19.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.db_log_dir: 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.wal_dir: 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.advise_random_on_open: 1 2023-12-14T18:21:19.144 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.db_write_buffer_size: 0 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.write_buffer_manager: 0x557bcca6e3c0 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.rate_limiter: (nil) 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.wal_recovery_mode: 2 2023-12-14T18:21:19.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.enable_thread_tracking: 0 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.enable_pipelined_write: 0 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.unordered_write: 0 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.row_cache: None 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.wal_filter: None 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-14T18:21:19.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.allow_ingest_behind: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.two_write_queues: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.manual_wal_flush: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.wal_compression: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.atomic_flush: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-14T18:21:19.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.log_readahead_size: 0 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.best_efforts_recovery: 0 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.allow_data_in_errors: 0 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.db_host_id: __hostname__ 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_background_jobs: 2 2023-12-14T18:21:19.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_background_compactions: -1 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_subcompactions: 1 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_total_wal_size: 0 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-14T18:21:19.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_open_files: -1 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bytes_per_sync: 0 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_readahead_size: 0 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_background_flushes: -1 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Compression algorithms supported: 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kZSTD supported: 0 2023-12-14T18:21:19.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kXpressCompression supported: 0 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kLZ4HCCompression supported: 1 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kZlibCompression supported: 1 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kSnappyCompression supported: 1 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kLZ4Compression supported: 1 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: kBZip2Compression supported: 0 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-14T18:21:19.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.merge_operator: 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_filter: None 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_filter_factory: None 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.sst_partitioner_factory: None 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557bcd7fd4e0) 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2023-12-14T18:21:19.152 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2023-12-14T18:21:19.153 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x557bccb0f1f0 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2023-12-14T18:21:19.154 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2023-12-14T18:21:19.155 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2023-12-14T18:21:19.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.write_buffer_size: 33554432 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_write_buffer_number: 2 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression: NoCompression 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression: Disabled 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.prefix_extractor: nullptr 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.num_levels: 7 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-14T18:21:19.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:21:19.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.level: 32767 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.strategy: 0 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.enabled: false 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:21:19.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.target_file_size_base: 67108864 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-14T18:21:19.160 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.arena_block_size: 1048576 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-14T18:21:19.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.disable_auto_compactions: 0 2023-12-14T18:21:19.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-14T18:21:19.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-14T18:21:19.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-14T18:21:19.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-14T18:21:19.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-14T18:21:19.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-14T18:21:19.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-14T18:21:19.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.table_properties_collectors: 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.inplace_update_support: 0 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-14T18:21:19.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.bloom_locality: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.max_successive_merges: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.paranoid_file_checks: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.force_consistency_checks: 1 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.report_bg_io_stats: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.ttl: 2592000 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-14T18:21:19.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.enable_blob_files: false 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.min_blob_size: 0 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.blob_file_size: 268435456 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.blob_file_starting_level: 0 2023-12-14T18:21:19.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8e4a4944-73e0-4e58-80cf-63008b21e415 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578078993336, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578078993902, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702578078, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8e4a4944-73e0-4e58-80cf-63008b21e415", "db_session_id": "ECBVC8S5Z0QBOX7T19YL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578078993992, "job": 1, "event": "recovery_finished"} 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-14T18:21:19.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557bccbac000 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: DB pointer 0x557bccb98000 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:21:19.168 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:21:19.169 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-14T18:21:19.170 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.44 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x557bccb0f1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-14T18:21:19.171 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 bash[43813]: 47946612c9cbf71e5ad2a474cde65b5346b6155beac4daaeaaffa6ae22e4fd2c 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:18 smithi204 ceph-mon[43866]: mon.c@-1(???) e0 preinit fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 systemd[1]: Started Ceph mon.c for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).mds e1 new map 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:21:19.172 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-14T18:21:19.173 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.174 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4151493448' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-14T18:21:19.175 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4151493448' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.176 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c;count:3 2023-12-14T18:21:19.177 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: Deploying daemon mon.c on smithi204 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expand map: {default=false} 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta from 'false' to 'false' 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expanded map: {default=false} 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expand map: {default=info} 2023-12-14T18:21:19.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta from 'info' to 'info' 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expanded map: {default=info} 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expand map: {default=daemon} 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expanded map: {default=daemon} 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expand map: {default=debug} 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta from 'debug' to 'debug' 2023-12-14T18:21:19.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:19 smithi204 ceph-mon[43866]: expand_channel_meta expanded map: {default=debug} 2023-12-14T18:21:20.351 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-14T18:21:20.351 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-14T18:21:20.668 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:24.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:24.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: mon.a calling monitor election 2023-12-14T18:21:24.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:24.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: mon.c calling monitor election 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: monmap e2: 2 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} 2023-12-14T18:21:24.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: fsmap 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: mgrmap e13: x(active, since 36s) 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: overall HEALTH_OK 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:24.467 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: mon.a calling monitor election 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: mon.c calling monitor election 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:24.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: monmap e2: 2 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: fsmap 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: mgrmap e13: x(active, since 36s) 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: overall HEALTH_OK 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:24.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 -- 172.21.15.204:0/2726713009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 msgr2=0x7f8930005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 --2- 172.21.15.204:0/2726713009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8930005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f89400fec70 tx=0x7f893400feb0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 -- 172.21.15.204:0/2726713009 shutdown_connections 2023-12-14T18:21:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 --2- 172.21.15.204:0/2726713009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8930005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 -- 172.21.15.204:0/2726713009 >> 172.21.15.204:0/2726713009 conn(0x7f89400fa860 msgr2=0x7f89400fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:24.648 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 -- 172.21.15.204:0/2726713009 shutdown_connections 2023-12-14T18:21:24.649 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.646+0000 7f8946c17700 1 -- 172.21.15.204:0/2726713009 wait complete. 2023-12-14T18:21:24.652 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f8946c17700 1 Processor -- start 2023-12-14T18:21:24.652 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f8946c17700 1 -- start start 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f8946c17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8940195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f8946c17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f8946c17700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940195dd0 con 0x7f89300035c0 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f8946c17700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8940195f40 con 0x7f8940195850 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f893ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:43906/0 (socket says 172.21.15.204:43906) 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.647+0000 7f89449b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8940195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893ffff700 1 -- 172.21.15.204:0/3075234328 learned_addr learned my addr 172.21.15.204:0/3075234328 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:21:24.653 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893ffff700 1 -- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 msgr2=0x7f8940199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893ffff700 1 -- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 msgr2=0x7f8940199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893ffff700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893ffff700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f89449b3700 1 -- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 msgr2=0x7f8940199cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f89449b3700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f89449b3700 1 -- 172.21.15.204:0/3075234328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893400f900 con 0x7f89300035c0 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f89449b3700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8940195310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f893400f800 tx=0x7f89340047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8934013b00 con 0x7f89300035c0 2023-12-14T18:21:24.654 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89340044f0 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8940106b90 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.648+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f8934003d00 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.649+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8940106a00 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.650+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8934011470 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.650+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8940106d20 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.650+0000 7f893dffb700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89240386b0 0x7f892403ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.650+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f893404de50 con 0x7f89300035c0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.650+0000 7f893ffff700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89240386b0 0x7f892403ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:24.655 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.651+0000 7f893ffff700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89240386b0 0x7f892403ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f892c009180 tx=0x7f892c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:24.657 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.655+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f89340109c0 con 0x7f89300035c0 2023-12-14T18:21:24.927 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.925+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f89401967f0 con 0x7f89300035c0 2023-12-14T18:21:24.927 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.927+0000 7f893dffb700 1 -- 172.21.15.204:0/3075234328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f8934003e60 con 0x7f89300035c0 2023-12-14T18:21:24.928 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:21:24.928 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","modified":"2023-12-14T18:21:19.019753Z","created":"2023-12-14T18:19:52.681816Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-14T18:21:24.930 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89240386b0 msgr2=0x7f892403ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:24.930 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89240386b0 0x7f892403ab60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f892c009180 tx=0x7f892c008040 comp rx=0 tx=0).stop 2023-12-14T18:21:24.930 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 msgr2=0x7f8940195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:24.930 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8940195310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f893400f800 tx=0x7f89340047a0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 shutdown_connections 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89300035c0 0x7f8940195310 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89240386b0 0x7f892403ab60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 --2- 172.21.15.204:0/3075234328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8940195850 0x7f8940199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 >> 172.21.15.204:0/3075234328 conn(0x7f89400fa860 msgr2=0x7f8940068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 shutdown_connections 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:24.928+0000 7f8946c17700 1 -- 172.21.15.204:0/3075234328 wait complete. 2023-12-14T18:21:24.931 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2023-12-14T18:21:25.214 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:25 smithi204 ceph-mon[43866]: Deploying daemon mon.b on smithi176 2023-12-14T18:21:25.214 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:25 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/3075234328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:21:25.215 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:25 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:25 smithi031 ceph-mon[42085]: Deploying daemon mon.b on smithi176 2023-12-14T18:21:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:25 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/3075234328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:21:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:25 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:26.247 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-14T18:21:26.248 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-14T18:21:26.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:26 smithi031 ceph-mon[42085]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 systemd[1]: Starting Ceph mon.b for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:21:26.563 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:26 smithi204 ceph-mon[43866]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: set uid:gid to 167:167 (ceph:ceph) 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev), process ceph-mon, pid 7 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pidfile_write: ignore empty --pid-file 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 bash[43468]: a6c8807c56e12e9bee281c938134c65cec6627fb31245506b7f1792ec5755cd3 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: load: jerasure load: lrc 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: RocksDB version: 7.9.2 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Git sha 0 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Compile date 2023-12-13 16:19:48 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: DB SUMMARY 2023-12-14T18:21:26.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: DB Session ID: C1NL5KKLJYH4NELW8O47 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: CURRENT file: CURRENT 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: IDENTITY file: IDENTITY 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.error_if_exists: 0 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.create_if_missing: 0 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.paranoid_checks: 1 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.env: 0x55ceb7503720 2023-12-14T18:21:26.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.fs: PosixFileSystem 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.info_log: 0x55ceb95b93c0 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_file_opening_threads: 16 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.statistics: (nil) 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.use_fsync: 0 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_log_file_size: 0 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.keep_log_file_num: 1000 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.recycle_log_file_num: 0 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.allow_fallocate: 1 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.allow_mmap_reads: 0 2023-12-14T18:21:26.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.allow_mmap_writes: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.use_direct_reads: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.create_missing_column_families: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.db_log_dir: 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.wal_dir: 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.advise_random_on_open: 1 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.db_write_buffer_size: 0 2023-12-14T18:21:26.997 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.write_buffer_manager: 0x55ceb882a3c0 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.rate_limiter: (nil) 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.wal_recovery_mode: 2 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.enable_thread_tracking: 0 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.enable_pipelined_write: 0 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.unordered_write: 0 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.row_cache: None 2023-12-14T18:21:26.998 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.wal_filter: None 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.allow_ingest_behind: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.two_write_queues: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.manual_wal_flush: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.wal_compression: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.atomic_flush: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.log_readahead_size: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.best_efforts_recovery: 0 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-14T18:21:26.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.allow_data_in_errors: 0 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.db_host_id: __hostname__ 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_background_jobs: 2 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_background_compactions: -1 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_subcompactions: 1 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_total_wal_size: 0 2023-12-14T18:21:27.000 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_open_files: -1 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bytes_per_sync: 0 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_readahead_size: 0 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_background_flushes: -1 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Compression algorithms supported: 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kZSTD supported: 0 2023-12-14T18:21:27.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kXpressCompression supported: 0 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kLZ4HCCompression supported: 1 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kZlibCompression supported: 1 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kSnappyCompression supported: 1 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kLZ4Compression supported: 1 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: kBZip2Compression supported: 0 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.merge_operator: 2023-12-14T18:21:27.002 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_filter: None 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_filter_factory: None 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.sst_partitioner_factory: None 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ceb95b94e0) 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: cache_index_and_filter_blocks: 1 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: pin_top_level_index_and_filter: 1 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: index_type: 0 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: data_block_index_type: 0 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: index_shortening: 1 2023-12-14T18:21:27.003 INFO:journalctl@ceph.mon.b.smithi176.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: checksum: 4 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: no_block_cache: 0 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_cache: 0x55ceb88cb1f0 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_cache_name: BinnedLRUCache 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_cache_options: 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: capacity : 536870912 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: num_shard_bits : 4 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: strict_capacity_limit : 0 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: high_pri_pool_ratio: 0.000 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_cache_compressed: (nil) 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: persistent_cache: (nil) 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_size: 4096 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_size_deviation: 10 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_restart_interval: 16 2023-12-14T18:21:27.004 INFO:journalctl@ceph.mon.b.smithi176.stdout: index_block_restart_interval: 1 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: metadata_block_size: 4096 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: partition_filters: 0 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: use_delta_encoding: 1 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: filter_policy: bloomfilter 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: whole_key_filtering: 1 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: verify_compression: 0 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: read_amp_bytes_per_bit: 0 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: format_version: 5 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: enable_index_compression: 1 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: block_align: 0 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: max_auto_readahead_size: 262144 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: prepopulate_block_cache: 0 2023-12-14T18:21:27.005 INFO:journalctl@ceph.mon.b.smithi176.stdout: initial_auto_readahead_size: 8192 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout: num_file_reads_for_auto_readahead: 2 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.write_buffer_size: 33554432 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_write_buffer_number: 2 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression: NoCompression 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression: Disabled 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.prefix_extractor: nullptr 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.num_levels: 7 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-14T18:21:27.006 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.level: 32767 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.strategy: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:21:27.007 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.enabled: false 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.target_file_size_base: 67108864 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-14T18:21:27.008 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.arena_block_size: 1048576 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.disable_auto_compactions: 0 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-14T18:21:27.009 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.table_properties_collectors: 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.inplace_update_support: 0 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.bloom_locality: 0 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.max_successive_merges: 0 2023-12-14T18:21:27.010 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.paranoid_file_checks: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.force_consistency_checks: 1 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.report_bg_io_stats: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.ttl: 2592000 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.enable_blob_files: false 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.min_blob_size: 0 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.blob_file_size: 268435456 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-14T18:21:27.011 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.blob_file_starting_level: 0 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0cec9d7c-4691-491a-8504-1006eafbc2d2 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578086580090, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578086580514, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702578086, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0cec9d7c-4691-491a-8504-1006eafbc2d2", "db_session_id": "C1NL5KKLJYH4NELW8O47", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578086580592, "job": 1, "event": "recovery_finished"} 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 systemd[1]: Started Ceph mon.b for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-14T18:21:27.012 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ceb8968000 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: DB pointer 0x55ceb8954000 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: ** DB Stats ** 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:21:27.013 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: ** Compaction Stats [default] ** 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: ** Compaction Stats [default] ** 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-14T18:21:27.014 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Block cache BinnedLRUCache@0x55ceb88cb1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.5e-05 secs_since: 0 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-14T18:21:27.015 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: using public_addr v2:172.21.15.176:0/0 -> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: starting mon.b rank -1 at public addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] at bind addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(???) e0 preinit fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).mds e1 new map 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout: e1 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout: legacy client fscid: -1 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout: 2023-12-14T18:21:27.016 INFO:journalctl@ceph.mon.b.smithi176.stdout: No filesystems configured 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:21:27.017 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:27.018 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4151493448' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4151493448' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:27.019 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Saving service mon spec with placement smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c;count:3 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Deploying daemon mon.c on smithi204 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.a calling monitor election 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.020 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.c calling monitor election 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: monmap e2: 2 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: fsmap 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mgrmap e13: x(active, since 36s) 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: overall HEALTH_OK 2023-12-14T18:21:27.021 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: Deploying daemon mon.b on smithi176 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/3075234328' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expand map: {default=false} 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta from 'false' to 'false' 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expanded map: {default=false} 2023-12-14T18:21:27.022 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expand map: {default=info} 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta from 'info' to 'info' 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expanded map: {default=info} 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expand map: {default=daemon} 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expanded map: {default=daemon} 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expand map: {default=debug} 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta from 'debug' to 'debug' 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: expand_channel_meta expanded map: {default=debug} 2023-12-14T18:21:27.023 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:26 smithi176 ceph-mon[43506]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/2739789585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d00035c0 msgr2=0x7fd9d0005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/2739789585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d00035c0 0x7fd9d0005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80fec70 tx=0x7fd9d800fe60 comp rx=0 tx=0).stop 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/2739789585 shutdown_connections 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/2739789585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9d00035c0 0x7fd9d0005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/2739789585 >> 172.21.15.204:0/2739789585 conn(0x7fd9e80fa860 msgr2=0x7fd9e80fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/2739789585 shutdown_connections 2023-12-14T18:21:31.913 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.906+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/2739789585 wait complete. 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.907+0000 7fd9ee6b2700 1 Processor -- start 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.907+0000 7fd9ee6b2700 1 -- start start 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ee6b2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ee6b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9e81947c0 0x7fd9e8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ee6b2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 0x7fd9e8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ee6b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e8194e90 con 0x7fd9e81947c0 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ee6b2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e81a0ee0 con 0x7fd9e8199170 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ee6b2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e81a1020 con 0x7fd9d00035c0 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:31.914 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:36856/0 (socket says 172.21.15.204:36856) 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 -- 172.21.15.204:0/1785169063 learned_addr learned my addr 172.21.15.204:0/1785169063 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e77fe700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9e81947c0 0x7fd9e8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 msgr2=0x7fd9e8194280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 msgr2=0x7fd9e8194280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e7fff700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ecc4f700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 0x7fd9e8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ecc4f700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 msgr2=0x7fd9e8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ecc4f700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 msgr2=0x7fd9e8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:21:31.915 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ecc4f700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 0x7fd9e8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9ecc4f700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 0x7fd9e8199610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e77fe700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 msgr2=0x7fd9e8194280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e77fe700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e77fe700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 msgr2=0x7fd9e8199610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e77fe700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 0x7fd9e8199610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.908+0000 7fd9e77fe700 1 -- 172.21.15.204:0/1785169063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d800f8b0 con 0x7fd9e81947c0 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.909+0000 7fd9e77fe700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9e81947c0 0x7fd9e8198c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b580 tx=0x7fd9dc00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.909+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc004230 con 0x7fd9e81947c0 2023-12-14T18:21:31.916 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.909+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9dc004390 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.909+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e8106b40 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.909+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc011a70 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.909+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e81069b0 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.910+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd9dc011c10 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.910+0000 7fd9e57fa700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd9d4038d40 0x7fd9d403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.910+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd9dc04bcf0 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.910+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e8106cd0 con 0x7fd9e81947c0 2023-12-14T18:21:31.917 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.911+0000 7fd9e7fff700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd9d4038d40 0x7fd9d403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:31.918 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.911+0000 7fd9e7fff700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd9d4038d40 0x7fd9d403b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8005d10 tx=0x7fd9d8005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:31.918 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:31.916+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd9dc01a400 con 0x7fd9e81947c0 2023-12-14T18:21:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: mon.a calling monitor election 2023-12-14T18:21:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: mon.c calling monitor election 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: mon.b calling monitor election 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:31.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: monmap e3: 3 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],b=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: fsmap 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: mgrmap e13: x(active, since 44s) 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: overall HEALTH_OK 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:31.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:32.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: mon.a calling monitor election 2023-12-14T18:21:32.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: mon.c calling monitor election 2023-12-14T18:21:32.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: mon.b calling monitor election 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-14T18:21:32.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: monmap e3: 3 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],b=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: fsmap 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: mgrmap e13: x(active, since 44s) 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: overall HEALTH_OK 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.201 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.200+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd9e81a1ba0 con 0x7fd9e81947c0 2023-12-14T18:21:32.202 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.200+0000 7fd9e57fa700 1 -- 172.21.15.204:0/1785169063 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1276 (secure 0 0 0) 0x7fd9dc049020 con 0x7fd9e81947c0 2023-12-14T18:21:32.202 INFO:teuthology.orchestra.run.smithi204.stdout: 2023-12-14T18:21:32.202 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","modified":"2023-12-14T18:21:26.605845Z","created":"2023-12-14T18:19:52.681816Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-12-14T18:21:32.205 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd9d4038d40 msgr2=0x7fd9d403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:32.205 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd9d4038d40 0x7fd9d403b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8005d10 tx=0x7fd9d8005c80 comp rx=0 tx=0).stop 2023-12-14T18:21:32.205 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9e81947c0 msgr2=0x7fd9e8198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:32.205 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9e81947c0 0x7fd9e8198c30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b580 tx=0x7fd9dc00b940 comp rx=0 tx=0).stop 2023-12-14T18:21:32.205 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 shutdown_connections 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd9e8199170 0x7fd9e8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd9d4038d40 0x7fd9d403b1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9d00035c0 0x7fd9e8194280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 --2- 172.21.15.204:0/1785169063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9e81947c0 0x7fd9e8198c30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 >> 172.21.15.204:0/1785169063 conn(0x7fd9e80fa860 msgr2=0x7fd9e818d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 shutdown_connections 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:32.203+0000 7fd9ee6b2700 1 -- 172.21.15.204:0/1785169063 wait complete. 2023-12-14T18:21:32.206 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: mon.a calling monitor election 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: mon.c calling monitor election 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: mon.b calling monitor election 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: monmap e3: 3 mons at {a=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],b=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: fsmap 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: mgrmap e13: x(active, since 44s) 2023-12-14T18:21:32.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: overall HEALTH_OK 2023-12-14T18:21:32.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.673 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-14T18:21:32.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph config generate-minimal-conf 2023-12-14T18:21:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:32 smithi031 ceph-mon[42085]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:32 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/1785169063' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:21:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:32 smithi176 ceph-mon[43506]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:32 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/1785169063' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:21:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:33.031 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:32 smithi204 ceph-mon[43866]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:32 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/1785169063' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:21:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:33.922 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.923 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:33.924 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.101+0000 7f59314aa700 1 -- 172.21.15.31:0/385583522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c0fed90 msgr2=0x7f592c0ff160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.101+0000 7f59314aa700 1 --2- 172.21.15.31:0/385583522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c0fed90 0x7f592c0ff160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f591c009910 tx=0x7f591c009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.101+0000 7f59314aa700 1 -- 172.21.15.31:0/385583522 shutdown_connections 2023-12-14T18:21:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.101+0000 7f59314aa700 1 --2- 172.21.15.31:0/385583522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c0fed90 0x7f592c0ff160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.102+0000 7f59314aa700 1 -- 172.21.15.31:0/385583522 >> 172.21.15.31:0/385583522 conn(0x7f592c074350 msgr2=0x7f592c074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.102+0000 7f59314aa700 1 -- 172.21.15.31:0/385583522 shutdown_connections 2023-12-14T18:21:34.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.102+0000 7f59314aa700 1 -- 172.21.15.31:0/385583522 wait complete. 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.102+0000 7f59314aa700 1 Processor -- start 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 -- start start 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c0fed90 0x7f592c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 0x7f592c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c19c3f0 0x7f592c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c198180 con 0x7f592c197ab0 2023-12-14T18:21:34.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1a4160 con 0x7f592c0fed90 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f59314aa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1a42a0 con 0x7f592c19c3f0 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 0x7f592c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 0x7f592c19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54016/0 (socket says 172.21.15.31:54016) 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 -- 172.21.15.31:0/1752530939 learned_addr learned my addr 172.21.15.31:0/1752530939 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592affd700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c0fed90 0x7f592c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592b7fe700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c19c3f0 0x7f592c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 -- 172.21.15.31:0/1752530939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c19c3f0 msgr2=0x7f592c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c19c3f0 0x7f592c19c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 -- 172.21.15.31:0/1752530939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c0fed90 msgr2=0x7f592c197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c0fed90 0x7f592c197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592a7fc700 1 -- 172.21.15.31:0/1752530939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0095f0 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.103+0000 7f592b7fe700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c19c3f0 0x7f592c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.104+0000 7f592a7fc700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 0x7f592c19beb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f592000b7b0 tx=0x7f592000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.104+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920004230 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.104+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5920004390 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.104+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c10a130 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.104+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920011ad0 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.104+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c109fa0 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.105+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5920011c30 con 0x7f592c197ab0 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.105+0000 7f5913fff700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5914038cf0 0x7f591403b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:34.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.105+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f592004ac40 con 0x7f592c197ab0 2023-12-14T18:21:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.105+0000 7f592affd700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5914038cf0 0x7f591403b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.105+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c10a2c0 con 0x7f592c197ab0 2023-12-14T18:21:34.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.106+0000 7f592affd700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5914038cf0 0x7f591403b1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f591c006010 tx=0x7f591c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.110+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f592001a400 con 0x7f592c197ab0 2023-12-14T18:21:34.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:34.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.332+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f592c1a4e20 con 0x7f592c197ab0 2023-12-14T18:21:34.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.333+0000 7f5913fff700 1 -- 172.21.15.31:0/1752530939 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f5920048030 con 0x7f592c197ab0 2023-12-14T18:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2023-12-14T18:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:21:34.334 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2023-12-14T18:21:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5914038cf0 msgr2=0x7f591403b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:34.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5914038cf0 0x7f591403b1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f591c006010 tx=0x7f591c00b540 comp rx=0 tx=0).stop 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 msgr2=0x7f592c19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 0x7f592c19beb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f592000b7b0 tx=0x7f592000bac0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 shutdown_connections 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5914038cf0 0x7f591403b1a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f592c19c3f0 0x7f592c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c0fed90 0x7f592c197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 --2- 172.21.15.31:0/1752530939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f592c197ab0 0x7f592c19beb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 >> 172.21.15.31:0/1752530939 conn(0x7f592c074350 msgr2=0x7f592c106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:34.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 shutdown_connections 2023-12-14T18:21:34.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:34.335+0000 7f59314aa700 1 -- 172.21.15.31:0/1752530939 wait complete. 2023-12-14T18:21:34.816 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-14T18:21:34.816 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:21:34.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2023-12-14T18:21:34.854 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:21:34.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:34.928 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Reconfiguring mon.a (unknown last config time)... 2023-12-14T18:21:34.929 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:34.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:34.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:34.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: Reconfiguring daemon mon.a on smithi031 2023-12-14T18:21:34.930 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:34 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1752530939' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:34.931 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:21:34.931 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.conf 2023-12-14T18:21:34.966 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:21:34.967 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:35.033 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:35.033 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:21:35.033 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:21:35.033 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Reconfiguring mon.a (unknown last config time)... 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:35.034 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:35.035 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: Reconfiguring daemon mon.a on smithi031 2023-12-14T18:21:35.035 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:34 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1752530939' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:35.037 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:21:35.037 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2023-12-14T18:21:35.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:35.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:21:35.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:21:35.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:21:35.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:35.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Reconfiguring mon.a (unknown last config time)... 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: Reconfiguring daemon mon.a on smithi031 2023-12-14T18:21:35.070 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:34 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1752530939' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:35.074 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:21:35.074 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:21:35.145 INFO:tasks.cephadm:Adding mgr.x on smithi031 2023-12-14T18:21:35.145 INFO:tasks.cephadm:Adding mgr.y on smithi176 2023-12-14T18:21:35.145 INFO:tasks.cephadm:Adding mgr.z on smithi204 2023-12-14T18:21:35.146 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch apply mgr '3;smithi031=x;smithi176=y;smithi204=z' 2023-12-14T18:21:35.486 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: Reconfiguring mon.b (monmap changed)... 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: Reconfiguring daemon mon.b on smithi176 2023-12-14T18:21:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:36 smithi176 ceph-mon[43506]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:36.310 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: Reconfiguring mon.b (monmap changed)... 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: Reconfiguring daemon mon.b on smithi176 2023-12-14T18:21:36.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:36 smithi204 ceph-mon[43866]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:36.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:36.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:36.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: Reconfiguring mon.b (monmap changed)... 2023-12-14T18:21:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: Reconfiguring daemon mon.b on smithi176 2023-12-14T18:21:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:36 smithi031 ceph-mon[42085]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.481+0000 7f765b936700 1 -- 172.21.15.204:0/2133896519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76540fed20 msgr2=0x7f76540ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.481+0000 7f765b936700 1 --2- 172.21.15.204:0/2133896519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76540fed20 0x7f76540ff130 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7650009910 tx=0x7f7650009c20 comp rx=0 tx=0).stop 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.481+0000 7f765b936700 1 -- 172.21.15.204:0/2133896519 shutdown_connections 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.481+0000 7f765b936700 1 --2- 172.21.15.204:0/2133896519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76540fed20 0x7f76540ff130 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.481+0000 7f765b936700 1 -- 172.21.15.204:0/2133896519 >> 172.21.15.204:0/2133896519 conn(0x7f76540fa390 msgr2=0x7f76540fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.481+0000 7f765b936700 1 -- 172.21.15.204:0/2133896519 shutdown_connections 2023-12-14T18:21:36.483 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 -- 172.21.15.204:0/2133896519 wait complete. 2023-12-14T18:21:36.485 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 Processor -- start 2023-12-14T18:21:36.485 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 -- start start 2023-12-14T18:21:36.485 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 0x7f7654193f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:36.485 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7654194480 0x7f76541994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:36.485 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76541948f0 0x7f765419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:36.485 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419c200 con 0x7f76541948f0 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419c370 con 0x7f7654194480 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.482+0000 7f765b936700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765419c530 con 0x7f76540fed20 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 0x7f7654193f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 0x7f7654193f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38274/0 (socket says 172.21.15.204:38274) 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 -- 172.21.15.204:0/935819712 learned_addr learned my addr 172.21.15.204:0/935819712 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f7659ed3700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76541948f0 0x7f765419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f7658ed1700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7654194480 0x7f76541994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 -- 172.21.15.204:0/935819712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7654194480 msgr2=0x7f76541994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7654194480 0x7f76541994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 -- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76541948f0 msgr2=0x7f765419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76541948f0 0x7f765419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.486 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 -- 172.21.15.204:0/935819712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500095f0 con 0x7f76540fed20 2023-12-14T18:21:36.487 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f7658ed1700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7654194480 0x7f76541994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:21:36.487 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f76596d2700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 0x7f7654193f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7650005950 tx=0x7f7650004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:36.487 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765001d070 con 0x7f76540fed20 2023-12-14T18:21:36.487 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f765000be20 con 0x7f76540fed20 2023-12-14T18:21:36.487 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650022d80 con 0x7f76540fed20 2023-12-14T18:21:36.487 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.483+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7654104b70 con 0x7f76540fed20 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.484+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76541049e0 con 0x7f76540fed20 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.484+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f765000f460 con 0x7f76540fed20 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.484+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765418d8a0 con 0x7f76540fed20 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.485+0000 7f764a7fc700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7640038d40 0x7f764003b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.485+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f765004c720 con 0x7f76540fed20 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.485+0000 7f7658ed1700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7640038d40 0x7f764003b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:36.488 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.485+0000 7f7658ed1700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7640038d40 0x7f764003b1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7644009ec0 tx=0x7f7644009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:36.491 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.490+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7650020730 con 0x7f76540fed20 2023-12-14T18:21:36.703 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.702+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi176=y;smithi204=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f7654061ae0 con 0x7f7640038d40 2023-12-14T18:21:36.710 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.708+0000 7f764a7fc700 1 -- 172.21.15.204:0/935819712 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7654061ae0 con 0x7f7640038d40 2023-12-14T18:21:36.710 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7640038d40 msgr2=0x7f764003b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7640038d40 0x7f764003b1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7644009ec0 tx=0x7f7644009210 comp rx=0 tx=0).stop 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 msgr2=0x7f7654193f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 0x7f7654193f40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7650005950 tx=0x7f7650004cb0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 shutdown_connections 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76541948f0 0x7f765419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7640038d40 0x7f764003b1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.712 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76540fed20 0x7f7654193f40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.713 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 --2- 172.21.15.204:0/935819712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7654194480 0x7f76541994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:36.713 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.710+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 >> 172.21.15.204:0/935819712 conn(0x7f76540fa390 msgr2=0x7f76540faff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:36.713 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.711+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 shutdown_connections 2023-12-14T18:21:36.713 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:21:36.711+0000 7f765b936700 1 -- 172.21.15.204:0/935819712 wait complete. 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: Reconfiguring mon.c (monmap changed)... 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:37.263 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: Reconfiguring daemon mon.c on smithi204 2023-12-14T18:21:37.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi176=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:37.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: Saving service mgr spec with placement smithi031=x;smithi176=y;smithi204=z;count:3 2023-12-14T18:21:37.264 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.281 DEBUG:teuthology.orchestra.run.smithi176:mgr.y> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.y.service 2023-12-14T18:21:37.284 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.z.service 2023-12-14T18:21:37.286 INFO:tasks.cephadm:Deploying OSDs... 2023-12-14T18:21:37.286 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:21:37.286 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2023-12-14T18:21:37.308 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-14T18:21:37.309 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2023-12-14T18:21:37.312 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.312 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.312 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: Reconfiguring mon.c (monmap changed)... 2023-12-14T18:21:37.312 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:37.312 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:37.312 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:37.313 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: Reconfiguring daemon mon.c on smithi204 2023-12-14T18:21:37.313 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi176=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:37.313 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: Saving service mgr spec with placement smithi031=x;smithi176=y;smithi204=z;count:3 2023-12-14T18:21:37.313 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.315 INFO:journalctl@ceph.mgr.z.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:21:37.318 INFO:journalctl@ceph.mgr.y.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:21:37.370 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 122560 Links: 1 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-12-14 18:20:56.677635485 +0000 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-12-14 18:14:02.190070357 +0000 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-12-14 18:14:02.190070357 +0000 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-12-14T18:21:37.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: Reconfiguring mon.c (monmap changed)... 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: Reconfiguring daemon mon.c on smithi204 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi031=x;smithi176=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: Saving service mgr spec with placement smithi031=x;smithi176=y;smithi204=z;count:3 2023-12-14T18:21:37.441 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:37.444 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-12-14T18:21:37.444 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-12-14T18:21:37.444 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000298163 s, 1.7 MB/s 2023-12-14T18:21:37.445 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-14T18:21:37.509 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2023-12-14T18:21:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-14T18:21:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:37.572 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 122655 Links: 1 2023-12-14T18:21:37.573 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:37.573 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:37.573 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-12-14 18:20:56.677635485 +0000 2023-12-14T18:21:37.573 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-12-14 18:14:02.875058677 +0000 2023-12-14T18:21:37.573 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-12-14 18:14:02.875058677 +0000 2023-12-14T18:21:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-12-14T18:21:37.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-14T18:21:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-12-14T18:21:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-12-14T18:21:37.646 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000330219 s, 1.6 MB/s 2023-12-14T18:21:37.648 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-14T18:21:37.711 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 127382 Links: 1 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-12-14 18:20:56.677635485 +0000 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-12-14 18:14:03.431049196 +0000 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-12-14 18:14:03.431049196 +0000 2023-12-14T18:21:37.773 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-12-14T18:21:37.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-14T18:21:37.846 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-12-14T18:21:37.846 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-12-14T18:21:37.846 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000238119 s, 2.2 MB/s 2023-12-14T18:21:37.847 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-14T18:21:37.910 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2023-12-14T18:21:37.973 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-14T18:21:37.973 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 129204 Links: 1 2023-12-14T18:21:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-12-14 18:20:56.677635485 +0000 2023-12-14T18:21:37.974 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-12-14 18:14:03.987039716 +0000 2023-12-14T18:21:37.974 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-12-14 18:14:03.987039716 +0000 2023-12-14T18:21:37.974 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-12-14T18:21:37.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-14T18:21:38.047 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-12-14T18:21:38.048 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-12-14T18:21:38.048 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00025149 s, 2.0 MB/s 2023-12-14T18:21:38.049 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-14T18:21:38.113 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2023-12-14T18:21:38.113 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2023-12-14T18:21:38.134 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-14T18:21:38.134 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2023-12-14T18:21:38.194 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-14T18:21:38.194 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:38.194 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 6h/6d Inode: 119260 Links: 1 2023-12-14T18:21:38.194 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:38.195 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:38.195 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2023-12-14 18:21:26.189064020 +0000 2023-12-14T18:21:38.195 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2023-12-14 18:12:59.133500038 +0000 2023-12-14T18:21:38.195 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2023-12-14 18:12:59.133500038 +0000 2023-12-14T18:21:38.195 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2023-12-14T18:21:38.195 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-14T18:21:38.264 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2023-12-14T18:21:38.264 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2023-12-14T18:21:38.264 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000271445 s, 1.9 MB/s 2023-12-14T18:21:38.265 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-14T18:21:38.325 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 6h/6d Inode: 119525 Links: 1 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2023-12-14 18:21:26.189064020 +0000 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2023-12-14 18:12:59.666494926 +0000 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2023-12-14 18:12:59.666494926 +0000 2023-12-14T18:21:38.384 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2023-12-14T18:21:38.385 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-14T18:21:38.451 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2023-12-14T18:21:38.452 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2023-12-14T18:21:38.452 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000233816 s, 2.2 MB/s 2023-12-14T18:21:38.453 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-14T18:21:38.512 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2023-12-14T18:21:38.573 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 6h/6d Inode: 128228 Links: 1 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2023-12-14 18:21:26.189064020 +0000 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2023-12-14 18:13:00.173490064 +0000 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2023-12-14 18:13:00.173490064 +0000 2023-12-14T18:21:38.574 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2023-12-14T18:21:38.575 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:38.640 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:38.641 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:38 smithi176 ceph-mon[43506]: Deploying daemon mgr.z on smithi204 2023-12-14T18:21:38.643 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2023-12-14T18:21:38.643 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2023-12-14T18:21:38.643 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000224807 s, 2.3 MB/s 2023-12-14T18:21:38.644 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-14T18:21:38.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:38.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.678 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:38.679 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:38.680 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:38 smithi204 ceph-mon[43866]: Deploying daemon mgr.z on smithi204 2023-12-14T18:21:38.703 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 6h/6d Inode: 119679 Links: 1 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2023-12-14 18:21:26.189064020 +0000 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2023-12-14 18:13:00.660485394 +0000 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2023-12-14 18:13:00.660485394 +0000 2023-12-14T18:21:38.763 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2023-12-14T18:21:38.764 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-14T18:21:38.830 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2023-12-14T18:21:38.830 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2023-12-14T18:21:38.830 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000248877 s, 2.1 MB/s 2023-12-14T18:21:38.832 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-14T18:21:38.891 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2023-12-14T18:21:38.891 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2023-12-14T18:21:38.910 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-14T18:21:38.911 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:38.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:38.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:21:38.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:38.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:21:38.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:38.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:38.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:38 smithi031 ceph-mon[42085]: Deploying daemon mgr.z on smithi204 2023-12-14T18:21:38.968 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-14T18:21:38.968 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:38.968 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 122701 Links: 1 2023-12-14T18:21:38.968 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:38.969 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:38.969 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2023-12-14 18:21:38.725147606 +0000 2023-12-14T18:21:38.969 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2023-12-14 18:13:05.475340036 +0000 2023-12-14T18:21:38.969 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2023-12-14 18:13:05.475340036 +0000 2023-12-14T18:21:38.969 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2023-12-14T18:21:38.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-14T18:21:39.014 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:38 smithi204 systemd[1]: Starting Ceph mgr.z for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:21:39.037 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2023-12-14T18:21:39.037 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2023-12-14T18:21:39.038 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000236913 s, 2.2 MB/s 2023-12-14T18:21:39.039 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-14T18:21:39.097 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2023-12-14T18:21:39.157 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-14T18:21:39.157 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:39.157 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 125321 Links: 1 2023-12-14T18:21:39.157 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:39.158 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:39.158 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2023-12-14 18:21:38.725147606 +0000 2023-12-14T18:21:39.158 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2023-12-14 18:13:05.994332824 +0000 2023-12-14T18:21:39.158 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2023-12-14 18:13:05.994332824 +0000 2023-12-14T18:21:39.158 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2023-12-14T18:21:39.158 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-14T18:21:39.225 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2023-12-14T18:21:39.225 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2023-12-14T18:21:39.225 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000215978 s, 2.4 MB/s 2023-12-14T18:21:39.227 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-14T18:21:39.286 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2023-12-14T18:21:39.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 ceph-mgr[45807]: -- 172.21.15.204:0/1645693320 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6a7e8d520 con 0x55b6a7e62c00 2023-12-14T18:21:39.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 bash[45688]: 98094b690650cab19faf79720083d2e2efd61856c8c47426a6a49c33432c7436 2023-12-14T18:21:39.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 systemd[1]: Started Ceph mgr.z for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:21:39.344 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-14T18:21:39.344 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:39.344 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 129055 Links: 1 2023-12-14T18:21:39.344 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:39.344 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:39.345 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2023-12-14 18:21:38.725147606 +0000 2023-12-14T18:21:39.345 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2023-12-14 18:13:06.496325849 +0000 2023-12-14T18:21:39.345 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2023-12-14 18:13:06.496325849 +0000 2023-12-14T18:21:39.345 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2023-12-14T18:21:39.345 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-14T18:21:39.410 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2023-12-14T18:21:39.410 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2023-12-14T18:21:39.410 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000235058 s, 2.2 MB/s 2023-12-14T18:21:39.412 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-14T18:21:39.472 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 6h/6d Inode: 125549 Links: 1 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2023-12-14 18:21:38.725147606 +0000 2023-12-14T18:21:39.532 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2023-12-14 18:13:06.997318887 +0000 2023-12-14T18:21:39.533 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2023-12-14 18:13:06.997318887 +0000 2023-12-14T18:21:39.533 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: - 2023-12-14T18:21:39.533 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-14T18:21:39.599 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2023-12-14T18:21:39.599 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2023-12-14T18:21:39.599 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000244757 s, 2.1 MB/s 2023-12-14T18:21:39.600 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-14T18:21:39.660 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2023-12-14T18:21:39.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-14T18:21:39.688 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:39.484+0000 7fbd09bb1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:21:39.688 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:39.593+0000 7fbd09bb1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:21:40.007 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:40.089 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:39.688+0000 7fbd09bb1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:21:40.089 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:39.772+0000 7fbd09bb1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:21:40.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:40.403 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:21:40.404 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:40.404 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:40.404 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: Deploying daemon mgr.y on smithi176 2023-12-14T18:21:40.404 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mon[43506]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:21:40.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:40.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:40.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: Deploying daemon mgr.y on smithi176 2023-12-14T18:21:40.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:40 smithi031 ceph-mon[42085]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:40.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:40.396+0000 7fbd09bb1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:21:40.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: Deploying daemon mgr.y on smithi176 2023-12-14T18:21:40.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-mon[43866]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:40.702 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:40 smithi176 systemd[1]: Starting Ceph mgr.y for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:21:40.919 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:40.631+0000 7fbd09bb1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:21:40.919 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:40.716+0000 7fbd09bb1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:21:40.919 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:40.827+0000 7fbd09bb1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:21:40.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:40 smithi176 ceph-mgr[44977]: -- 172.21.15.176:0/3041329627 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bb26903520 con 0x55bb268d8c00 2023-12-14T18:21:40.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:40 smithi176 bash[44939]: 39a0076bb17812f35a52760cce65644e6e98d00d02fa5b642822e1305b3a6b3d 2023-12-14T18:21:40.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:40 smithi176 systemd[1]: Started Ceph mgr.y for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:21:41.204 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:40.919+0000 7fbd09bb1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:21:41.204 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:41.116+0000 7fbd09bb1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:21:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-14T18:21:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-14T18:21:41.355 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-12-14T18:21:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-12-14T18:21:41.355 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120586 s, 870 MB/s 2023-12-14T18:21:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-12-14T18:21:41.539 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:41.249+0000 7fa1fed1c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:21:41.540 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:41.357+0000 7fa1fed1c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:21:41.540 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:41.447+0000 7fa1fed1c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:21:41.540 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:41.538+0000 7fa1fed1c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:21:41.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2023-12-14T18:21:41.914 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:41.203+0000 7fbd09bb1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:21:41.914 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:41.622+0000 7fbd09bb1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:21:41.988 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:41.988 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:41 smithi176 ceph-mon[43506]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:42.095 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:42.219 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:41.913+0000 7fbd09bb1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:42.220 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:41 smithi204 ceph-mon[43866]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:42.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.196+0000 7fa1fed1c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:21:42.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:42.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:42.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:42.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:42.255 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:41 smithi031 ceph-mon[42085]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:42.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:42.218+0000 7fbd09bb1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:21:42.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:42.316+0000 7fbd09bb1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:21:42.720 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.436+0000 7fa1fed1c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:21:42.720 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.521+0000 7fa1fed1c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:21:42.720 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.629+0000 7fa1fed1c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:21:42.720 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.718+0000 7fa1fed1c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:21:42.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.912+0000 7fa1fed1c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:21:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 -- 172.21.15.31:0/643520469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c0ff1e0 msgr2=0x7fea7c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 --2- 172.21.15.31:0/643520469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c0ff1e0 0x7fea7c0ff5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea78009990 tx=0x7fea78009ca0 comp rx=0 tx=0).stop 2023-12-14T18:21:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 -- 172.21.15.31:0/643520469 shutdown_connections 2023-12-14T18:21:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 --2- 172.21.15.31:0/643520469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7c100d70 0x7fea7c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 --2- 172.21.15.31:0/643520469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c0ff1e0 0x7fea7c0ff5f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:43.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 --2- 172.21.15.31:0/643520469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 -- 172.21.15.31:0/643520469 >> 172.21.15.31:0/643520469 conn(0x7fea7c0fa7f0 msgr2=0x7fea7c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 -- 172.21.15.31:0/643520469 shutdown_connections 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.226+0000 7fea833df700 1 -- 172.21.15.31:0/643520469 wait complete. 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 Processor -- start 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 -- start start 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7c0ff1e0 0x7fea7c1960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c1965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c100d70 0x7fea7c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:43.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c19dc40 con 0x7fea7c100d70 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c19dd80 con 0x7fea7c0ff1e0 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea833df700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c19df10 con 0x7fea7c1003e0 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea8197c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c100d70 0x7fea7c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea8097a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c1965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea8097a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c1965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:39930/0 (socket says 172.21.15.31:39930) 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea8097a700 1 -- 172.21.15.31:0/2257065296 learned_addr learned my addr 172.21.15.31:0/2257065296 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.227+0000 7fea8117b700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7c0ff1e0 0x7fea7c1960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:43.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8197c700 1 -- 172.21.15.31:0/2257065296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 msgr2=0x7fea7c1965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8197c700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c1965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8197c700 1 -- 172.21.15.31:0/2257065296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7c0ff1e0 msgr2=0x7fea7c1960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8197c700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7c0ff1e0 0x7fea7c1960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8197c700 1 -- 172.21.15.31:0/2257065296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea780095f0 con 0x7fea7c100d70 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8097a700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c1965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea8197c700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c100d70 0x7fea7c19d670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea74007f40 tx=0x7fea7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74004db0 con 0x7fea7c100d70 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c10ab90 con 0x7fea7c100d70 2023-12-14T18:21:43.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea7400f040 con 0x7fea7c100d70 2023-12-14T18:21:43.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740059d0 con 0x7fea7c100d70 2023-12-14T18:21:43.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.228+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c10aa00 con 0x7fea7c100d70 2023-12-14T18:21:43.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.229+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fea74015400 con 0x7fea7c100d70 2023-12-14T18:21:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.229+0000 7fea727fc700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fea680389e0 0x7fea6803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.229+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c066ea0 con 0x7fea7c100d70 2023-12-14T18:21:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.229+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea7404ab10 con 0x7fea7c100d70 2023-12-14T18:21:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.229+0000 7fea8117b700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fea680389e0 0x7fea6803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:43.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.230+0000 7fea8117b700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fea680389e0 0x7fea6803ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b5c0 tx=0x7fea78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:43.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.234+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fea740138f0 con 0x7fea7c100d70 2023-12-14T18:21:43.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:43.071+0000 7fbd09bb1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:21:43.378 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:43.168+0000 7fbd09bb1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:21:43.378 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:43.317+0000 7fbd09bb1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:21:43.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: Reconfiguring mgr.x (unknown last config time)... 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: Reconfiguring daemon mgr.x on smithi031 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.380 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:42.996+0000 7fa1fed1c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:43.381 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: Reconfiguring mgr.x (unknown last config time)... 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: Reconfiguring daemon mgr.x on smithi031 2023-12-14T18:21:43.382 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.415 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: Reconfiguring mgr.x (unknown last config time)... 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: Reconfiguring daemon mgr.x on smithi031 2023-12-14T18:21:43.416 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:43.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:43.456+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fea7c068d00 con 0x7fea680389e0 2023-12-14T18:21:43.634 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:43.368+0000 7fa1fed1c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:21:43.920 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:43.633+0000 7fa1fed1c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:21:44.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:43 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:43.918+0000 7fa1fed1c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:21:44.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:44.011+0000 7fa1fed1c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:21:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:44.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:44.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:44.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:21:44.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:44.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:44.317 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:44.318 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:44.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:44.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:21:44.319 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:44 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:44.336 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:43.954+0000 7fbd09bb1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:21:44.336 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:44.139+0000 7fbd09bb1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:21:44.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:44.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:44.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:44.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:44.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:44.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:21:44.338 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:45.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:44.758+0000 7fa1fed1c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:21:45.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:44.853+0000 7fa1fed1c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:21:45.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:45.000+0000 7fa1fed1c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:21:45.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:44 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:44.335+0000 7fbd09bb1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:21:45.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:45.047+0000 7fbd09bb1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:21:45.807 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:45.625+0000 7fa1fed1c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:21:45.808 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/761275602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ee717b-c58b-4f49-b38f-1ba95d414f5b"}]: dispatch 2023-12-14T18:21:45.808 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/761275602' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60ee717b-c58b-4f49-b38f-1ba95d414f5b"}]': finished 2023-12-14T18:21:45.808 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-mon[43506]: osdmap e5: 1 total, 0 up, 1 in 2023-12-14T18:21:45.808 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:21:45.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/761275602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ee717b-c58b-4f49-b38f-1ba95d414f5b"}]: dispatch 2023-12-14T18:21:45.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/761275602' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60ee717b-c58b-4f49-b38f-1ba95d414f5b"}]': finished 2023-12-14T18:21:45.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-mon[43866]: osdmap e5: 1 total, 0 up, 1 in 2023-12-14T18:21:45.809 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:21:45.810 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:45.527+0000 7fbd09bb1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:21:45.810 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:45.616+0000 7fbd09bb1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:21:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:45 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/761275602' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ee717b-c58b-4f49-b38f-1ba95d414f5b"}]: dispatch 2023-12-14T18:21:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:45 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/761275602' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60ee717b-c58b-4f49-b38f-1ba95d414f5b"}]': finished 2023-12-14T18:21:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:45 smithi031 ceph-mon[42085]: osdmap e5: 1 total, 0 up, 1 in 2023-12-14T18:21:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:45 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:21:46.089 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:21:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:21:45.808+0000 7fbd09bb1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:21:46.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:45.805+0000 7fa1fed1c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:21:46.242 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:45.999+0000 7fa1fed1c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:21:46.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:46.509+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7fea74005000 con 0x7fea7c100d70 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: Standby manager daemon z started 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:21:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:46 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/674718643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:46.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:46.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: Standby manager daemon z started 2023-12-14T18:21:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:21:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:21:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:21:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:21:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:46 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/674718643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:46.705+0000 7fa1fed1c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: Standby manager daemon z started 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/1153646686' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:21:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:46 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/674718643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:47.454 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:47.177+0000 7fa1fed1c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:21:47.454 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:47.264+0000 7fa1fed1c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:21:47.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:47.518+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fea74015ad0 con 0x7fea7c100d70 2023-12-14T18:21:47.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:21:47.453+0000 7fa1fed1c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: mgrmap e14: x(active, since 59s), standbys: z 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: Standby manager daemon y started 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:21:47.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:47 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: mgrmap e14: x(active, since 59s), standbys: z 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: Standby manager daemon y started 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:21:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:47 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:21:47.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: mgrmap e14: x(active, since 59s), standbys: z 2023-12-14T18:21:47.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:21:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: Standby manager daemon y started 2023-12-14T18:21:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:21:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:21:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:21:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:47 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/953094615' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:21:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:48 smithi204 ceph-mon[43866]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:48 smithi204 ceph-mon[43866]: mgrmap e15: x(active, since 60s), standbys: z, y 2023-12-14T18:21:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:21:48.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:48 smithi031 ceph-mon[42085]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:48 smithi031 ceph-mon[42085]: mgrmap e15: x(active, since 60s), standbys: z, y 2023-12-14T18:21:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:21:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:48 smithi176 ceph-mon[43506]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:48 smithi176 ceph-mon[43506]: mgrmap e15: x(active, since 60s), standbys: z, y 2023-12-14T18:21:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:21:50.660 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:50 smithi031 ceph-mon[42085]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:50 smithi204 ceph-mon[43866]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:50 smithi176 ceph-mon[43506]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-14T18:21:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-14T18:21:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:51.991 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-14T18:21:51.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:52 smithi204 ceph-mon[43866]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:52 smithi204 ceph-mon[43866]: Deploying daemon osd.0 on smithi031 2023-12-14T18:21:52.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:52 smithi031 ceph-mon[42085]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:52.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:52 smithi031 ceph-mon[42085]: Deploying daemon osd.0 on smithi031 2023-12-14T18:21:52.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:52 smithi176 ceph-mon[43506]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:52.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:52 smithi176 ceph-mon[43506]: Deploying daemon osd.0 on smithi031 2023-12-14T18:21:54.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:54 smithi204 ceph-mon[43866]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:54.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:54 smithi031 ceph-mon[42085]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:54.991 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:54 smithi176 ceph-mon[43506]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:56.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:56 smithi031 ceph-mon[42085]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:56 smithi204 ceph-mon[43866]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:56.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:56.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:56.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:56.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:56 smithi176 ceph-mon[43506]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:57.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:57.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:57.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.753+0000 7fea727fc700 1 -- 172.21.15.31:0/2257065296 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea7c068d00 con 0x7fea680389e0 2023-12-14T18:21:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2023-12-14T18:21:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fea680389e0 msgr2=0x7fea6803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fea680389e0 0x7fea6803ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b5c0 tx=0x7fea78005fb0 comp rx=0 tx=0).stop 2023-12-14T18:21:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c100d70 msgr2=0x7fea7c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c100d70 0x7fea7c19d670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fea74007f40 tx=0x7fea7400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:21:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 shutdown_connections 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fea7c100d70 0x7fea7c19d670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea7c0ff1e0 0x7fea7c1960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fea680389e0 0x7fea6803ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 --2- 172.21.15.31:0/2257065296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea7c1003e0 0x7fea7c1965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 >> 172.21.15.31:0/2257065296 conn(0x7fea7c0fa7f0 msgr2=0x7fea7c0685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 shutdown_connections 2023-12-14T18:21:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:21:57.756+0000 7fea833df700 1 -- 172.21.15.31:0/2257065296 wait complete. 2023-12-14T18:21:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:57 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:57 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:57 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:57 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:57 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:57 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:57 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:57 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:57 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:57 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:58.156 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.0.service 2023-12-14T18:21:58.159 INFO:tasks.cephadm:Deploying osd.1 on smithi176 with /dev/vg_nvme/lv_4... 2023-12-14T18:21:58.159 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-14T18:21:58.195 INFO:journalctl@ceph.osd.0.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:21:58.472 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:58.900 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:21:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:59.667 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:21:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0[54961]: 2023-12-14T18:21:59.319+0000 7f54c4816640 -1 osd.0 0 log_to_monitors true 2023-12-14T18:21:59.731 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-14T18:21:59.731 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-14T18:21:59.731 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2023-12-14T18:21:59.731 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2023-12-14T18:21:59.731 INFO:teuthology.orchestra.run.smithi176.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115093 s, 911 MB/s 2023-12-14T18:21:59.731 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2023-12-14T18:21:59.919 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:21:59 smithi031 ceph-mon[42085]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-14T18:21:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:21:59 smithi176 ceph-mon[43506]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-14T18:22:00.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:21:59 smithi204 ceph-mon[43866]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-14T18:22:00.113 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_4 2023-12-14T18:22:00.417 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:22:00.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:22:00.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-14T18:22:00.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: osdmap e6: 1 total, 0 up, 1 in 2023-12-14T18:22:00.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:00.824 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:00.825 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:22:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-14T18:22:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: osdmap e6: 1 total, 0 up, 1 in 2023-12-14T18:22:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:01.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.214 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0[54961]: 2023-12-14T18:22:00.726+0000 7f54b9e8d700 -1 osd.0 0 waiting for initial osdmap 2023-12-14T18:22:01.214 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0[54961]: 2023-12-14T18:22:00.732+0000 7f54b1c7a700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:22:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-14T18:22:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: osdmap e6: 1 total, 0 up, 1 in 2023-12-14T18:22:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 -- 172.21.15.176:0/33073995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294103690 msgr2=0x7f8294105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 --2- 172.21.15.176:0/33073995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294103690 0x7f8294105a70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8290009990 tx=0x7f8290009ca0 comp rx=0 tx=0).stop 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 -- 172.21.15.176:0/33073995 shutdown_connections 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 --2- 172.21.15.176:0/33073995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8294105fb0 0x7f8294108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 --2- 172.21.15.176:0/33073995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294100d70 0x7f8294103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 --2- 172.21.15.176:0/33073995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294103690 0x7f8294105a70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 -- 172.21.15.176:0/33073995 >> 172.21.15.176:0/33073995 conn(0x7f82940fa7f0 msgr2=0x7f82940fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:01.391 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 -- 172.21.15.176:0/33073995 shutdown_connections 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 -- 172.21.15.176:0/33073995 wait complete. 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 Processor -- start 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.384+0000 7f829b4f0700 1 -- start start 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 0x7f82941006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8294103690 0x7f82940fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294105fb0 0x7f82940ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940ff9e0 con 0x7f8294103690 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940ffb50 con 0x7f8294100d70 2023-12-14T18:22:01.392 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82941a00f0 con 0x7f8294105fb0 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 0x7f82941006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 0x7f82941006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42740/0 (socket says 172.21.15.176:42740) 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 -- 172.21.15.176:0/1820645279 learned_addr learned my addr 172.21.15.176:0/1820645279 (peer_addr_for_me v2:172.21.15.176:0/0) 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f8298a8b700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8294103690 0x7f82940fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f8299a8d700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294105fb0 0x7f82940ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 -- 172.21.15.176:0/1820645279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294105fb0 msgr2=0x7f82940ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294105fb0 0x7f82940ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 -- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8294103690 msgr2=0x7f82940fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8294103690 0x7f82940fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 -- 172.21.15.176:0/1820645279 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900095f0 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f8299a8d700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294105fb0 0x7f82940ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829928c700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 0x7f82941006d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f828400d950 tx=0x7f828400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284009940 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8284010460 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829410bcb0 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400f620 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.385+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829410bb20 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.386+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82940fc360 con 0x7f8294100d70 2023-12-14T18:22:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.387+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f82840108d0 con 0x7f8294100d70 2023-12-14T18:22:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.388+0000 7f828a7fc700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f82800a10c0 0x7f82800a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.388+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f82840cabb0 con 0x7f8294100d70 2023-12-14T18:22:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.388+0000 7f8298a8b700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f82800a10c0 0x7f82800a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.389+0000 7f8298a8b700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f82800a10c0 0x7f82800a3570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f829000b5c0 tx=0x7f8290005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.392+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8284095e90 con 0x7f8294100d70 2023-12-14T18:22:01.590 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:01.589+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8294061aa0 con 0x7f82800a10c0 2023-12-14T18:22:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi031 2023-12-14T18:22:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi031 to 17118M 2023-12-14T18:22:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:22:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: osdmap e7: 1 total, 0 up, 1 in 2023-12-14T18:22:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi031 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi031 to 17118M 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: osdmap e7: 1 total, 0 up, 1 in 2023-12-14T18:22:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi031 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi031 to 17118M 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: from='osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: osdmap e7: 1 total, 0 up, 1 in 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:02.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:02.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:22:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:22:02.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:22:02.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:02.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:02.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] boot 2023-12-14T18:22:02.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: osdmap e8: 1 total, 1 up, 1 in 2023-12-14T18:22:02.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:22:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:22:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:22:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:03.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:03.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] boot 2023-12-14T18:22:03.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: osdmap e8: 1 total, 1 up, 1 in 2023-12-14T18:22:03.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: osd.0 [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] boot 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: osdmap e8: 1 total, 1 up, 1 in 2023-12-14T18:22:03.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:22:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:03 smithi204 ceph-mon[43866]: osdmap e9: 1 total, 1 up, 1 in 2023-12-14T18:22:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:03 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/656202168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]: dispatch 2023-12-14T18:22:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:03 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]: dispatch 2023-12-14T18:22:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:03 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]': finished 2023-12-14T18:22:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:03 smithi204 ceph-mon[43866]: osdmap e10: 2 total, 1 up, 2 in 2023-12-14T18:22:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:03 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:03 smithi031 ceph-mon[42085]: osdmap e9: 1 total, 1 up, 1 in 2023-12-14T18:22:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:03 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/656202168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]: dispatch 2023-12-14T18:22:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:03 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]: dispatch 2023-12-14T18:22:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:03 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]': finished 2023-12-14T18:22:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:03 smithi031 ceph-mon[42085]: osdmap e10: 2 total, 1 up, 2 in 2023-12-14T18:22:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:03 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:03 smithi176 ceph-mon[43506]: osdmap e9: 1 total, 1 up, 1 in 2023-12-14T18:22:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:03 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/656202168' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]: dispatch 2023-12-14T18:22:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:03 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]: dispatch 2023-12-14T18:22:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:03 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79738a41-e749-4359-85db-3cc4494bf2f1"}]': finished 2023-12-14T18:22:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:03 smithi176 ceph-mon[43506]: osdmap e10: 2 total, 1 up, 2 in 2023-12-14T18:22:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:03 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:05.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:04 smithi204 ceph-mon[43866]: pgmap v37: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:05.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:04 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/3222838759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:05.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:04 smithi031 ceph-mon[42085]: pgmap v37: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:05.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:04 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/3222838759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:04 smithi176 ceph-mon[43506]: pgmap v37: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:04 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/3222838759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:06 smithi204 ceph-mon[43866]: pgmap v39: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:07.152 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:06 smithi176 ceph-mon[43506]: pgmap v39: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:07.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:06 smithi031 ceph-mon[42085]: pgmap v39: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:08.969 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:08 smithi176 ceph-mon[43506]: pgmap v40: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:09.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:08 smithi204 ceph-mon[43866]: pgmap v40: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:09.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:08 smithi031 ceph-mon[42085]: pgmap v40: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:10.056 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-14T18:22:10.056 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-14T18:22:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:10.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-14T18:22:10.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:11.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:10 smithi204 ceph-mon[43866]: Deploying daemon osd.1 on smithi176 2023-12-14T18:22:11.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:10 smithi204 ceph-mon[43866]: pgmap v41: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:11.188 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:10 smithi176 ceph-mon[43506]: Deploying daemon osd.1 on smithi176 2023-12-14T18:22:11.188 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:10 smithi176 ceph-mon[43506]: pgmap v41: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:10 smithi031 ceph-mon[42085]: Deploying daemon osd.1 on smithi176 2023-12-14T18:22:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:10 smithi031 ceph-mon[42085]: pgmap v41: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:12 smithi176 ceph-mon[43506]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:13.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:12 smithi204 ceph-mon[43866]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:13.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:12 smithi031 ceph-mon[42085]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:14.232 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:14.232 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:14.232 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:14.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:14.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:14.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:15.150 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.148+0000 7f828a7fc700 1 -- 172.21.15.176:0/1820645279 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8294061aa0 con 0x7f82800a10c0 2023-12-14T18:22:15.150 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 1 on host 'smithi176' 2023-12-14T18:22:15.152 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.150+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f82800a10c0 msgr2=0x7f82800a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:15.152 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f82800a10c0 0x7f82800a3570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f829000b5c0 tx=0x7f8290005fb0 comp rx=0 tx=0).stop 2023-12-14T18:22:15.152 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 msgr2=0x7f82941006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:15.152 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 0x7f82941006d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f828400d950 tx=0x7f828400dc60 comp rx=0 tx=0).stop 2023-12-14T18:22:15.152 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 shutdown_connections 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f82800a10c0 0x7f82800a3570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8294105fb0 0x7f82940ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8294100d70 0x7f82941006d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 --2- 172.21.15.176:0/1820645279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8294103690 0x7f82940fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 >> 172.21.15.176:0/1820645279 conn(0x7f82940fa7f0 msgr2=0x7f82941019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 shutdown_connections 2023-12-14T18:22:15.153 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:15.151+0000 7f829b4f0700 1 -- 172.21.15.176:0/1820645279 wait complete. 2023-12-14T18:22:15.174 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:14 smithi176 ceph-mon[43506]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:15.174 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.175 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.175 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:15.175 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:15.175 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:14 smithi031 ceph-mon[42085]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:15.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:14 smithi204 ceph-mon[43866]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:15.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:15.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:15.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:15.535 DEBUG:teuthology.orchestra.run.smithi176:osd.1> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.1.service 2023-12-14T18:22:15.538 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2023-12-14T18:22:15.538 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-14T18:22:15.568 INFO:journalctl@ceph.osd.1.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:22:15.852 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:22:16.050 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:15 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.050 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:15 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:16.050 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:15 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.050 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:15 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:16.050 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:15 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:16.050 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:15 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:16.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:15 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:15 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:16.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:15 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:15 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:16.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:15 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:16.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:15 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:16.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:15 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:15 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:16.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:15 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:15 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:16.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:15 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:16.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:15 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:17.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:16 smithi204 ceph-mon[43866]: pgmap v44: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:17.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:16 smithi204 ceph-mon[43866]: from='osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:22:17.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:16 smithi204 ceph-mon[43866]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:22:17.137 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-14T18:22:17.137 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-14T18:22:17.137 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2023-12-14T18:22:17.137 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2023-12-14T18:22:17.137 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111863 s, 937 MB/s 2023-12-14T18:22:17.137 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2023-12-14T18:22:17.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:16 smithi031 ceph-mon[42085]: pgmap v44: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:17.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:16 smithi031 ceph-mon[42085]: from='osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:22:17.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:16 smithi031 ceph-mon[42085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:22:17.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:16 smithi176 ceph-mon[43506]: pgmap v44: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:17.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:16 smithi176 ceph-mon[43506]: from='osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:22:17.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:16 smithi176 ceph-mon[43506]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:22:17.242 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:16 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1[48490]: 2023-12-14T18:22:16.848+0000 7f4c296e6640 -1 osd.1 0 log_to_monitors true 2023-12-14T18:22:17.502 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2023-12-14T18:22:17.805 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:22:18.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi176 2023-12-14T18:22:18.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-14T18:22:18.286 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: osdmap e11: 2 total, 1 up, 2 in 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi176 to 17114M 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:18.287 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:18.288 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.288 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:18.288 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:18.288 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:18 smithi204 ceph-mon[43866]: pgmap v46: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi176 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: osdmap e11: 2 total, 1 up, 2 in 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi176 to 17114M 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:18.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:18 smithi031 ceph-mon[42085]: pgmap v46: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:18.492 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1[48490]: 2023-12-14T18:22:18.119+0000 7f4c1ed5d700 -1 osd.1 0 waiting for initial osdmap 2023-12-14T18:22:18.492 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1[48490]: 2023-12-14T18:22:18.123+0000 7f4c18b4e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:18.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi176 2023-12-14T18:22:18.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: osdmap e11: 2 total, 1 up, 2 in 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi176 to 17114M 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:18.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:18.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:18.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:18.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:18.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:18.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:18 smithi176 ceph-mon[43506]: pgmap v46: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:18.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 -- 172.21.15.204:0/2303326174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 msgr2=0x7f1334100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:18.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 --2- 172.21.15.204:0/2303326174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f1334100830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1324009990 tx=0x7f1324009ca0 comp rx=0 tx=0).stop 2023-12-14T18:22:18.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 -- 172.21.15.204:0/2303326174 shutdown_connections 2023-12-14T18:22:18.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 --2- 172.21.15.204:0/2303326174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1334100d70 0x7f1334107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:18.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 --2- 172.21.15.204:0/2303326174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13340ff1e0 0x7f13340ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:18.786 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 --2- 172.21.15.204:0/2303326174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f1334100830 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 -- 172.21.15.204:0/2303326174 >> 172.21.15.204:0/2303326174 conn(0x7f13340fa7f0 msgr2=0x7f13340fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 -- 172.21.15.204:0/2303326174 shutdown_connections 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 -- 172.21.15.204:0/2303326174 wait complete. 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 Processor -- start 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.782+0000 7f133c005700 1 -- start start 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13340ff1e0 0x7f1334069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f133419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1334100d70 0x7f133419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419dbf0 con 0x7f1334100d70 2023-12-14T18:22:18.787 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419dd60 con 0x7f13340ff1e0 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133419df20 con 0x7f13341003e0 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f133419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f133419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:50504/0 (socket says 172.21.15.204:50504) 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 -- 172.21.15.204:0/3819345754 learned_addr learned my addr 172.21.15.204:0/3819345754 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f1339da1700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13340ff1e0 0x7f1334069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 -- 172.21.15.204:0/3819345754 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13340ff1e0 msgr2=0x7f1334069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13340ff1e0 0x7f1334069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 -- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1334100d70 msgr2=0x7f133419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133a5a2700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1334100d70 0x7f133419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1334100d70 0x7f133419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 -- 172.21.15.204:0/3819345754 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240095f0 con 0x7f13341003e0 2023-12-14T18:22:18.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f13395a0700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f133419ae10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f13240095c0 tx=0x7f132400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:18.789 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132401d070 con 0x7f13341003e0 2023-12-14T18:22:18.789 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133410ab80 con 0x7f13341003e0 2023-12-14T18:22:18.789 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1324022470 con 0x7f13341003e0 2023-12-14T18:22:18.789 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324017650 con 0x7f13341003e0 2023-12-14T18:22:18.789 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.783+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133410a9f0 con 0x7f13341003e0 2023-12-14T18:22:18.789 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.784+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133419e4c0 con 0x7f13341003e0 2023-12-14T18:22:18.790 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.785+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f1324022890 con 0x7f13341003e0 2023-12-14T18:22:18.790 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.786+0000 7f132affd700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13200a10c0 0x7f13200a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:18.790 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.786+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f13240cc710 con 0x7f13341003e0 2023-12-14T18:22:18.790 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.786+0000 7f1339da1700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13200a10c0 0x7f13200a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:18.790 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.787+0000 7f1339da1700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13200a10c0 0x7f13200a3570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1334100240 tx=0x7f133000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:18.791 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.790+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1324097860 con 0x7f13341003e0 2023-12-14T18:22:18.985 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:18.984+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1334060f30 con 0x7f13200a10c0 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: osdmap e12: 2 total, 1 up, 2 in 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:19.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: osdmap e12: 2 total, 1 up, 2 in 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: osdmap e12: 2 total, 1 up, 2 in 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:19.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:19.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] boot 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: osdmap e13: 2 total, 2 up, 2 in 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:20 smithi031 ceph-mon[42085]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] boot 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: osdmap e13: 2 total, 2 up, 2 in 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:20.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:20 smithi176 ceph-mon[43506]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:20.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:22:20.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:22:20.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:20.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: osd.1 [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] boot 2023-12-14T18:22:20.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: osdmap e13: 2 total, 2 up, 2 in 2023-12-14T18:22:20.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:22:20.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:20 smithi204 ceph-mon[43866]: pgmap v49: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:22:21.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:21 smithi031 ceph-mon[42085]: osdmap e14: 2 total, 2 up, 2 in 2023-12-14T18:22:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:21 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/2715106911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]: dispatch 2023-12-14T18:22:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:21 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]: dispatch 2023-12-14T18:22:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:21 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]': finished 2023-12-14T18:22:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:21 smithi031 ceph-mon[42085]: osdmap e15: 3 total, 2 up, 3 in 2023-12-14T18:22:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:21.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:21 smithi176 ceph-mon[43506]: osdmap e14: 2 total, 2 up, 2 in 2023-12-14T18:22:21.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:21 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/2715106911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]: dispatch 2023-12-14T18:22:21.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:21 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]: dispatch 2023-12-14T18:22:21.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:21 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]': finished 2023-12-14T18:22:21.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:21 smithi176 ceph-mon[43506]: osdmap e15: 3 total, 2 up, 3 in 2023-12-14T18:22:21.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:21.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:21 smithi204 ceph-mon[43866]: osdmap e14: 2 total, 2 up, 2 in 2023-12-14T18:22:21.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:21 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/2715106911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]: dispatch 2023-12-14T18:22:21.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:21 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]: dispatch 2023-12-14T18:22:21.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:21 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5119d0ba-0044-4206-b4e1-898df98d5f0c"}]': finished 2023-12-14T18:22:21.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:21 smithi204 ceph-mon[43866]: osdmap e15: 3 total, 2 up, 3 in 2023-12-14T18:22:21.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:22.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:22 smithi031 ceph-mon[42085]: pgmap v52: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:22.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:22 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/1541005836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:22.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:22 smithi176 ceph-mon[43506]: pgmap v52: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:22.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:22 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/1541005836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:22.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:22 smithi204 ceph-mon[43866]: pgmap v52: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:22.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:22 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/1541005836' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:24.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:24 smithi204 ceph-mon[43866]: pgmap v53: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:24.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:24 smithi031 ceph-mon[42085]: pgmap v53: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:24.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:24 smithi176 ceph-mon[43506]: pgmap v53: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:26.615 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:26 smithi204 ceph-mon[43866]: pgmap v54: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:26.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:26 smithi031 ceph-mon[42085]: pgmap v54: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:26.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:26 smithi176 ceph-mon[43506]: pgmap v54: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-14T18:22:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:27 smithi204 ceph-mon[43866]: Deploying daemon osd.2 on smithi204 2023-12-14T18:22:27.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-14T18:22:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:27 smithi031 ceph-mon[42085]: Deploying daemon osd.2 on smithi204 2023-12-14T18:22:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-14T18:22:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:27 smithi176 ceph-mon[43506]: Deploying daemon osd.2 on smithi204 2023-12-14T18:22:28.714 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:28 smithi204 ceph-mon[43866]: pgmap v55: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:28.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:28 smithi031 ceph-mon[42085]: pgmap v55: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:28 smithi176 ceph-mon[43506]: pgmap v55: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:30 smithi204 ceph-mon[43866]: pgmap v56: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:30.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:30 smithi031 ceph-mon[42085]: pgmap v56: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:30 smithi176 ceph-mon[43506]: pgmap v56: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:32.693 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.691+0000 7f132affd700 1 -- 172.21.15.204:0/3819345754 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1334060f30 con 0x7f13200a10c0 2023-12-14T18:22:32.694 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.693+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13200a10c0 msgr2=0x7f13200a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.693+0000 7f133c005700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13200a10c0 0x7f13200a3570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1334100240 tx=0x7f133000b410 comp rx=0 tx=0).stop 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.693+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 msgr2=0x7f133419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.693+0000 7f133c005700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f133419ae10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f13240095c0 tx=0x7f132400f690 comp rx=0 tx=0).stop 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 shutdown_connections 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1334100d70 0x7f133419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13340ff1e0 0x7f1334069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13200a10c0 0x7f13200a3570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:32.696 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 --2- 172.21.15.204:0/3819345754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13341003e0 0x7f133419ae10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:32.697 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 >> 172.21.15.204:0/3819345754 conn(0x7f13340fa7f0 msgr2=0x7f13340fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:32.697 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 shutdown_connections 2023-12-14T18:22:32.697 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:22:32.694+0000 7f133c005700 1 -- 172.21.15.204:0/3819345754 wait complete. 2023-12-14T18:22:32.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: pgmap v57: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:32.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:32.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:32.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:32.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:32.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:32.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:32.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:32.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:32.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:32.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:32.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.118 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.2.service 2023-12-14T18:22:33.120 INFO:tasks.cephadm:Deploying osd.3 on smithi031 with /dev/vg_nvme/lv_3... 2023-12-14T18:22:33.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-14T18:22:33.151 INFO:journalctl@ceph.osd.2.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:22:33.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: pgmap v57: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:33.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:33.156 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:33.157 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: pgmap v57: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:33.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:33.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:33.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:33.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:33.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:33.477 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:22:34.589 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2[49631]: 2023-12-14T18:22:34.268+0000 7f8deb110640 -1 osd.2 0 log_to_monitors true 2023-12-14T18:22:34.841 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-14T18:22:34.842 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-14T18:22:34.842 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-12-14T18:22:34.842 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-12-14T18:22:34.842 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.016971 s, 618 MB/s 2023-12-14T18:22:34.842 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-12-14T18:22:35.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:34 smithi031 ceph-mon[42085]: pgmap v58: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:35.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:34 smithi031 ceph-mon[42085]: from='osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:22:35.042 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:34 smithi031 ceph-mon[42085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:22:35.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:34 smithi204 ceph-mon[43866]: pgmap v58: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:35.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:34 smithi204 ceph-mon[43866]: from='osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:22:35.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:34 smithi204 ceph-mon[43866]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:22:35.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:34 smithi176 ceph-mon[43506]: pgmap v58: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:35.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:34 smithi176 ceph-mon[43506]: from='osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:22:35.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:34 smithi176 ceph-mon[43506]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:22:35.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2023-12-14T18:22:35.648 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:22:36.089 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2[49631]: 2023-12-14T18:22:35.771+0000 7f8de0787700 -1 osd.2 0 waiting for initial osdmap 2023-12-14T18:22:36.089 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2[49631]: 2023-12-14T18:22:35.776+0000 7f8dd8574700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:36.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: osdmap e16: 3 total, 2 up, 3 in 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi204 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi204 to 17113M 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:36.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:36.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:35 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:36.101 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-14T18:22:36.101 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: osdmap e16: 3 total, 2 up, 3 in 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi204 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi204 to 17113M 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:36.102 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:35 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-14T18:22:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: osdmap e16: 3 total, 2 up, 3 in 2023-12-14T18:22:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:22:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi204 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi204 to 17113M 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:36.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:36.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:36.244 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:35 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 -- 172.21.15.31:0/407753579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c100320 msgr2=0x7f398c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 --2- 172.21.15.31:0/407753579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c100320 0x7f398c100730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3980009990 tx=0x7f3980009ca0 comp rx=0 tx=0).stop 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 -- 172.21.15.31:0/407753579 shutdown_connections 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 --2- 172.21.15.31:0/407753579 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c101eb0 0x7f398c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 --2- 172.21.15.31:0/407753579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c100320 0x7f398c100730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 --2- 172.21.15.31:0/407753579 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f398c101520 0x7f398c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:36.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 -- 172.21.15.31:0/407753579 >> 172.21.15.31:0/407753579 conn(0x7f398c0fb970 msgr2=0x7f398c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 -- 172.21.15.31:0/407753579 shutdown_connections 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.617+0000 7f3992fad700 1 -- 172.21.15.31:0/407753579 wait complete. 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 Processor -- start 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 -- start start 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c100320 0x7f398c196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f398c101520 0x7f398c196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 0x7f398c19d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c19dbc0 con 0x7f398c101eb0 2023-12-14T18:22:36.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c19dd00 con 0x7f398c101520 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3992fad700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c19de90 con 0x7f398c100320 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 0x7f398c19d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 0x7f398c19d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36770/0 (socket says 172.21.15.31:36770) 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 -- 172.21.15.31:0/2903119700 learned_addr learned my addr 172.21.15.31:0/2903119700 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f398bfff700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f398c101520 0x7f398c196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f3990d49700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c100320 0x7f398c196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 -- 172.21.15.31:0/2903119700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c100320 msgr2=0x7f398c196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c100320 0x7f398c196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 -- 172.21.15.31:0/2903119700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f398c101520 msgr2=0x7f398c196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:36.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f398c101520 0x7f398c196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.618+0000 7f399154a700 1 -- 172.21.15.31:0/2903119700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39800095f0 con 0x7f398c101eb0 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.619+0000 7f399154a700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 0x7f398c19d5f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f398400ef60 tx=0x7f398400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.619+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984018070 con 0x7f398c101eb0 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.619+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f398400f040 con 0x7f398c101eb0 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.619+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c10bc90 con 0x7f398c101eb0 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.619+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400ccc0 con 0x7f398c101eb0 2023-12-14T18:22:36.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.619+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c10bb00 con 0x7f398c101eb0 2023-12-14T18:22:36.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.620+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c19e4c0 con 0x7f398c101eb0 2023-12-14T18:22:36.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.621+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f3984003a00 con 0x7f398c101eb0 2023-12-14T18:22:36.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.622+0000 7f3989ffb700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f397c0a1140 0x7f397c0a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:36.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.622+0000 7f3990d49700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f397c0a1140 0x7f397c0a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:36.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.622+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f39840cacf0 con 0x7f398c101eb0 2023-12-14T18:22:36.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.622+0000 7f3990d49700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f397c0a1140 0x7f397c0a35f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f398000b2e0 tx=0x7f3980005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:36.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.626+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3984095b80 con 0x7f398c101eb0 2023-12-14T18:22:36.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:36.848+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f398c19eb30 con 0x7f397c0a1140 2023-12-14T18:22:37.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:36 smithi204 ceph-mon[43866]: pgmap v60: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:37.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:36 smithi204 ceph-mon[43866]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:22:37.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:36 smithi204 ceph-mon[43866]: osdmap e17: 3 total, 2 up, 3 in 2023-12-14T18:22:37.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:37.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:37.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:36 smithi031 ceph-mon[42085]: pgmap v60: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:37.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:36 smithi031 ceph-mon[42085]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:22:37.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:36 smithi031 ceph-mon[42085]: osdmap e17: 3 total, 2 up, 3 in 2023-12-14T18:22:37.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:37.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:36 smithi176 ceph-mon[43506]: pgmap v60: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:36 smithi176 ceph-mon[43506]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:22:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:36 smithi176 ceph-mon[43506]: osdmap e17: 3 total, 2 up, 3 in 2023-12-14T18:22:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:22:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:22:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] boot 2023-12-14T18:22:38.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: osdmap e18: 3 total, 3 up, 3 in 2023-12-14T18:22:38.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:38.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:38.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:38.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:22:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:22:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] boot 2023-12-14T18:22:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: osdmap e18: 3 total, 3 up, 3 in 2023-12-14T18:22:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:38.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:38.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:38.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:22:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:22:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: osd.2 [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] boot 2023-12-14T18:22:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: osdmap e18: 3 total, 3 up, 3 in 2023-12-14T18:22:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:22:38.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:38.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:38.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:38.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: pgmap v63: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:22:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-14T18:22:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: osdmap e19: 3 total, 3 up, 3 in 2023-12-14T18:22:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:22:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3803263617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb73492-97c9-4615-9eed-75737af46d2e"}]: dispatch 2023-12-14T18:22:39.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:22:39.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3803263617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfb73492-97c9-4615-9eed-75737af46d2e"}]': finished 2023-12-14T18:22:39.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: osdmap e20: 4 total, 3 up, 4 in 2023-12-14T18:22:39.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: pgmap v63: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:22:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-14T18:22:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: osdmap e19: 3 total, 3 up, 3 in 2023-12-14T18:22:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:22:39.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3803263617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb73492-97c9-4615-9eed-75737af46d2e"}]: dispatch 2023-12-14T18:22:39.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:22:39.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3803263617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfb73492-97c9-4615-9eed-75737af46d2e"}]': finished 2023-12-14T18:22:39.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: osdmap e20: 4 total, 3 up, 4 in 2023-12-14T18:22:39.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:39.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: pgmap v63: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:22:39.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:22:39.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-14T18:22:39.461 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: osdmap e19: 3 total, 3 up, 3 in 2023-12-14T18:22:39.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:22:39.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3803263617' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bfb73492-97c9-4615-9eed-75737af46d2e"}]: dispatch 2023-12-14T18:22:39.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:22:39.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3803263617' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bfb73492-97c9-4615-9eed-75737af46d2e"}]': finished 2023-12-14T18:22:39.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: osdmap e20: 4 total, 3 up, 4 in 2023-12-14T18:22:39.462 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:39.713 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58223]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:22:39.714 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:39.714 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58223]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:39.714 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58223]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:39 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3891986823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:39 smithi031 ceph-mon[42085]: osdmap e21: 4 total, 3 up, 4 in 2023-12-14T18:22:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:39 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:39 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:40.215 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58272]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:22:40.215 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58272]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:40.215 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58272]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:40.215 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:22:39 smithi031 sudo[58272]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:39 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3891986823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:39 smithi176 ceph-mon[43506]: osdmap e21: 4 total, 3 up, 4 in 2023-12-14T18:22:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:39 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:39 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:40.242 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50871]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:22:40.242 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50871]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:40.242 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50871]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:40.242 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50871]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:39 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3891986823' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:39 smithi204 ceph-mon[43866]: osdmap e21: 4 total, 3 up, 4 in 2023-12-14T18:22:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:39 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:39 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:40.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50874]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:22:40.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:40.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50874]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:40.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:22:40 smithi176 sudo[50874]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:40.984+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3984003cb0 con 0x7f398c101eb0 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51863]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51863]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51863]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51863]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51866]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51866]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51866]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:40.990 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:22:40 smithi204 sudo[51866]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:41.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:40 smithi176 ceph-mon[43506]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:41.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:41.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:40 smithi204 ceph-mon[43866]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:41.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:41.453 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:40 smithi031 ceph-mon[42085]: pgmap v66: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:41.453 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:41.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59018]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:22:41.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:41.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59018]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:41.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59018]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:42.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59205]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:22:42.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59205]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:42.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59205]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:42.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:41 smithi031 sudo[59205]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:42.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51869]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:22:42.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51869]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:42.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51869]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:42.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51869]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:42.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51872]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51872]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51872]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 sudo[51872]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: mgrmap e16: x(active, since 113s), standbys: z, y 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:42.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: mgrmap e16: x(active, since 113s), standbys: z, y 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:42.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: mgrmap e16: x(active, since 113s), standbys: z, y 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:42.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:42.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:42.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50877]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:22:42.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:42.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50877]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:42.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50877]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50880]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:22:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50880]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:22:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50880]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:22:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:42 smithi176 sudo[50880]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:43.679 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:43 smithi031 ceph-mon[42085]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:43 smithi176 ceph-mon[43506]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:22:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:43 smithi204 ceph-mon[43866]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:22:44.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:44 smithi031 ceph-mon[42085]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:44.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:44 smithi176 ceph-mon[43506]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:44.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:44 smithi204 ceph-mon[43866]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:45.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:45 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-14T18:22:45.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:45 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:45.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:45 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-14T18:22:45.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:45 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:45 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-14T18:22:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:45 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:46.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:46 smithi031 ceph-mon[42085]: Deploying daemon osd.3 on smithi031 2023-12-14T18:22:46.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:46 smithi031 ceph-mon[42085]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-12-14T18:22:46.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:46 smithi176 ceph-mon[43506]: Deploying daemon osd.3 on smithi031 2023-12-14T18:22:46.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:46 smithi176 ceph-mon[43506]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-12-14T18:22:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:46 smithi204 ceph-mon[43866]: Deploying daemon osd.3 on smithi031 2023-12-14T18:22:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:46 smithi204 ceph-mon[43866]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 7.5 KiB/s rd, 326 KiB/s wr, 22 op/s 2023-12-14T18:22:49.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:48 smithi176 ceph-mon[43506]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-12-14T18:22:49.305 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:48 smithi031 ceph-mon[42085]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-12-14T18:22:49.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:48 smithi204 ceph-mon[43866]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 5.0 KiB/s rd, 218 KiB/s wr, 15 op/s 2023-12-14T18:22:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:50 smithi031 ceph-mon[42085]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-12-14T18:22:50.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:50.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:50.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:50.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:50 smithi176 ceph-mon[43506]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-12-14T18:22:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:50 smithi204 ceph-mon[43866]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 163 KiB/s wr, 11 op/s 2023-12-14T18:22:52.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.043+0000 7f3989ffb700 1 -- 172.21.15.31:0/2903119700 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f398c19eb30 con 0x7f397c0a1140 2023-12-14T18:22:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 3 on host 'smithi031' 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f397c0a1140 msgr2=0x7f397c0a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f397c0a1140 0x7f397c0a35f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f398000b2e0 tx=0x7f3980005fb0 comp rx=0 tx=0).stop 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 msgr2=0x7f398c19d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 0x7f398c19d5f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f398400ef60 tx=0x7f398400ef90 comp rx=0 tx=0).stop 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 shutdown_connections 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f398c101eb0 0x7f398c19d5f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:52.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f397c0a1140 0x7f397c0a35f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:52.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f398c100320 0x7f398c196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:52.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 --2- 172.21.15.31:0/2903119700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f398c101520 0x7f398c196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:52.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 >> 172.21.15.31:0/2903119700 conn(0x7f398c0fb970 msgr2=0x7f398c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:52.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 shutdown_connections 2023-12-14T18:22:52.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:22:52.046+0000 7f3992fad700 1 -- 172.21.15.31:0/2903119700 wait complete. 2023-12-14T18:22:52.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:52 smithi031 ceph-mon[42085]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:52 smithi204 ceph-mon[43866]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-12-14T18:22:52.431 DEBUG:teuthology.orchestra.run.smithi031:osd.3> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.3.service 2023-12-14T18:22:52.434 INFO:tasks.cephadm:Deploying osd.4 on smithi176 with /dev/vg_nvme/lv_3... 2023-12-14T18:22:52.434 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:52.467 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:52 smithi176 ceph-mon[43506]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2023-12-14T18:22:52.473 INFO:journalctl@ceph.osd.3.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:22:52.752 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:22:53.165 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:53.166 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:53.166 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:53.214 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:22:53 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:22:53.185+0000 7f08b3f33640 -1 osd.3 0 log_to_monitors true 2023-12-14T18:22:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:53.986 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-14T18:22:53.986 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-14T18:22:53.986 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2023-12-14T18:22:53.986 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2023-12-14T18:22:53.986 INFO:teuthology.orchestra.run.smithi176.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112195 s, 935 MB/s 2023-12-14T18:22:53.986 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2023-12-14T18:22:54.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:54 smithi031 ceph-mon[42085]: from='osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:22:54.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:54 smithi031 ceph-mon[42085]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:22:54.082 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:54 smithi031 ceph-mon[42085]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-12-14T18:22:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:54 smithi204 ceph-mon[43866]: from='osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:22:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:54 smithi204 ceph-mon[43866]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:22:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:54 smithi204 ceph-mon[43866]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-12-14T18:22:54.368 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:54 smithi176 ceph-mon[43506]: from='osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:22:54.368 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:54 smithi176 ceph-mon[43506]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:22:54.368 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:54 smithi176 ceph-mon[43506]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-12-14T18:22:54.370 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_3 2023-12-14T18:22:54.716 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:22:55.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-14T18:22:55.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: osdmap e22: 4 total, 3 up, 4 in 2023-12-14T18:22:55.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:55.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:55.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:55.157 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi031 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi031 to 8559M 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.158 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:22:55.159 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:55 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: osdmap e22: 4 total, 3 up, 4 in 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi031 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi031 to 8559M 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:22:55.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:55 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:22:55.464 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:22:55.052+0000 7f08a95aa700 -1 osd.3 0 waiting for initial osdmap 2023-12-14T18:22:55.464 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:22:55.059+0000 7f08a539f700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:55.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-14T18:22:55.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: osdmap e22: 4 total, 3 up, 4 in 2023-12-14T18:22:55.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:55.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:55.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi031 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi031 to 8559M 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:55.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:22:55.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:22:55.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:55 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 -- 172.21.15.176:0/1713571163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be0100620 msgr2=0x7f3be0100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 --2- 172.21.15.176:0/1713571163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be0100620 0x7f3be0100a70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009990 tx=0x7f3bdc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 -- 172.21.15.176:0/1713571163 shutdown_connections 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 --2- 172.21.15.176:0/1713571163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3be0100fb0 0x7f3be0107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 --2- 172.21.15.176:0/1713571163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be00ff420 0x7f3be00ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 --2- 172.21.15.176:0/1713571163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be0100620 0x7f3be0100a70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:55.714 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 -- 172.21.15.176:0/1713571163 >> 172.21.15.176:0/1713571163 conn(0x7f3be00faa50 msgr2=0x7f3be00fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 -- 172.21.15.176:0/1713571163 shutdown_connections 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.709+0000 7f3be8763700 1 -- 172.21.15.176:0/1713571163 wait complete. 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 Processor -- start 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 -- start start 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 0x7f3be0069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3be0100620 0x7f3be019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be0100fb0 0x7f3be019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019dda0 con 0x7f3be0100fb0 2023-12-14T18:22:55.715 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019df10 con 0x7f3be00ff420 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be8763700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019e0d0 con 0x7f3be0100620 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 0x7f3be0069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 0x7f3be0069be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:52144/0 (socket says 172.21.15.176:52144) 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 -- 172.21.15.176:0/2505425326 learned_addr learned my addr 172.21.15.176:0/2505425326 (peer_addr_for_me v2:172.21.15.176:0/0) 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be6d00700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be0100fb0 0x7f3be019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be5cfe700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3be0100620 0x7f3be019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 -- 172.21.15.176:0/2505425326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3be0100620 msgr2=0x7f3be019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3be0100620 0x7f3be019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 -- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be0100fb0 msgr2=0x7f3be019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.710+0000 7f3be64ff700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be0100fb0 0x7f3be019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:55.716 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3be64ff700 1 -- 172.21.15.176:0/2505425326 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0095f0 con 0x7f3be00ff420 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3be64ff700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 0x7f3be0069be0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d980 tx=0x7f3bd400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4004020 con 0x7f3be00ff420 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bd4009730 con 0x7f3be00ff420 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be010add0 con 0x7f3be00ff420 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4003a80 con 0x7f3be00ff420 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.711+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be010ac40 con 0x7f3be00ff420 2023-12-14T18:22:55.717 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.712+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be0106a10 con 0x7f3be00ff420 2023-12-14T18:22:55.718 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.714+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3bd40098a0 con 0x7f3be00ff420 2023-12-14T18:22:55.718 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.714+0000 7f3bd37fe700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3bcc0a11d0 0x7f3bcc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:55.718 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.714+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7f3bd40cb590 con 0x7f3be00ff420 2023-12-14T18:22:55.719 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.714+0000 7f3be5cfe700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3bcc0a11d0 0x7f3bcc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:55.719 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.715+0000 7f3be5cfe700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3bcc0a11d0 0x7f3bcc0a3680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b2e0 tx=0x7f3bdc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:55.719 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.718+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3bd40960a0 con 0x7f3be00ff420 2023-12-14T18:22:55.916 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:22:55.914+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3be0060f30 con 0x7f3bcc0a11d0 2023-12-14T18:22:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:22:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: osdmap e23: 4 total, 3 up, 4 in 2023-12-14T18:22:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:56.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: pgmap v77: 1 pgs: 1 peering; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:56.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:56.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:56.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:56.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: osdmap e23: 4 total, 3 up, 4 in 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: pgmap v77: 1 pgs: 1 peering; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:56.354 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:56.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: osdmap e23: 4 total, 3 up, 4 in 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: pgmap v77: 1 pgs: 1 peering; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:57 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:22:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:57 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:22:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:57 smithi176 ceph-mon[43506]: osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] boot 2023-12-14T18:22:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:57 smithi176 ceph-mon[43506]: osdmap e24: 4 total, 4 up, 4 in 2023-12-14T18:22:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:57 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:57 smithi176 ceph-mon[43506]: osdmap e25: 4 total, 4 up, 4 in 2023-12-14T18:22:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:57 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:22:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:57 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:22:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:57 smithi204 ceph-mon[43866]: osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] boot 2023-12-14T18:22:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:57 smithi204 ceph-mon[43866]: osdmap e24: 4 total, 4 up, 4 in 2023-12-14T18:22:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:57 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:57 smithi204 ceph-mon[43866]: osdmap e25: 4 total, 4 up, 4 in 2023-12-14T18:22:57.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:57 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:22:57.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:57 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:22:57.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:57 smithi031 ceph-mon[42085]: osd.3 [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] boot 2023-12-14T18:22:57.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:57 smithi031 ceph-mon[42085]: osdmap e24: 4 total, 4 up, 4 in 2023-12-14T18:22:57.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:57 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:22:57.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:57 smithi031 ceph-mon[42085]: osdmap e25: 4 total, 4 up, 4 in 2023-12-14T18:22:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:58 smithi204 ceph-mon[43866]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:22:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:58 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/2703832497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]: dispatch 2023-12-14T18:22:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:58 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]: dispatch 2023-12-14T18:22:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:58 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]': finished 2023-12-14T18:22:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:58 smithi204 ceph-mon[43866]: osdmap e26: 5 total, 4 up, 5 in 2023-12-14T18:22:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:58.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:58 smithi031 ceph-mon[42085]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:22:58.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:58 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/2703832497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]: dispatch 2023-12-14T18:22:58.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:58 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]: dispatch 2023-12-14T18:22:58.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:58 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]': finished 2023-12-14T18:22:58.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:58 smithi031 ceph-mon[42085]: osdmap e26: 5 total, 4 up, 5 in 2023-12-14T18:22:58.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:58 smithi176 ceph-mon[43506]: pgmap v80: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:22:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:58 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/2703832497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]: dispatch 2023-12-14T18:22:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:58 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]: dispatch 2023-12-14T18:22:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:58 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1c7deda-98af-495d-9680-d1b95ceb100b"}]': finished 2023-12-14T18:22:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:58 smithi176 ceph-mon[43506]: osdmap e26: 5 total, 4 up, 5 in 2023-12-14T18:22:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:22:59 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/4018087650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:59.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:22:59 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/4018087650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:59.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:22:59 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/4018087650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:00.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:00 smithi204 ceph-mon[43866]: osdmap e27: 5 total, 4 up, 5 in 2023-12-14T18:23:00.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:00.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:00 smithi204 ceph-mon[43866]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:23:00.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:00 smithi031 ceph-mon[42085]: osdmap e27: 5 total, 4 up, 5 in 2023-12-14T18:23:00.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:00.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:00 smithi031 ceph-mon[42085]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:23:00.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:00 smithi176 ceph-mon[43506]: osdmap e27: 5 total, 4 up, 5 in 2023-12-14T18:23:00.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:00.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:00 smithi176 ceph-mon[43506]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:23:01.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:01 smithi204 ceph-mon[43866]: osdmap e28: 5 total, 4 up, 5 in 2023-12-14T18:23:01.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:01.410 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:01 smithi176 ceph-mon[43506]: osdmap e28: 5 total, 4 up, 5 in 2023-12-14T18:23:01.410 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:01.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:01 smithi031 ceph-mon[42085]: osdmap e28: 5 total, 4 up, 5 in 2023-12-14T18:23:01.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:02.447 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:02 smithi176 ceph-mon[43506]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-12-14T18:23:02.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:02 smithi031 ceph-mon[42085]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-12-14T18:23:02.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:02 smithi204 ceph-mon[43866]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-12-14T18:23:04.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:03 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-14T18:23:04.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:03 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:03 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-14T18:23:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:03 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:04.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:03 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-14T18:23:04.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:03 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:04 smithi176 ceph-mon[43506]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-14T18:23:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:04 smithi176 ceph-mon[43506]: Deploying daemon osd.4 on smithi176 2023-12-14T18:23:05.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:04 smithi204 ceph-mon[43866]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-14T18:23:05.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:04 smithi204 ceph-mon[43866]: Deploying daemon osd.4 on smithi176 2023-12-14T18:23:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:04 smithi031 ceph-mon[42085]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-14T18:23:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:04 smithi031 ceph-mon[42085]: Deploying daemon osd.4 on smithi176 2023-12-14T18:23:06.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:05 smithi176 ceph-mon[43506]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-12-14T18:23:06.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:05 smithi204 ceph-mon[43866]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-12-14T18:23:06.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:05 smithi031 ceph-mon[42085]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-12-14T18:23:08.166 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:08.166 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:08.166 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:08.167 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:08 smithi176 ceph-mon[43506]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-14T18:23:08.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:08.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:08.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:08.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:08 smithi204 ceph-mon[43866]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-14T18:23:08.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:08.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:08.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:08.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:08 smithi031 ceph-mon[42085]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-14T18:23:09.377 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.376+0000 7f3bd37fe700 1 -- 172.21.15.176:0/2505425326 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3be0060f30 con 0x7f3bcc0a11d0 2023-12-14T18:23:09.377 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 4 on host 'smithi176' 2023-12-14T18:23:09.379 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.378+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3bcc0a11d0 msgr2=0x7f3bcc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.378+0000 7f3be8763700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3bcc0a11d0 0x7f3bcc0a3680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00b2e0 tx=0x7f3bdc005fb0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.378+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 msgr2=0x7f3be0069be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.378+0000 7f3be8763700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 0x7f3be0069be0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400d980 tx=0x7f3bd400dc90 comp rx=0 tx=0).stop 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 shutdown_connections 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3be0100fb0 0x7f3be019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3be00ff420 0x7f3be0069be0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3bcc0a11d0 0x7f3bcc0a3680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 --2- 172.21.15.176:0/2505425326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3be0100620 0x7f3be019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.380 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 >> 172.21.15.176:0/2505425326 conn(0x7f3be00faa50 msgr2=0x7f3be0103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:09.381 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 shutdown_connections 2023-12-14T18:23:09.381 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:09.379+0000 7f3be8763700 1 -- 172.21.15.176:0/2505425326 wait complete. 2023-12-14T18:23:09.688 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.688 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.688 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.689 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:09 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:09.797 DEBUG:teuthology.orchestra.run.smithi176:osd.4> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.4.service 2023-12-14T18:23:09.800 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2023-12-14T18:23:09.800 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-14T18:23:09.833 INFO:journalctl@ceph.osd.4.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:23:09.834 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.835 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:09 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:09.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:09 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:10.120 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:23:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:10 smithi204 ceph-mon[43866]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:23:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:10 smithi031 ceph-mon[42085]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:23:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:10 smithi176 ceph-mon[43506]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:23:11.348 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-14T18:23:11.348 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-14T18:23:11.349 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2023-12-14T18:23:11.349 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2023-12-14T18:23:11.349 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112013 s, 936 MB/s 2023-12-14T18:23:11.349 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2023-12-14T18:23:11.382 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4[53974]: 2023-12-14T18:23:11.067+0000 7f02f40e5640 -1 osd.4 0 log_to_monitors true 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:11.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:11.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:11.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:11 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.680 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:11.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:11.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:11 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:23:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:23:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:11.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:11.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:11.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:11.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:11 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:12.025 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:23:12.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi176 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi176 to 8557M 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: osdmap e29: 5 total, 4 up, 5 in 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: from='osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:23:12.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:12 smithi204 ceph-mon[43866]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:23:12.961 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/3406186629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 msgr2=0x7fb5d4108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:12.961 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/3406186629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d4108740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00b0a0 tx=0x7fb5cc00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/3406186629 shutdown_connections 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/3406186629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d4108740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/3406186629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5d41002f0 0x7fb5d4100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/3406186629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5d41014f0 0x7fb5d4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/3406186629 >> 172.21.15.204:0/3406186629 conn(0x7fb5d40fb900 msgr2=0x7fb5d40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/3406186629 shutdown_connections 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/3406186629 wait complete. 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 Processor -- start 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.958+0000 7fb5d9fbe700 1 -- start start 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5d41002f0 0x7fb5d41960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:12.962 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5d41014f0 0x7fb5d4196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d419dc80 con 0x7fb5d41002f0 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d419ddc0 con 0x7fb5d41014f0 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d419df50 con 0x7fb5d4101e80 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39964/0 (socket says 172.21.15.204:39964) 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 -- 172.21.15.204:0/968710122 learned_addr learned my addr 172.21.15.204:0/968710122 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d2ffd700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5d41014f0 0x7fb5d4196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d37fe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5d41002f0 0x7fb5d41960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 -- 172.21.15.204:0/968710122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5d41014f0 msgr2=0x7fb5d4196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:12.963 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5d41014f0 0x7fb5d4196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 -- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5d41002f0 msgr2=0x7fb5d41960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5d41002f0 0x7fb5d41960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 -- 172.21.15.204:0/968710122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc009d00 con 0x7fb5d4101e80 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d3fff700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d419d6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc007ab0 tx=0x7fb5cc004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00e040 con 0x7fb5d4101e80 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5cc00cec0 con 0x7fb5d4101e80 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d410bc50 con 0x7fb5d4101e80 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc01bb30 con 0x7fb5d4101e80 2023-12-14T18:23:12.964 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.959+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d410bac0 con 0x7fb5d4101e80 2023-12-14T18:23:12.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-14T18:23:12.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi176 2023-12-14T18:23:12.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi176 to 8557M 2023-12-14T18:23:12.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:12.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:23:12.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:12.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-14T18:23:12.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: osdmap e29: 5 total, 4 up, 5 in 2023-12-14T18:23:12.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:12.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: from='osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:23:12.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:12 smithi031 ceph-mon[42085]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.960+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4107860 con 0x7fb5d4101e80 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.962+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb5cc01bc90 con 0x7fb5d4101e80 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.962+0000 7fb5d0ff9700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb5bc0a1150 0x7fb5bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.963+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7fb5cc0cd8c0 con 0x7fb5d4101e80 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.963+0000 7fb5d37fe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb5bc0a1150 0x7fb5bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.963+0000 7fb5d37fe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb5bc0a1150 0x7fb5bc0a3600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4101350 tx=0x7fb5c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:12.967 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:12.966+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb5cc098230 con 0x7fb5d4101e80 2023-12-14T18:23:12.992 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4[53974]: 2023-12-14T18:23:12.537+0000 7f02e975c700 -1 osd.4 0 waiting for initial osdmap 2023-12-14T18:23:12.992 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4[53974]: 2023-12-14T18:23:12.544+0000 7f02e354d700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:23:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-14T18:23:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi176 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi176 to 8557M 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: osdmap e29: 5 total, 4 up, 5 in 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: from='osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:23:12.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:12 smithi176 ceph-mon[43506]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:23:13.163 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:13.162+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5d4061b00 con 0x7fb5bc0a1150 2023-12-14T18:23:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:23:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: osdmap e30: 5 total, 4 up, 5 in 2023-12-14T18:23:13.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:13.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:13.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:13.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:13.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: osdmap e30: 5 total, 4 up, 5 in 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: osdmap e30: 5 total, 4 up, 5 in 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:23:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:23:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:23:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] boot 2023-12-14T18:23:14.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: osdmap e31: 5 total, 5 up, 5 in 2023-12-14T18:23:14.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:23:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:23:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:23:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] boot 2023-12-14T18:23:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: osdmap e31: 5 total, 5 up, 5 in 2023-12-14T18:23:14.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: osd.4 [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] boot 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: osdmap e31: 5 total, 5 up, 5 in 2023-12-14T18:23:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: osdmap e32: 5 total, 5 up, 5 in 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/1465801042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]: dispatch 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]: dispatch 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]': finished 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: osdmap e33: 6 total, 5 up, 6 in 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:15 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/864197869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:16.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: osdmap e32: 5 total, 5 up, 5 in 2023-12-14T18:23:16.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/1465801042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]: dispatch 2023-12-14T18:23:16.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]: dispatch 2023-12-14T18:23:16.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]': finished 2023-12-14T18:23:16.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: osdmap e33: 6 total, 5 up, 6 in 2023-12-14T18:23:16.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:16.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:15 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/864197869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:16.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: osdmap e32: 5 total, 5 up, 5 in 2023-12-14T18:23:16.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/1465801042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]: dispatch 2023-12-14T18:23:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]: dispatch 2023-12-14T18:23:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e58c9b5f-bcfc-48af-a8bf-e1744c662b84"}]': finished 2023-12-14T18:23:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: osdmap e33: 6 total, 5 up, 6 in 2023-12-14T18:23:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:15 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/864197869' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:17.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:16 smithi176 ceph-mon[43506]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:17.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:16 smithi204 ceph-mon[43866]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:17.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:16 smithi031 ceph-mon[42085]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:19.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:18 smithi176 ceph-mon[43506]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:19.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:18 smithi204 ceph-mon[43866]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:19.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:18 smithi031 ceph-mon[42085]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:21.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:20 smithi176 ceph-mon[43506]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:21.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-14T18:23:21.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:21.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:20 smithi204 ceph-mon[43866]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:21.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-14T18:23:21.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:21.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:20 smithi031 ceph-mon[42085]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:21.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-14T18:23:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:21 smithi204 ceph-mon[43866]: Deploying daemon osd.5 on smithi204 2023-12-14T18:23:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:21 smithi204 ceph-mon[43866]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:21 smithi176 ceph-mon[43506]: Deploying daemon osd.5 on smithi204 2023-12-14T18:23:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:21 smithi176 ceph-mon[43506]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:22.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:21 smithi031 ceph-mon[42085]: Deploying daemon osd.5 on smithi204 2023-12-14T18:23:22.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:21 smithi031 ceph-mon[42085]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:25.236 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:24 smithi204 ceph-mon[43866]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:25.236 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:25.236 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:25.236 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:24 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:25.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:24 smithi176 ceph-mon[43506]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:25.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:24 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:25.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:24 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:25.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:24 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:25.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:24 smithi031 ceph-mon[42085]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:24 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:26 smithi204 ceph-mon[43866]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:26 smithi031 ceph-mon[42085]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:26 smithi176 ceph-mon[43506]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:26.970 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.968+0000 7fb5d0ff9700 1 -- 172.21.15.204:0/968710122 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb5d4061b00 con 0x7fb5bc0a1150 2023-12-14T18:23:26.970 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2023-12-14T18:23:26.972 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb5bc0a1150 msgr2=0x7fb5bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:26.972 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb5bc0a1150 0x7fb5bc0a3600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4101350 tx=0x7fb5c400b410 comp rx=0 tx=0).stop 2023-12-14T18:23:26.972 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 msgr2=0x7fb5d419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d419d6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc007ab0 tx=0x7fb5cc004d00 comp rx=0 tx=0).stop 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 shutdown_connections 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb5bc0a1150 0x7fb5bc0a3600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d4101e80 0x7fb5d419d6b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5d41002f0 0x7fb5d41960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 --2- 172.21.15.204:0/968710122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5d41014f0 0x7fb5d4196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.970+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 >> 172.21.15.204:0/968710122 conn(0x7fb5d40fb900 msgr2=0x7fb5d41049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.971+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 shutdown_connections 2023-12-14T18:23:26.973 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:23:26.971+0000 7fb5d9fbe700 1 -- 172.21.15.204:0/968710122 wait complete. 2023-12-14T18:23:27.372 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.5.service 2023-12-14T18:23:27.374 INFO:tasks.cephadm:Deploying osd.6 on smithi031 with /dev/vg_nvme/lv_2... 2023-12-14T18:23:27.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.401 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:23:27.402 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:27.402 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.402 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.402 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:27.408 INFO:journalctl@ceph.osd.5.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:27.728 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:23:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:27.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:27.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:28.979 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:23:28 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:23:28.699+0000 7fc760557640 -1 osd.5 0 log_to_monitors true 2023-12-14T18:23:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-14T18:23:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-14T18:23:29.117 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-12-14T18:23:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-12-14T18:23:29.117 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123146 s, 851 MB/s 2023-12-14T18:23:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-12-14T18:23:29.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:28 smithi176 ceph-mon[43506]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:29.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:28 smithi176 ceph-mon[43506]: from='osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:23:29.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:28 smithi176 ceph-mon[43506]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:23:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:28 smithi204 ceph-mon[43866]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:28 smithi204 ceph-mon[43866]: from='osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:23:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:28 smithi204 ceph-mon[43866]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:23:29.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:28 smithi031 ceph-mon[42085]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:29.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:28 smithi031 ceph-mon[42085]: from='osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:23:29.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:28 smithi031 ceph-mon[42085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:23:29.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2023-12-14T18:23:29.873 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:23:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi204 2023-12-14T18:23:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.260 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi204 to 8556M 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:30.261 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:30.589 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:23:30.318+0000 7fc755bce700 -1 osd.5 0 waiting for initial osdmap 2023-12-14T18:23:30.589 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:23:30.327+0000 7fc7509c1700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:23:30.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi204 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi204 to 8556M 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: osdmap e34: 6 total, 5 up, 6 in 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:23:30.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:30 smithi204 ceph-mon[43866]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: osdmap e34: 6 total, 5 up, 6 in 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:23:30.696 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:30 smithi031 ceph-mon[42085]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:30.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi204 2023-12-14T18:23:30.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi204 to 8556M 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: osdmap e34: 6 total, 5 up, 6 in 2023-12-14T18:23:30.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:30.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:23:30.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:23:30.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:30 smithi176 ceph-mon[43506]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:30.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 -- 172.21.15.31:0/286184287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1764101200 msgr2=0x7f17640fe3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 --2- 172.21.15.31:0/286184287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1764101200 0x7f17640fe3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1750009990 tx=0x7f1750009ca0 comp rx=0 tx=0).stop 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 -- 172.21.15.31:0/286184287 shutdown_connections 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 --2- 172.21.15.31:0/286184287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17640fea60 0x7f17640fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 --2- 172.21.15.31:0/286184287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1764100820 0x7f1764100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 --2- 172.21.15.31:0/286184287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1764101200 0x7f17640fe3d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 -- 172.21.15.31:0/286184287 >> 172.21.15.31:0/286184287 conn(0x7f17640f9fa0 msgr2=0x7f17640fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:30.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 -- 172.21.15.31:0/286184287 shutdown_connections 2023-12-14T18:23:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.882+0000 7f176af6d700 1 -- 172.21.15.31:0/286184287 wait complete. 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 Processor -- start 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 -- start start 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 0x7f1764198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1764100820 0x7f1764198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1764101200 0x7f176419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176419fdb0 con 0x7f17640fea60 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176419fef0 con 0x7f1764101200 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.883+0000 7f176af6d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17641a0080 con 0x7f1764100820 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 0x7f1764198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 0x7f1764198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54640/0 (socket says 172.21.15.31:54640) 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 -- 172.21.15.31:0/1878146723 learned_addr learned my addr 172.21.15.31:0/1878146723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f176950a700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1764101200 0x7f176419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:30.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1763fff700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1764100820 0x7f1764198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 -- 172.21.15.31:0/1878146723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1764100820 msgr2=0x7f1764198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1764100820 0x7f1764198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 -- 172.21.15.31:0/1878146723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1764101200 msgr2=0x7f176419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1764101200 0x7f176419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 -- 172.21.15.31:0/1878146723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500095f0 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1768d09700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 0x7f1764198220 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f175800d9b0 tx=0x7f175800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800b950 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1758007620 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176410de80 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.884+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800f3c0 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.885+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176410dcf0 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.886+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176410a260 con 0x7f17640fea60 2023-12-14T18:23:30.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.887+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1758010460 con 0x7f17640fea60 2023-12-14T18:23:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.887+0000 7f1761ffb700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17540a11d0 0x7f17540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.887+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7f17580cb650 con 0x7f17640fea60 2023-12-14T18:23:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.887+0000 7f1763fff700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17540a11d0 0x7f17540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.888+0000 7f1763fff700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17540a11d0 0x7f17540a3680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f1750005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:30.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:30.892+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1758095d80 con 0x7f17640fea60 2023-12-14T18:23:31.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:31.113+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1764061c80 con 0x7f17540a11d0 2023-12-14T18:23:31.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:23:31.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: osdmap e35: 6 total, 5 up, 6 in 2023-12-14T18:23:31.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:31.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:31.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:31.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:31.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: osdmap e35: 6 total, 5 up, 6 in 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:31.606 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: osdmap e35: 6 total, 5 up, 6 in 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:32.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:23:32.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:23:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] boot 2023-12-14T18:23:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: osdmap e36: 6 total, 6 up, 6 in 2023-12-14T18:23:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:32 smithi204 ceph-mon[43866]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] boot 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: osdmap e36: 6 total, 6 up, 6 in 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:32 smithi031 ceph-mon[42085]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: osd.5 [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] boot 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: osdmap e36: 6 total, 6 up, 6 in 2023-12-14T18:23:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:23:32.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:32 smithi176 ceph-mon[43506]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:23:33.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:33 smithi204 ceph-mon[43866]: osdmap e37: 6 total, 6 up, 6 in 2023-12-14T18:23:33.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:33 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1148483634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78200a1a-a3f7-48fb-aeb7-c335cc8b9e86"}]: dispatch 2023-12-14T18:23:33.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:33 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1148483634' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78200a1a-a3f7-48fb-aeb7-c335cc8b9e86"}]': finished 2023-12-14T18:23:33.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:33 smithi204 ceph-mon[43866]: osdmap e38: 7 total, 6 up, 7 in 2023-12-14T18:23:33.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:33 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:33.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:33 smithi031 ceph-mon[42085]: osdmap e37: 6 total, 6 up, 6 in 2023-12-14T18:23:33.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:33 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1148483634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78200a1a-a3f7-48fb-aeb7-c335cc8b9e86"}]: dispatch 2023-12-14T18:23:33.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:33 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1148483634' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78200a1a-a3f7-48fb-aeb7-c335cc8b9e86"}]': finished 2023-12-14T18:23:33.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:33 smithi031 ceph-mon[42085]: osdmap e38: 7 total, 6 up, 7 in 2023-12-14T18:23:33.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:33 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:33 smithi176 ceph-mon[43506]: osdmap e37: 6 total, 6 up, 6 in 2023-12-14T18:23:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:33 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1148483634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "78200a1a-a3f7-48fb-aeb7-c335cc8b9e86"}]: dispatch 2023-12-14T18:23:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:33 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1148483634' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "78200a1a-a3f7-48fb-aeb7-c335cc8b9e86"}]': finished 2023-12-14T18:23:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:33 smithi176 ceph-mon[43506]: osdmap e38: 7 total, 6 up, 7 in 2023-12-14T18:23:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:33 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:34 smithi204 ceph-mon[43866]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:34 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3100884494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:34.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:34 smithi031 ceph-mon[42085]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:34.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:34 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3100884494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:34 smithi176 ceph-mon[43506]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:34 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3100884494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:35.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:35 smithi031 ceph-mon[42085]: osdmap e39: 7 total, 6 up, 7 in 2023-12-14T18:23:35.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:35 smithi176 ceph-mon[43506]: osdmap e39: 7 total, 6 up, 7 in 2023-12-14T18:23:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:35 smithi204 ceph-mon[43866]: osdmap e39: 7 total, 6 up, 7 in 2023-12-14T18:23:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:36.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:36 smithi031 ceph-mon[42085]: osdmap e40: 7 total, 6 up, 7 in 2023-12-14T18:23:36.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:36.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:36 smithi031 ceph-mon[42085]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:36.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:36 smithi176 ceph-mon[43506]: osdmap e40: 7 total, 6 up, 7 in 2023-12-14T18:23:36.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:36.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:36 smithi176 ceph-mon[43506]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:36 smithi204 ceph-mon[43866]: osdmap e40: 7 total, 6 up, 7 in 2023-12-14T18:23:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:36 smithi204 ceph-mon[43866]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:38.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:38 smithi031 ceph-mon[42085]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-12-14T18:23:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:38 smithi204 ceph-mon[43866]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-12-14T18:23:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:38 smithi176 ceph-mon[43506]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2023-12-14T18:23:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:40 smithi204 ceph-mon[43866]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-14T18:23:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-14T18:23:40.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:40.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:40 smithi204 ceph-mon[43866]: Deploying daemon osd.6 on smithi031 2023-12-14T18:23:40.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:40 smithi031 ceph-mon[42085]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-14T18:23:40.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-14T18:23:40.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:40.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:40 smithi031 ceph-mon[42085]: Deploying daemon osd.6 on smithi031 2023-12-14T18:23:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:40 smithi176 ceph-mon[43506]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2023-12-14T18:23:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-14T18:23:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:40 smithi176 ceph-mon[43506]: Deploying daemon osd.6 on smithi031 2023-12-14T18:23:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:42 smithi031 ceph-mon[42085]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-14T18:23:42.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:42 smithi204 ceph-mon[43866]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-14T18:23:42.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:42 smithi176 ceph-mon[43506]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-14T18:23:44.948 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:44 smithi031 ceph-mon[42085]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-14T18:23:44.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:44.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:44.949 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:44 smithi176 ceph-mon[43506]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-14T18:23:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:45.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:44 smithi204 ceph-mon[43866]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2023-12-14T18:23:45.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:45.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:45.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 6 on host 'smithi031' 2023-12-14T18:23:46.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.261+0000 7f1761ffb700 1 -- 172.21.15.31:0/1878146723 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1764061c80 con 0x7f17540a11d0 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17540a11d0 msgr2=0x7f17540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17540a11d0 0x7f17540a3680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1750006010 tx=0x7f1750005960 comp rx=0 tx=0).stop 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 msgr2=0x7f1764198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 0x7f1764198220 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f175800d9b0 tx=0x7f175800dcc0 comp rx=0 tx=0).stop 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 shutdown_connections 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1764101200 0x7f176419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17640fea60 0x7f1764198220 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:46.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17540a11d0 0x7f17540a3680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 --2- 172.21.15.31:0/1878146723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1764100820 0x7f1764198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 >> 172.21.15.31:0/1878146723 conn(0x7f17640f9fa0 msgr2=0x7f17640fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 shutdown_connections 2023-12-14T18:23:46.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:23:46.265+0000 7f176af6d700 1 -- 172.21.15.31:0/1878146723 wait complete. 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.430 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:23:46.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:46.431 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:46 smithi031 ceph-mon[42085]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:23:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:46.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:46 smithi176 ceph-mon[43506]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:46 smithi204 ceph-mon[43866]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:46.668 DEBUG:teuthology.orchestra.run.smithi031:osd.6> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.6.service 2023-12-14T18:23:46.670 INFO:tasks.cephadm:Deploying osd.7 on smithi176 with /dev/vg_nvme/lv_2... 2023-12-14T18:23:46.670 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-14T18:23:46.725 INFO:journalctl@ceph.osd.6.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:23:46.998 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:23:47.385 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:47.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:47.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:47.415 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:47.415 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:47.415 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:47.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:47.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:47.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:47.964 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:23:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:23:47.638+0000 7fe59fe0a640 -1 osd.6 0 log_to_monitors true 2023-12-14T18:23:48.233 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-14T18:23:48.233 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-14T18:23:48.233 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2023-12-14T18:23:48.233 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2023-12-14T18:23:48.234 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113127 s, 927 MB/s 2023-12-14T18:23:48.234 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2023-12-14T18:23:48.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:48 smithi176 ceph-mon[43506]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:48.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:48 smithi176 ceph-mon[43506]: from='osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:23:48.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:48 smithi176 ceph-mon[43506]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:23:48.493 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:48 smithi031 ceph-mon[42085]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:48.493 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:48 smithi031 ceph-mon[42085]: from='osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:23:48.493 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:48 smithi031 ceph-mon[42085]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:23:48.570 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_2 2023-12-14T18:23:48.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:48 smithi204 ceph-mon[43866]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:48.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:48 smithi204 ceph-mon[43866]: from='osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:23:48.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:48 smithi204 ceph-mon[43866]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:23:48.884 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:23:49.298 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: osdmap e41: 7 total, 6 up, 7 in 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi031 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.299 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi031 to 5706M 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:23:49.300 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:49 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:23:49.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-14T18:23:49.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: osdmap e41: 7 total, 6 up, 7 in 2023-12-14T18:23:49.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi031 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi031 to 5706M 2023-12-14T18:23:49.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:49.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:49.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:49.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:23:49.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:49 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:23:49.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:23:49.252+0000 7fe595481700 -1 osd.6 0 waiting for initial osdmap 2023-12-14T18:23:49.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:23:49.262+0000 7fe58fa73700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:23:49.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-14T18:23:49.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: osdmap e41: 7 total, 6 up, 7 in 2023-12-14T18:23:49.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi031 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi031 to 5706M 2023-12-14T18:23:49.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:49.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:49.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:49.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:23:49.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:23:49.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:49 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:23:49.842 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 -- 172.21.15.176:0/1684656948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 msgr2=0x7fa3d4101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:49.842 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 --2- 172.21.15.176:0/1684656948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d4101a90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40099c0 tx=0x7fa3c4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 -- 172.21.15.176:0/1684656948 shutdown_connections 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 --2- 172.21.15.176:0/1684656948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4101fd0 0x7fa3d4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 --2- 172.21.15.176:0/1684656948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 0x7fa3d40fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 --2- 172.21.15.176:0/1684656948 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d4101a90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 -- 172.21.15.176:0/1684656948 >> 172.21.15.176:0/1684656948 conn(0x7fa3d40f9f70 msgr2=0x7fa3d40fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 -- 172.21.15.176:0/1684656948 shutdown_connections 2023-12-14T18:23:49.843 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 -- 172.21.15.176:0/1684656948 wait complete. 2023-12-14T18:23:49.846 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.841+0000 7fa3da21d700 1 Processor -- start 2023-12-14T18:23:49.846 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 -- start start 2023-12-14T18:23:49.846 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 0x7fa3d4196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:49.846 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d41965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4101fd0 0x7fa3d419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419dc00 con 0x7fa3d40fe5c0 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419dd40 con 0x7fa3d4101640 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419ded0 con 0x7fa3d4101fd0 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d41965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d41965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:51252/0 (socket says 172.21.15.176:51252) 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 -- 172.21.15.176:0/3013277824 learned_addr learned my addr 172.21.15.176:0/3013277824 (peer_addr_for_me v2:172.21.15.176:0/0) 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d37fe700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 0x7fa3d4196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:49.847 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d3fff700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4101fd0 0x7fa3d419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 -- 172.21.15.176:0/3013277824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4101fd0 msgr2=0x7fa3d419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4101fd0 0x7fa3d419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 -- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 msgr2=0x7fa3d4196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 0x7fa3d4196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 -- 172.21.15.176:0/3013277824 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40095f0 con 0x7fa3d4101640 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d37fe700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 0x7fa3d4196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d2ffd700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d41965b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c010 tx=0x7fa3c4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c401d070 con 0x7fa3d4101640 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3c40040d0 con 0x7fa3d4101640 2023-12-14T18:23:49.848 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.842+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d410bc90 con 0x7fa3d4101640 2023-12-14T18:23:49.849 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.843+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4005290 con 0x7fa3d4101640 2023-12-14T18:23:49.849 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.843+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d410bb00 con 0x7fa3d4101640 2023-12-14T18:23:49.849 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.844+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d4066ea0 con 0x7fa3d4101640 2023-12-14T18:23:49.849 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.845+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa3c4004810 con 0x7fa3d4101640 2023-12-14T18:23:49.850 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.845+0000 7fa3d0ff9700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3bc0a1150 0x7fa3bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:49.850 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.846+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4685+0+0 (secure 0 0 0) 0x7fa3c40cce90 con 0x7fa3d4101640 2023-12-14T18:23:49.850 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.846+0000 7fa3d37fe700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3bc0a1150 0x7fa3bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:49.850 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.846+0000 7fa3d37fe700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3bc0a1150 0x7fa3bc0a3600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8005950 tx=0x7fa3c80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:49.851 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:49.849+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa3c4097400 con 0x7fa3d4101640 2023-12-14T18:23:50.047 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:23:50.045+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d419e940 con 0x7fa3bc0a1150 2023-12-14T18:23:50.536 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: osdmap e42: 7 total, 6 up, 7 in 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:50.537 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:50.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:23:50.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: osdmap e42: 7 total, 6 up, 7 in 2023-12-14T18:23:50.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:50.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:50.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:50.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:50.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:50.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: osdmap e42: 7 total, 6 up, 7 in 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:50.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:51.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:23:51.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:23:51.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] boot 2023-12-14T18:23:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: osdmap e43: 7 total, 7 up, 7 in 2023-12-14T18:23:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:51.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:23:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:23:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] boot 2023-12-14T18:23:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: osdmap e43: 7 total, 7 up, 7 in 2023-12-14T18:23:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:51.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:23:51.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:23:51.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:51.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:51.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: osd.6 [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] boot 2023-12-14T18:23:51.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: osdmap e43: 7 total, 7 up, 7 in 2023-12-14T18:23:51.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: osdmap e44: 7 total, 7 up, 7 in 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/1059782109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]: dispatch 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]: dispatch 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]': finished 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: osdmap e45: 8 total, 7 up, 8 in 2023-12-14T18:23:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:23:52.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: osdmap e44: 7 total, 7 up, 7 in 2023-12-14T18:23:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/1059782109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]: dispatch 2023-12-14T18:23:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]: dispatch 2023-12-14T18:23:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]': finished 2023-12-14T18:23:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: osdmap e45: 8 total, 7 up, 8 in 2023-12-14T18:23:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: osdmap e44: 7 total, 7 up, 7 in 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/1059782109' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]: dispatch 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]: dispatch 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d0a6115-2273-425b-84a6-086924f6da23"}]': finished 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: osdmap e45: 8 total, 7 up, 8 in 2023-12-14T18:23:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:23:53.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:53 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/3630770302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:53 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/3630770302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:53.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:53 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/3630770302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:54 smithi204 ceph-mon[43866]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:54 smithi031 ceph-mon[42085]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:54.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:54 smithi176 ceph-mon[43506]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:56.788 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:56 smithi176 ceph-mon[43506]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:56.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:56 smithi204 ceph-mon[43866]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:56.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:56 smithi031 ceph-mon[42085]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:58.796 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:58 smithi176 ceph-mon[43506]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:58.797 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-14T18:23:58.797 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:58.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:58 smithi204 ceph-mon[43866]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:58.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-14T18:23:58.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:58.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:58 smithi031 ceph-mon[42085]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:23:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-14T18:23:58.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:23:59 smithi204 ceph-mon[43866]: Deploying daemon osd.7 on smithi176 2023-12-14T18:23:59.841 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:23:59 smithi176 ceph-mon[43506]: Deploying daemon osd.7 on smithi176 2023-12-14T18:23:59.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:23:59 smithi031 ceph-mon[42085]: Deploying daemon osd.7 on smithi176 2023-12-14T18:24:00.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:00 smithi176 ceph-mon[43506]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:00 smithi204 ceph-mon[43866]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:00.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:00 smithi031 ceph-mon[42085]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:02.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:02 smithi031 ceph-mon[42085]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:02.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:02.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:02.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:02 smithi176 ceph-mon[43506]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:02 smithi204 ceph-mon[43866]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:03.953 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 7 on host 'smithi176' 2023-12-14T18:24:03.953 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.951+0000 7fa3d0ff9700 1 -- 172.21.15.176:0/3013277824 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa3d419e940 con 0x7fa3bc0a1150 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3bc0a1150 msgr2=0x7fa3bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3bc0a1150 0x7fa3bc0a3600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8005950 tx=0x7fa3c80058e0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 msgr2=0x7fa3d41965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d41965b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa3c400c010 tx=0x7fa3c4009580 comp rx=0 tx=0).stop 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 shutdown_connections 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3bc0a1150 0x7fa3bc0a3600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4101fd0 0x7fa3d419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40fe5c0 0x7fa3d4196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 --2- 172.21.15.176:0/3013277824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4101640 0x7fa3d41965b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 >> 172.21.15.176:0/3013277824 conn(0x7fa3d40f9f70 msgr2=0x7fa3d40fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 shutdown_connections 2023-12-14T18:24:03.955 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:03.953+0000 7fa3da21d700 1 -- 172.21.15.176:0/3013277824 wait complete. 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:04.402 DEBUG:teuthology.orchestra.run.smithi176:osd.7> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.7.service 2023-12-14T18:24:04.405 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2023-12-14T18:24:04.405 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-14T18:24:04.435 INFO:journalctl@ceph.osd.7.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:04.737 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:24:05.940 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:24:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:24:05.626+0000 7fb259057640 -1 osd.7 0 log_to_monitors true 2023-12-14T18:24:05.971 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-14T18:24:05.971 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-14T18:24:05.972 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2023-12-14T18:24:05.972 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2023-12-14T18:24:05.972 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0194111 s, 540 MB/s 2023-12-14T18:24:05.972 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2023-12-14T18:24:06.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2023-12-14T18:24:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:24:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:24:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:06.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.618 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:24:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:24:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:06.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:07.592 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.586+0000 7f7e98686700 1 -- 172.21.15.204:0/130973408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 msgr2=0x7f7e90100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.593 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.586+0000 7f7e98686700 1 --2- 172.21.15.204:0/130973408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90100930 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009990 tx=0x7f7e84009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.593 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.586+0000 7f7e98686700 1 -- 172.21.15.204:0/130973408 shutdown_connections 2023-12-14T18:24:07.593 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.586+0000 7f7e98686700 1 --2- 172.21.15.204:0/130973408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e901020b0 0x7f7e90108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.593 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 --2- 172.21.15.204:0/130973408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90100930 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.593 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 --2- 172.21.15.204:0/130973408 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 0x7f7e90101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.593 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- 172.21.15.204:0/130973408 >> 172.21.15.204:0/130973408 conn(0x7f7e900fbb50 msgr2=0x7f7e900fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- 172.21.15.204:0/130973408 shutdown_connections 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- 172.21.15.204:0/130973408 wait complete. 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 Processor -- start 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- start start 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 0x7f7e90196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e901020b0 0x7f7e9019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9019dee0 con 0x7f7e901020b0 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9019e020 con 0x7f7e90101720 2023-12-14T18:24:07.594 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e98686700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9019e1b0 con 0x7f7e90100520 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e96422700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e96422700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:38206/0 (socket says 172.21.15.204:38206) 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.587+0000 7f7e96422700 1 -- 172.21.15.204:0/3376589549 learned_addr learned my addr 172.21.15.204:0/3376589549 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96c23700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e901020b0 0x7f7e9019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e95c21700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 0x7f7e90196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96422700 1 -- 172.21.15.204:0/3376589549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 msgr2=0x7f7e90196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96422700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 0x7f7e90196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96422700 1 -- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e901020b0 msgr2=0x7f7e9019d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96422700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e901020b0 0x7f7e9019d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96422700 1 -- 172.21.15.204:0/3376589549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840095f0 con 0x7f7e90100520 2023-12-14T18:24:07.595 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e95c21700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 0x7f7e90196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e96422700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90196350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c010 tx=0x7f7e8400f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8401d070 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8400fc90 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84017650 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9010bf70 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.588+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9010bde0 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.589+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e90067b00 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.590+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7e84022470 con 0x7f7e90100520 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.591+0000 7f7e837fe700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e7c0a1150 0x7f7e7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.596 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.591+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f7e840cd6d0 con 0x7f7e90100520 2023-12-14T18:24:07.597 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.591+0000 7f7e95c21700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e7c0a1150 0x7f7e7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.597 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.591+0000 7f7e95c21700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e7c0a1150 0x7f7e7c0a3600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c007900 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:07.597 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.594+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7e84097b30 con 0x7f7e90100520 2023-12-14T18:24:07.788 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:07.788+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e9019eb90 con 0x7f7e7c0a1150 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi176 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi176 to 5704M 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-14T18:24:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: osdmap e46: 8 total, 7 up, 8 in 2023-12-14T18:24:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: from='osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:24:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:24:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:24:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: osdmap e47: 8 total, 7 up, 8 in 2023-12-14T18:24:07.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:07 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi176 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi176 to 5704M 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: osdmap e46: 8 total, 7 up, 8 in 2023-12-14T18:24:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:07.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: from='osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:24:07.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:24:07.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:24:07.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: osdmap e47: 8 total, 7 up, 8 in 2023-12-14T18:24:07.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:07.992 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:24:07.516+0000 7fb24e6ce700 -1 osd.7 0 waiting for initial osdmap 2023-12-14T18:24:07.992 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:24:07.527+0000 7fb2474bd700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:24:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi176 2023-12-14T18:24:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi176 to 5704M 2023-12-14T18:24:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: osdmap e46: 8 total, 7 up, 8 in 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: from='osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: osdmap e47: 8 total, 7 up, 8 in 2023-12-14T18:24:07.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:07 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] boot 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: osdmap e48: 8 total, 8 up, 8 in 2023-12-14T18:24:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:08 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] boot 2023-12-14T18:24:08.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: osdmap e48: 8 total, 8 up, 8 in 2023-12-14T18:24:08.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:08 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: osd.7 [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] boot 2023-12-14T18:24:08.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: osdmap e48: 8 total, 8 up, 8 in 2023-12-14T18:24:08.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:08 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:24:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:09 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:24:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:09 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:24:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:09 smithi204 ceph-mon[43866]: osdmap e49: 8 total, 8 up, 8 in 2023-12-14T18:24:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:09 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:24:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:09 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:24:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:09 smithi031 ceph-mon[42085]: osdmap e49: 8 total, 8 up, 8 in 2023-12-14T18:24:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:09 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:24:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:09 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:24:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:09 smithi176 ceph-mon[43506]: osdmap e49: 8 total, 8 up, 8 in 2023-12-14T18:24:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/255063057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]: dispatch 2023-12-14T18:24:10.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]: dispatch 2023-12-14T18:24:10.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]': finished 2023-12-14T18:24:10.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: osdmap e50: 9 total, 8 up, 9 in 2023-12-14T18:24:10.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:10.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/3494541664' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/255063057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]: dispatch 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]: dispatch 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]': finished 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: osdmap e50: 9 total, 8 up, 9 in 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/3494541664' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/255063057' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]: dispatch 2023-12-14T18:24:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]: dispatch 2023-12-14T18:24:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb114398-c403-4e55-83ea-cdab039ce2e4"}]': finished 2023-12-14T18:24:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: osdmap e50: 9 total, 8 up, 9 in 2023-12-14T18:24:10.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:10.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/3494541664' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:12.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:12 smithi204 ceph-mon[43866]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:12.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:12 smithi204 ceph-mon[43866]: osdmap e51: 9 total, 8 up, 9 in 2023-12-14T18:24:12.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:12 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:12.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:12 smithi031 ceph-mon[42085]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:12.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:12 smithi031 ceph-mon[42085]: osdmap e51: 9 total, 8 up, 9 in 2023-12-14T18:24:12.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:12 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:12 smithi176 ceph-mon[43506]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:12 smithi176 ceph-mon[43506]: osdmap e51: 9 total, 8 up, 9 in 2023-12-14T18:24:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:12 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:13 smithi204 ceph-mon[43866]: osdmap e52: 9 total, 8 up, 9 in 2023-12-14T18:24:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:13 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:13 smithi031 ceph-mon[42085]: osdmap e52: 9 total, 8 up, 9 in 2023-12-14T18:24:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:13 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:13 smithi176 ceph-mon[43506]: osdmap e52: 9 total, 8 up, 9 in 2023-12-14T18:24:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:13 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:14 smithi204 ceph-mon[43866]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:14 smithi031 ceph-mon[42085]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:14 smithi176 ceph-mon[43506]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:16.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:16 smithi204 ceph-mon[43866]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:16.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-14T18:24:16.805 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:16.806 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:16 smithi204 ceph-mon[43866]: Deploying daemon osd.8 on smithi204 2023-12-14T18:24:16.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:16 smithi031 ceph-mon[42085]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-14T18:24:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:16 smithi031 ceph-mon[42085]: Deploying daemon osd.8 on smithi204 2023-12-14T18:24:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:16 smithi176 ceph-mon[43506]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-14T18:24:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:16 smithi176 ceph-mon[43506]: Deploying daemon osd.8 on smithi204 2023-12-14T18:24:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:18 smithi204 ceph-mon[43866]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:18.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:18 smithi031 ceph-mon[42085]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:18 smithi176 ceph-mon[43506]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:20.424 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:20.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:20.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:20.425 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:20 smithi204 ceph-mon[43866]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:20.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:20.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:20.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:20.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:20 smithi031 ceph-mon[42085]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:20.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:20.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:20.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:20.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:20 smithi176 ceph-mon[43506]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:21.846 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.845+0000 7f7e837fe700 1 -- 172.21.15.204:0/3376589549 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e9019eb90 con 0x7f7e7c0a1150 2023-12-14T18:24:21.847 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e7c0a1150 msgr2=0x7f7e7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e7c0a1150 0x7f7e7c0a3600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c007900 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 msgr2=0x7f7e90196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90196350 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c010 tx=0x7f7e8400f7b0 comp rx=0 tx=0).stop 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 shutdown_connections 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e901020b0 0x7f7e9019d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e7c0a1150 0x7f7e7c0a3600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e90100520 0x7f7e90196350 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 --2- 172.21.15.204:0/3376589549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e90101720 0x7f7e90196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 >> 172.21.15.204:0/3376589549 conn(0x7f7e900fbb50 msgr2=0x7f7e900fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 shutdown_connections 2023-12-14T18:24:21.849 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:24:21.847+0000 7f7e98686700 1 -- 172.21.15.204:0/3376589549 wait complete. 2023-12-14T18:24:22.020 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:22.021 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:21 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:22.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:21 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:22.231 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.8.service 2023-12-14T18:24:22.233 INFO:tasks.cephadm:Deploying osd.9 on smithi031 with /dev/vg_nvme/lv_1... 2023-12-14T18:24:22.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:21 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:22.265 INFO:journalctl@ceph.osd.8.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:24:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:22.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:22 smithi204 ceph-mon[43866]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:22.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:22 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:22 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:22 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:22 smithi031 ceph-mon[42085]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:22 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:22 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:22 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:23.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:22 smithi176 ceph-mon[43506]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:23.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:22 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:23.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:22 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:23.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:22 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:23.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:24:23.297+0000 7fa36a406640 -1 osd.8 0 log_to_monitors true 2023-12-14T18:24:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-14T18:24:23.932 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-14T18:24:23.933 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2023-12-14T18:24:23.933 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2023-12-14T18:24:23.933 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145204 s, 722 MB/s 2023-12-14T18:24:23.933 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2023-12-14T18:24:24.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:24:24.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:24:24.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:24.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi204 2023-12-14T18:24:24.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.237 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi204 to 5704M 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:24.238 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:23 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi204 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi204 to 5704M 2023-12-14T18:24:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:24.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:24.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:24.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:24.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:23 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:24.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi031:vg_nvme/lv_1 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi204 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi204 to 5704M 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:24.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:23 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:25.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:25 smithi031 ceph-mon[42085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-14T18:24:25.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:25 smithi031 ceph-mon[42085]: osdmap e53: 9 total, 8 up, 9 in 2023-12-14T18:24:25.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:25 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:25.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:25 smithi031 ceph-mon[42085]: from='osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:24:25.099 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:25 smithi031 ceph-mon[42085]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:24:25.339 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:24:25.018+0000 7fa35fa7d700 -1 osd.8 0 waiting for initial osdmap 2023-12-14T18:24:25.339 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:24:25.032+0000 7fa35986e700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:24:25.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-mon[43866]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-14T18:24:25.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-mon[43866]: osdmap e53: 9 total, 8 up, 9 in 2023-12-14T18:24:25.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:25.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-mon[43866]: from='osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:24:25.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:25 smithi204 ceph-mon[43866]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:24:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:25 smithi176 ceph-mon[43506]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-14T18:24:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:25 smithi176 ceph-mon[43506]: osdmap e53: 9 total, 8 up, 9 in 2023-12-14T18:24:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:25 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:25 smithi176 ceph-mon[43506]: from='osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:24:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:25 smithi176 ceph-mon[43506]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 -- 172.21.15.31:0/2776512683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 msgr2=0x7f04c40ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 --2- 172.21.15.31:0/2776512683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c40ff5f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009990 tx=0x7f04c0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 -- 172.21.15.31:0/2776512683 shutdown_connections 2023-12-14T18:24:25.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 --2- 172.21.15.31:0/2776512683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c4100d70 0x7f04c4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 --2- 172.21.15.31:0/2776512683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c40ff5f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 --2- 172.21.15.31:0/2776512683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04c41003e0 0x7f04c4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 -- 172.21.15.31:0/2776512683 >> 172.21.15.31:0/2776512683 conn(0x7f04c40fa7f0 msgr2=0x7f04c40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 -- 172.21.15.31:0/2776512683 shutdown_connections 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.653+0000 7f04cc033700 1 -- 172.21.15.31:0/2776512683 wait complete. 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 Processor -- start 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 -- start start 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c4103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04c41003e0 0x7f04c41017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c4100d70 0x7f04c4101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c4102480 con 0x7f04c40ff1e0 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41025f0 con 0x7f04c41003e0 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04cc033700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c41a0080 con 0x7f04c4100d70 2023-12-14T18:24:25.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c4103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c4103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36924/0 (socket says 172.21.15.31:36924) 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 -- 172.21.15.31:0/2946280091 learned_addr learned my addr 172.21.15.31:0/2946280091 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c95ce700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04c41003e0 0x7f04c41017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04ca5d0700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c4100d70 0x7f04c4101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 -- 172.21.15.31:0/2946280091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c4100d70 msgr2=0x7f04c4101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c4100d70 0x7f04c4101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 -- 172.21.15.31:0/2946280091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04c41003e0 msgr2=0x7f04c41017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04c41003e0 0x7f04c41017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.654+0000 7f04c9dcf700 1 -- 172.21.15.31:0/2946280091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c00095f0 con 0x7f04c40ff1e0 2023-12-14T18:24:25.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.655+0000 7f04c9dcf700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c4103170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b680 tx=0x7f04c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:25.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.655+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c001d070 con 0x7f04c40ff1e0 2023-12-14T18:24:25.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.655+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04c00054f0 con 0x7f04c40ff1e0 2023-12-14T18:24:25.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.655+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c410ab30 con 0x7f04c40ff1e0 2023-12-14T18:24:25.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.655+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0004be0 con 0x7f04c40ff1e0 2023-12-14T18:24:25.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.655+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c410a9a0 con 0x7f04c40ff1e0 2023-12-14T18:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.656+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c41a0670 con 0x7f04c40ff1e0 2023-12-14T18:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.657+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f04c000fbc0 con 0x7f04c40ff1e0 2023-12-14T18:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.658+0000 7f04baffd700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f04b00a11d0 0x7f04b00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.658+0000 7f04c95ce700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f04b00a11d0 0x7f04b00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.658+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f04c00cd470 con 0x7f04c40ff1e0 2023-12-14T18:24:25.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.658+0000 7f04c95ce700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f04b00a11d0 0x7f04b00a3680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009c60 tx=0x7f04b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:25.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.662+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f04c0097720 con 0x7f04c40ff1e0 2023-12-14T18:24:25.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:25.881+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f04c41a0ce0 con 0x7f04b00a11d0 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: osdmap e54: 9 total, 8 up, 9 in 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:26.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:26 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:26.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:24:26.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: osdmap e54: 9 total, 8 up, 9 in 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:26 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: osdmap e54: 9 total, 8 up, 9 in 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:26 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:27.175 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:27 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:24:27.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:27 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:24:27.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:27.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:27 smithi031 ceph-mon[42085]: osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] boot 2023-12-14T18:24:27.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:27 smithi031 ceph-mon[42085]: osdmap e55: 9 total, 9 up, 9 in 2023-12-14T18:24:27.176 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:27 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:27 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:24:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:27 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:24:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:27 smithi204 ceph-mon[43866]: osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] boot 2023-12-14T18:24:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:27 smithi204 ceph-mon[43866]: osdmap e55: 9 total, 9 up, 9 in 2023-12-14T18:24:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:27 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:27 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:24:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:27 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:24:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:27 smithi176 ceph-mon[43506]: osd.8 [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] boot 2023-12-14T18:24:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:27 smithi176 ceph-mon[43506]: osdmap e55: 9 total, 9 up, 9 in 2023-12-14T18:24:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:27 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:24:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:28 smithi204 ceph-mon[43866]: osdmap e56: 9 total, 9 up, 9 in 2023-12-14T18:24:28.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:28 smithi204 ceph-mon[43866]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:28.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:28 smithi031 ceph-mon[42085]: osdmap e56: 9 total, 9 up, 9 in 2023-12-14T18:24:28.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:28 smithi031 ceph-mon[42085]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:28.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:28 smithi176 ceph-mon[43506]: osdmap e56: 9 total, 9 up, 9 in 2023-12-14T18:24:28.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:28 smithi176 ceph-mon[43506]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:29 smithi204 ceph-mon[43866]: osdmap e57: 9 total, 9 up, 9 in 2023-12-14T18:24:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3083144439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d9d2a00-4cc2-40e3-8587-405fa776464d"}]: dispatch 2023-12-14T18:24:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3083144439' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d9d2a00-4cc2-40e3-8587-405fa776464d"}]': finished 2023-12-14T18:24:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:29 smithi204 ceph-mon[43866]: osdmap e58: 10 total, 9 up, 10 in 2023-12-14T18:24:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:29 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:29.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/901524987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:29.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:29 smithi031 ceph-mon[42085]: osdmap e57: 9 total, 9 up, 9 in 2023-12-14T18:24:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3083144439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d9d2a00-4cc2-40e3-8587-405fa776464d"}]: dispatch 2023-12-14T18:24:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3083144439' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d9d2a00-4cc2-40e3-8587-405fa776464d"}]': finished 2023-12-14T18:24:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:29 smithi031 ceph-mon[42085]: osdmap e58: 10 total, 9 up, 10 in 2023-12-14T18:24:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:29 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/901524987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:29 smithi176 ceph-mon[43506]: osdmap e57: 9 total, 9 up, 9 in 2023-12-14T18:24:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3083144439' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4d9d2a00-4cc2-40e3-8587-405fa776464d"}]: dispatch 2023-12-14T18:24:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3083144439' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4d9d2a00-4cc2-40e3-8587-405fa776464d"}]': finished 2023-12-14T18:24:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:29 smithi176 ceph-mon[43506]: osdmap e58: 10 total, 9 up, 10 in 2023-12-14T18:24:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:29 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/901524987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:30.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:30 smithi204 ceph-mon[43866]: osdmap e59: 10 total, 9 up, 10 in 2023-12-14T18:24:30.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:30.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:30 smithi204 ceph-mon[43866]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 168 KiB/s, 0 objects/s recovering 2023-12-14T18:24:30.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:30 smithi031 ceph-mon[42085]: osdmap e59: 10 total, 9 up, 10 in 2023-12-14T18:24:30.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:30.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:30 smithi031 ceph-mon[42085]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 168 KiB/s, 0 objects/s recovering 2023-12-14T18:24:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:30 smithi176 ceph-mon[43506]: osdmap e59: 10 total, 9 up, 10 in 2023-12-14T18:24:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:30 smithi176 ceph-mon[43506]: pgmap v160: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 168 KiB/s, 0 objects/s recovering 2023-12-14T18:24:31.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:31 smithi204 ceph-mon[43866]: osdmap e60: 10 total, 9 up, 10 in 2023-12-14T18:24:31.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:31 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:31.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:31 smithi031 ceph-mon[42085]: osdmap e60: 10 total, 9 up, 10 in 2023-12-14T18:24:31.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:31 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:31.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:31 smithi176 ceph-mon[43506]: osdmap e60: 10 total, 9 up, 10 in 2023-12-14T18:24:31.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:31 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:32.119 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:32 smithi031 ceph-mon[42085]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:24:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:32 smithi204 ceph-mon[43866]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:24:32.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:32 smithi176 ceph-mon[43506]: pgmap v162: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:24:34.194 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:34 smithi031 ceph-mon[42085]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-12-14T18:24:34.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:34 smithi204 ceph-mon[43866]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-12-14T18:24:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:34 smithi176 ceph-mon[43506]: pgmap v163: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-12-14T18:24:35.124 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-14T18:24:35.124 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:35.124 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:35 smithi031 ceph-mon[42085]: Deploying daemon osd.9 on smithi031 2023-12-14T18:24:35.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-14T18:24:35.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:35.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:35 smithi204 ceph-mon[43866]: Deploying daemon osd.9 on smithi031 2023-12-14T18:24:35.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-14T18:24:35.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:35.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:35 smithi176 ceph-mon[43506]: Deploying daemon osd.9 on smithi031 2023-12-14T18:24:36.126 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:36 smithi031 ceph-mon[42085]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-12-14T18:24:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:36 smithi204 ceph-mon[43866]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-12-14T18:24:36.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:36 smithi176 ceph-mon[43506]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-12-14T18:24:38.771 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:38 smithi031 ceph-mon[42085]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:38.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:38 smithi204 ceph-mon[43866]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:38 smithi176 ceph-mon[43506]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:40 smithi031 ceph-mon[42085]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:40 smithi204 ceph-mon[43866]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.886 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 9 on host 'smithi031' 2023-12-14T18:24:40.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.883+0000 7f04baffd700 1 -- 172.21.15.31:0/2946280091 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f04c41a0ce0 con 0x7f04b00a11d0 2023-12-14T18:24:40.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.886+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f04b00a11d0 msgr2=0x7f04b00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.886+0000 7f04cc033700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f04b00a11d0 0x7f04b00a3680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f04b4009c60 tx=0x7f04b4009260 comp rx=0 tx=0).stop 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.886+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 msgr2=0x7f04c4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.886+0000 7f04cc033700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c4103170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b680 tx=0x7f04c0004730 comp rx=0 tx=0).stop 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 shutdown_connections 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f04b00a11d0 0x7f04b00a3680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c4100d70 0x7f04c4101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04c40ff1e0 0x7f04c4103170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 --2- 172.21.15.31:0/2946280091 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04c41003e0 0x7f04c41017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 >> 172.21.15.31:0/2946280091 conn(0x7f04c40fa7f0 msgr2=0x7f04c4068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 shutdown_connections 2023-12-14T18:24:40.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:24:40.887+0000 7f04cc033700 1 -- 172.21.15.31:0/2946280091 wait complete. 2023-12-14T18:24:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:40 smithi176 ceph-mon[43506]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:41.333 DEBUG:teuthology.orchestra.run.smithi031:osd.9> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.9.service 2023-12-14T18:24:41.335 INFO:tasks.cephadm:Deploying osd.10 on smithi176 with /dev/vg_nvme/lv_1... 2023-12-14T18:24:41.336 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-14T18:24:41.370 INFO:journalctl@ceph.osd.9.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:24:41.654 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:24:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:41 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:41 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:41 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:41 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:41 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:41 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:42.139 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:41 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:42.139 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:41 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:42.139 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:41 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:42.139 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:42.139 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.139 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:42.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:24:42 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:24:42.240+0000 7f7655272640 -1 osd.9 0 log_to_monitors true 2023-12-14T18:24:42.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:42 smithi031 ceph-mon[42085]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:42.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:42 smithi031 ceph-mon[42085]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-14T18:24:43.006 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-14T18:24:43.006 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-14T18:24:43.006 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2023-12-14T18:24:43.006 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2023-12-14T18:24:43.006 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112681 s, 931 MB/s 2023-12-14T18:24:43.006 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2023-12-14T18:24:43.197 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:42 smithi176 ceph-mon[43506]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:43.197 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:42 smithi176 ceph-mon[43506]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-14T18:24:43.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:42 smithi204 ceph-mon[43866]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:43.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:42 smithi204 ceph-mon[43866]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-14T18:24:43.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_1 2023-12-14T18:24:43.655 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: osdmap e61: 10 total, 9 up, 10 in 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:44.047 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.214 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:24:43.845+0000 7f764a8e9700 -1 osd.9 0 waiting for initial osdmap 2023-12-14T18:24:44.214 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:24:43.862+0000 7f7644edb700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: osdmap e61: 10 total, 9 up, 10 in 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:44.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:44.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: osdmap e61: 10 total, 9 up, 10 in 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:44.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:44.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:44.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 -- 172.21.15.176:0/3126013216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101ea0 msgr2=0x7fd004108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/3126013216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101ea0 0x7fd004108760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00b0a0 tx=0x7fcffc00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 -- 172.21.15.176:0/3126013216 shutdown_connections 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/3126013216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101ea0 0x7fd004108760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/3126013216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd004100310 0x7fd004100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/3126013216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd004101510 0x7fd004101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 -- 172.21.15.176:0/3126013216 >> 172.21.15.176:0/3126013216 conn(0x7fd0040fb960 msgr2=0x7fd0040fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 -- 172.21.15.176:0/3126013216 shutdown_connections 2023-12-14T18:24:44.603 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.599+0000 7fd00c3ca700 1 -- 172.21.15.176:0/3126013216 wait complete. 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 Processor -- start 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 -- start start 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd004100310 0x7fd004196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 0x7fd0041966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd004101ea0 0x7fd00419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419dd00 con 0x7fd004101ea0 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419de40 con 0x7fd004101510 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd00c3ca700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00419dfd0 con 0x7fd004100310 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd009965700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 0x7fd0041966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd009965700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 0x7fd0041966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40180/0 (socket says 172.21.15.176:40180) 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.600+0000 7fd009965700 1 -- 172.21.15.176:0/1467026597 learned_addr learned my addr 172.21.15.176:0/1467026597 (peer_addr_for_me v2:172.21.15.176:0/0) 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd009965700 1 -- 172.21.15.176:0/1467026597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd004100310 msgr2=0x7fd004196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd009965700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd004100310 0x7fd004196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd009965700 1 -- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd004101ea0 msgr2=0x7fd00419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.604 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd00a967700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd004101ea0 0x7fd00419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd009965700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd004101ea0 0x7fd00419d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd009965700 1 -- 172.21.15.176:0/1467026597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc009d00 con 0x7fd004101510 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd009965700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 0x7fd0041966b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd00000b530 tx=0x7fd00000b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000011830 con 0x7fd004101510 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd000011e50 con 0x7fd004101510 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00410bcf0 con 0x7fd004101510 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000f2e0 con 0x7fd004101510 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.601+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00410bb60 con 0x7fd004101510 2023-12-14T18:24:44.605 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.602+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00419e7e0 con 0x7fd004101510 2023-12-14T18:24:44.606 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.604+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd000010580 con 0x7fd004101510 2023-12-14T18:24:44.606 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.605+0000 7fcffb7fe700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcff00a11d0 0x7fcff00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.606 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.605+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5981+0+0 (secure 0 0 0) 0x7fd0000cc070 con 0x7fd004101510 2023-12-14T18:24:44.606 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.605+0000 7fd00a166700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcff00a11d0 0x7fcff00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.607 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.606+0000 7fd00a166700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcff00a11d0 0x7fcff00a3680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd004101370 tx=0x7fcff4005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:44.609 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.609+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd000096040 con 0x7fd004101510 2023-12-14T18:24:44.806 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:44.804+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd00419ed90 con 0x7fcff00a11d0 2023-12-14T18:24:45.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi031 2023-12-14T18:24:45.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi031 to 4279M 2023-12-14T18:24:45.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:45.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:45.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:45.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: osdmap e62: 10 total, 9 up, 10 in 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:45.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi031 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi031 to 4279M 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: osdmap e62: 10 total, 9 up, 10 in 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:45.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:45.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:45.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi031 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi031 to 4279M 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: from='osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2023-12-14T18:24:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: osdmap e62: 10 total, 9 up, 10 in 2023-12-14T18:24:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:24:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:24:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:45 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:24:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:45 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:24:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:45 smithi031 ceph-mon[42085]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:45 smithi031 ceph-mon[42085]: osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] boot 2023-12-14T18:24:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:45 smithi031 ceph-mon[42085]: osdmap e63: 10 total, 10 up, 10 in 2023-12-14T18:24:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:45 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:45 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:24:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:45 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:24:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:45 smithi176 ceph-mon[43506]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:45 smithi176 ceph-mon[43506]: osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] boot 2023-12-14T18:24:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:45 smithi176 ceph-mon[43506]: osdmap e63: 10 total, 10 up, 10 in 2023-12-14T18:24:46.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:45 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:45 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:24:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:45 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:24:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:45 smithi204 ceph-mon[43866]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:45 smithi204 ceph-mon[43866]: osd.9 [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] boot 2023-12-14T18:24:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:45 smithi204 ceph-mon[43866]: osdmap e63: 10 total, 10 up, 10 in 2023-12-14T18:24:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:45 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: osdmap e64: 10 total, 10 up, 10 in 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/2882120976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]: dispatch 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]: dispatch 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]': finished 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: osdmap e65: 11 total, 10 up, 11 in 2023-12-14T18:24:47.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: osdmap e64: 10 total, 10 up, 10 in 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/2882120976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]: dispatch 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]: dispatch 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]': finished 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: osdmap e65: 11 total, 10 up, 11 in 2023-12-14T18:24:47.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: osdmap e64: 10 total, 10 up, 10 in 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/2882120976' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]: dispatch 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]: dispatch 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caf93ab4-d4cc-4d56-a291-abbc32df979b"}]': finished 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: osdmap e65: 11 total, 10 up, 11 in 2023-12-14T18:24:47.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:24:48.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:47 smithi031 ceph-mon[42085]: from='client.? 172.21.15.176:0/237301147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:48.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:47 smithi176 ceph-mon[43506]: from='client.? 172.21.15.176:0/237301147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:48.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:47 smithi204 ceph-mon[43866]: from='client.? 172.21.15.176:0/237301147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:24:49.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:48 smithi031 ceph-mon[42085]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:49.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:48 smithi176 ceph-mon[43506]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:49.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:48 smithi204 ceph-mon[43866]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:51.129 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:50 smithi176 ceph-mon[43506]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:51.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:50 smithi031 ceph-mon[42085]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:51.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:50 smithi204 ceph-mon[43866]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:53.198 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:52 smithi176 ceph-mon[43506]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:53.198 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-14T18:24:53.198 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:53.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:52 smithi031 ceph-mon[42085]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-14T18:24:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:52 smithi204 ceph-mon[43866]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-14T18:24:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:54.125 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:53 smithi176 ceph-mon[43506]: Deploying daemon osd.10 on smithi176 2023-12-14T18:24:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:53 smithi031 ceph-mon[42085]: Deploying daemon osd.10 on smithi176 2023-12-14T18:24:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:53 smithi204 ceph-mon[43866]: Deploying daemon osd.10 on smithi176 2023-12-14T18:24:54.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:54 smithi176 ceph-mon[43506]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:54 smithi031 ceph-mon[42085]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:54 smithi204 ceph-mon[43866]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:57.055 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:56 smithi176 ceph-mon[43506]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:57.056 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:57.056 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:57.056 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:56 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:57.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:56 smithi031 ceph-mon[42085]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:56 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:56 smithi204 ceph-mon[43866]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:56 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:58.612 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.610+0000 7fcffb7fe700 1 -- 172.21.15.176:0/1467026597 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd00419ed90 con 0x7fcff00a11d0 2023-12-14T18:24:58.612 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 10 on host 'smithi176' 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.612+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcff00a11d0 msgr2=0x7fcff00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.612+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcff00a11d0 0x7fcff00a3680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd004101370 tx=0x7fcff4005ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.612+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 msgr2=0x7fd0041966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.612+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 0x7fd0041966b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd00000b530 tx=0x7fd00000b8f0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 shutdown_connections 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd004101ea0 0x7fd00419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcff00a11d0 0x7fcff00a3680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.614 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd004100310 0x7fd004196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.615 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 --2- 172.21.15.176:0/1467026597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd004101510 0x7fd0041966b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.615 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 >> 172.21.15.176:0/1467026597 conn(0x7fd0040fb960 msgr2=0x7fd0040fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:58.615 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 shutdown_connections 2023-12-14T18:24:58.615 INFO:teuthology.orchestra.run.smithi176.stderr:2023-12-14T18:24:58.613+0000 7fd00c3ca700 1 -- 172.21.15.176:0/1467026597 wait complete. 2023-12-14T18:24:59.036 DEBUG:teuthology.orchestra.run.smithi176:osd.10> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.10.service 2023-12-14T18:24:59.039 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2023-12-14T18:24:59.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-14T18:24:59.072 INFO:journalctl@ceph.osd.10.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.145 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:24:59.146 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:24:59.146 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.146 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.146 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:24:58 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:24:59.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:24:59.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:24:58 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:24:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:59.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:59.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:24:59.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:24:59.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:24:59.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:24:58 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:59.360 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:25:00.561 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-14T18:25:00.561 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-14T18:25:00.562 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2023-12-14T18:25:00.562 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2023-12-14T18:25:00.562 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011032 s, 950 MB/s 2023-12-14T18:25:00.562 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2023-12-14T18:25:00.562 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:25:00.305+0000 7f163be49640 -1 osd.10 0 log_to_monitors true 2023-12-14T18:25:00.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:01.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:01.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:25:01.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:25:01.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:01.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.261 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:25:01.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:01.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:01.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:02.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 -- 172.21.15.204:0/3390435921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 msgr2=0x7f0150100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 --2- 172.21.15.204:0/3390435921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150100720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0144009990 tx=0x7f0144009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 -- 172.21.15.204:0/3390435921 shutdown_connections 2023-12-14T18:25:02.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 --2- 172.21.15.204:0/3390435921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0150101ea0 0x7f0150108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.206 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 --2- 172.21.15.204:0/3390435921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150100720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 --2- 172.21.15.204:0/3390435921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101510 0x7f0150101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 -- 172.21.15.204:0/3390435921 >> 172.21.15.204:0/3390435921 conn(0x7f01500fb960 msgr2=0x7f01500fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 -- 172.21.15.204:0/3390435921 shutdown_connections 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.201+0000 7f0157914700 1 -- 172.21.15.204:0/3390435921 wait complete. 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 Processor -- start 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 -- start start 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0150101510 0x7f01501965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101ea0 0x7f015019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015019dbf0 con 0x7f0150101ea0 2023-12-14T18:25:02.207 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015019dd30 con 0x7f0150101510 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0157914700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015019dec0 con 0x7f0150100310 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33556/0 (socket says 172.21.15.204:33556) 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 -- 172.21.15.204:0/2436834282 learned_addr learned my addr 172.21.15.204:0/2436834282 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0155eb1700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101ea0 0x7f015019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0154eaf700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0150101510 0x7f01501965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 -- 172.21.15.204:0/2436834282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0150101510 msgr2=0x7f01501965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0150101510 0x7f01501965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 -- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101ea0 msgr2=0x7f015019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101ea0 0x7f015019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f01556b0700 1 -- 172.21.15.204:0/2436834282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440095f0 con 0x7f0150100310 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.202+0000 7f0155eb1700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101ea0 0x7f015019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:02.208 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.203+0000 7f01556b0700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150196060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f014400c010 tx=0x7f014400f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.203+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014401d070 con 0x7f0150100310 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.203+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015010bc60 con 0x7f0150100310 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.203+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f014400fc90 con 0x7f0150100310 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.203+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144017650 con 0x7f0150100310 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.203+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015010bad0 con 0x7f0150100310 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.204+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0150107900 con 0x7f0150100310 2023-12-14T18:25:02.209 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.205+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0144022470 con 0x7f0150100310 2023-12-14T18:25:02.210 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.205+0000 7f01427fc700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f013c0a1150 0x7f013c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.210 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.205+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f01440cdb70 con 0x7f0150100310 2023-12-14T18:25:02.210 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.205+0000 7f0154eaf700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f013c0a1150 0x7f013c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.210 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.206+0000 7f0154eaf700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f013c0a1150 0x7f013c0a3600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f014c005950 tx=0x7f014c005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:02.210 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.209+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0144097ac0 con 0x7f0150100310 2023-12-14T18:25:02.242 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:25:01.970+0000 7f16314c0700 -1 osd.10 0 waiting for initial osdmap 2023-12-14T18:25:02.242 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:25:01.985+0000 7f16292ad700 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:25:02.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi176 2023-12-14T18:25:02.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi176 to 4278M 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: osdmap e66: 11 total, 10 up, 11 in 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: from='osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:25:02.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:01 smithi176 ceph-mon[43506]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:25:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi176 2023-12-14T18:25:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi176 to 4278M 2023-12-14T18:25:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:25:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-14T18:25:02.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: osdmap e66: 11 total, 10 up, 11 in 2023-12-14T18:25:02.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:02.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: from='osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:25:02.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:01 smithi204 ceph-mon[43866]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:25:02.409 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:02.408+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0150061a70 con 0x7f013c0a1150 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi176 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi176 to 4278M 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: osdmap e66: 11 total, 10 up, 11 in 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: from='osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:25:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:01 smithi031 ceph-mon[42085]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2023-12-14T18:25:03.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:03.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:25:03.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: osdmap e67: 11 total, 10 up, 11 in 2023-12-14T18:25:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:25:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:25:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:02 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: osdmap e67: 11 total, 10 up, 11 in 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:25:03.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:02 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: osdmap e67: 11 total, 10 up, 11 in 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:25:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:02 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:25:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:25:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] boot 2023-12-14T18:25:04.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: osdmap e68: 11 total, 11 up, 11 in 2023-12-14T18:25:04.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:04.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:03 smithi176 ceph-mon[43506]: osdmap e69: 11 total, 11 up, 11 in 2023-12-14T18:25:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:25:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:25:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] boot 2023-12-14T18:25:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: osdmap e68: 11 total, 11 up, 11 in 2023-12-14T18:25:04.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:04.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:03 smithi204 ceph-mon[43866]: osdmap e69: 11 total, 11 up, 11 in 2023-12-14T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: osd.10 [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] boot 2023-12-14T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: osdmap e68: 11 total, 11 up, 11 in 2023-12-14T18:25:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:25:04.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:03 smithi031 ceph-mon[42085]: osdmap e69: 11 total, 11 up, 11 in 2023-12-14T18:25:04.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:04.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: osdmap e70: 11 total, 11 up, 11 in 2023-12-14T18:25:04.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/3765640148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]: dispatch 2023-12-14T18:25:04.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]: dispatch 2023-12-14T18:25:04.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]': finished 2023-12-14T18:25:04.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: osdmap e71: 12 total, 11 up, 12 in 2023-12-14T18:25:04.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:04.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:04 smithi204 ceph-mon[43866]: from='client.? 172.21.15.204:0/1906177873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:25:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: osdmap e70: 11 total, 11 up, 11 in 2023-12-14T18:25:05.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/3765640148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]: dispatch 2023-12-14T18:25:05.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]: dispatch 2023-12-14T18:25:05.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]': finished 2023-12-14T18:25:05.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: osdmap e71: 12 total, 11 up, 12 in 2023-12-14T18:25:05.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:05.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:04 smithi176 ceph-mon[43506]: from='client.? 172.21.15.204:0/1906177873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:25:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:25:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: osdmap e70: 11 total, 11 up, 11 in 2023-12-14T18:25:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/3765640148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]: dispatch 2023-12-14T18:25:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]: dispatch 2023-12-14T18:25:05.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3cf56d3-54b6-43d9-91cd-1b869bcf5146"}]': finished 2023-12-14T18:25:05.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: osdmap e71: 12 total, 11 up, 12 in 2023-12-14T18:25:05.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:05.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:04 smithi031 ceph-mon[42085]: from='client.? 172.21.15.204:0/1906177873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:25:06.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:06 smithi031 ceph-mon[42085]: osdmap e72: 12 total, 11 up, 12 in 2023-12-14T18:25:06.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:06 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:06.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:06 smithi031 ceph-mon[42085]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:06 smithi176 ceph-mon[43506]: osdmap e72: 12 total, 11 up, 12 in 2023-12-14T18:25:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:06 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:06 smithi176 ceph-mon[43506]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:06 smithi204 ceph-mon[43866]: osdmap e72: 12 total, 11 up, 12 in 2023-12-14T18:25:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:06 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:06 smithi204 ceph-mon[43866]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:07.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:07 smithi031 ceph-mon[42085]: osdmap e73: 12 total, 11 up, 12 in 2023-12-14T18:25:07.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:07 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:07.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:07 smithi176 ceph-mon[43506]: osdmap e73: 12 total, 11 up, 12 in 2023-12-14T18:25:07.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:07 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:07.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:07 smithi204 ceph-mon[43866]: osdmap e73: 12 total, 11 up, 12 in 2023-12-14T18:25:07.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:07 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:08.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:08 smithi176 ceph-mon[43506]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:08.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:08 smithi204 ceph-mon[43866]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:08.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:08 smithi031 ceph-mon[42085]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:10 smithi204 ceph-mon[43866]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-12-14T18:25:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:10 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-14T18:25:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:10 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:10.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:10 smithi031 ceph-mon[42085]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-12-14T18:25:10.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-14T18:25:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:10 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:10 smithi176 ceph-mon[43506]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-12-14T18:25:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:10 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-14T18:25:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:10 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:11.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:11 smithi204 ceph-mon[43866]: Deploying daemon osd.11 on smithi204 2023-12-14T18:25:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:11 smithi031 ceph-mon[42085]: Deploying daemon osd.11 on smithi204 2023-12-14T18:25:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:11 smithi176 ceph-mon[43506]: Deploying daemon osd.11 on smithi204 2023-12-14T18:25:12.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:12 smithi204 ceph-mon[43866]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-14T18:25:12.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:12 smithi031 ceph-mon[42085]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-14T18:25:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:12 smithi176 ceph-mon[43506]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-14T18:25:14.942 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:14 smithi204 ceph-mon[43866]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-12-14T18:25:14.943 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:14.943 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:14.943 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:14 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:14 smithi031 ceph-mon[42085]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-12-14T18:25:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:15.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:14 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:15.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:14 smithi176 ceph-mon[43506]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2023-12-14T18:25:15.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:15.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:15.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:14 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:16.315 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.314+0000 7f01427fc700 1 -- 172.21.15.204:0/2436834282 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f0150061a70 con 0x7f013c0a1150 2023-12-14T18:25:16.315 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2023-12-14T18:25:16.317 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f013c0a1150 msgr2=0x7f013c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:16.317 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f013c0a1150 0x7f013c0a3600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f014c005950 tx=0x7f014c005cf0 comp rx=0 tx=0).stop 2023-12-14T18:25:16.317 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 msgr2=0x7f0150196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:16.317 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150196060 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f014400c010 tx=0x7f014400f7b0 comp rx=0 tx=0).stop 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 shutdown_connections 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0150101ea0 0x7f015019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f013c0a1150 0x7f013c0a3600 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0150100310 0x7f0150196060 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 --2- 172.21.15.204:0/2436834282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0150101510 0x7f01501965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.316+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 >> 172.21.15.204:0/2436834282 conn(0x7f01500fb960 msgr2=0x7f0150104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.317+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 shutdown_connections 2023-12-14T18:25:16.318 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:16.317+0000 7f0157914700 1 -- 172.21.15.204:0/2436834282 wait complete. 2023-12-14T18:25:16.690 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.11.service 2023-12-14T18:25:16.693 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-14T18:25:16.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd stat -f json 2023-12-14T18:25:16.727 INFO:journalctl@ceph.osd.11.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.824 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:16 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:16.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:25:16.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:16 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:25:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:16 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:17.053 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 -- 172.21.15.31:0/3834936442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 msgr2=0x7f30840ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 --2- 172.21.15.31:0/3834936442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f30840ff5f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3074009990 tx=0x7f3074009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 -- 172.21.15.31:0/3834936442 shutdown_connections 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 --2- 172.21.15.31:0/3834936442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3084100d70 0x7f3084107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 --2- 172.21.15.31:0/3834936442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f30840ff5f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 --2- 172.21.15.31:0/3834936442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f30841003e0 0x7f3084100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 -- 172.21.15.31:0/3834936442 >> 172.21.15.31:0/3834936442 conn(0x7f30840fa7f0 msgr2=0x7f30840fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 -- 172.21.15.31:0/3834936442 shutdown_connections 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.050+0000 7f3089a82700 1 -- 172.21.15.31:0/3834936442 wait complete. 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 Processor -- start 2023-12-14T18:25:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 -- start start 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f3084109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30841003e0 0x7f30841080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3084100d70 0x7f3084108620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084108da0 con 0x7f30840ff1e0 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084108f10 con 0x7f3084100d70 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3089a82700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841a0110 con 0x7f30841003e0 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f30837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f3084109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f30837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f3084109a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59640/0 (socket says 172.21.15.31:59640) 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f30837fe700 1 -- 172.21.15.31:0/2789218629 learned_addr learned my addr 172.21.15.31:0/2789218629 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.051+0000 7f3082ffd700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30841003e0 0x7f30841080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3083fff700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3084100d70 0x7f3084108620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f30837fe700 1 -- 172.21.15.31:0/2789218629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30841003e0 msgr2=0x7f30841080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f30837fe700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30841003e0 0x7f30841080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f30837fe700 1 -- 172.21.15.31:0/2789218629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3084100d70 msgr2=0x7f3084108620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f30837fe700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3084100d70 0x7f3084108620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f30837fe700 1 -- 172.21.15.31:0/2789218629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740095f0 con 0x7f30840ff1e0 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3083fff700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3084100d70 0x7f3084108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f30837fe700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f3084109a90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3074000c00 tx=0x7f3074009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307401d070 con 0x7f30840ff1e0 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308410bc80 con 0x7f30840ff1e0 2023-12-14T18:25:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30740047e0 con 0x7f30840ff1e0 2023-12-14T18:25:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074005480 con 0x7f30840ff1e0 2023-12-14T18:25:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.052+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308410baf0 con 0x7f30840ff1e0 2023-12-14T18:25:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.053+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3084106850 con 0x7f30840ff1e0 2023-12-14T18:25:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.055+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f307400fb40 con 0x7f30840ff1e0 2023-12-14T18:25:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.056+0000 7f3080ff9700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f30700a1150 0x7f30700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.056+0000 7f3082ffd700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f30700a1150 0x7f30700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.056+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6809+0+0 (secure 0 0 0) 0x7f30740cdb00 con 0x7f30840ff1e0 2023-12-14T18:25:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.056+0000 7f3082ffd700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f30700a1150 0x7f30700a3600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f306c005950 tx=0x7f306c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.060+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3074097820 con 0x7f30840ff1e0 2023-12-14T18:25:18.089 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:25:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:25:17.776+0000 7f039d165640 -1 osd.11 0 log_to_monitors true 2023-12-14T18:25:18.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.271+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f308402cfe0 con 0x7f30840ff1e0 2023-12-14T18:25:18.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.271+0000 7f3080ff9700 1 -- 172.21.15.31:0/2789218629 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f307409ae40 con 0x7f30840ff1e0 2023-12-14T18:25:18.272 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f30700a1150 msgr2=0x7f30700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f30700a1150 0x7f30700a3600 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f306c005950 tx=0x7f306c00b410 comp rx=0 tx=0).stop 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 msgr2=0x7f3084109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f3084109a90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3074000c00 tx=0x7f3074009580 comp rx=0 tx=0).stop 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 shutdown_connections 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3084100d70 0x7f3084108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30840ff1e0 0x7f3084109a90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f30700a1150 0x7f30700a3600 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 --2- 172.21.15.31:0/2789218629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30841003e0 0x7f30841080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.273+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 >> 172.21.15.31:0/2789218629 conn(0x7f30840fa7f0 msgr2=0x7f3084103990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.274+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 shutdown_connections 2023-12-14T18:25:18.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:18.274+0000 7f3089a82700 1 -- 172.21.15.31:0/2789218629 wait complete. 2023-12-14T18:25:18.685 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1702578302,"num_in_osds":12,"osd_in_since":1702578304,"num_remapped_pgs":0} 2023-12-14T18:25:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-14T18:25:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:25:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:25:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2789218629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:25:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:18.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:18 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2789218629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:18 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2789218629' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:18 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:19.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd stat -f json 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: Detected new or changed devices on smithi204 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi204 to 4278M 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-14T18:25:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: osdmap e74: 12 total, 11 up, 12 in 2023-12-14T18:25:19.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:19.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: from='osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:25:19.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:19 smithi031 ceph-mon[42085]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: Detected new or changed devices on smithi204 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi204 to 4278M 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-14T18:25:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: osdmap e74: 12 total, 11 up, 12 in 2023-12-14T18:25:19.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:19.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: from='osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:25:19.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:19 smithi176 ceph-mon[43506]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:25:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:20.089 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:25:19.615+0000 7f03927dc700 -1 osd.11 0 waiting for initial osdmap 2023-12-14T18:25:20.089 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:25:19.632+0000 7f038e5d1700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:25:20.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: Detected new or changed devices on smithi204 2023-12-14T18:25:20.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi204 to 4278M 2023-12-14T18:25:20.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: osdmap e74: 12 total, 11 up, 12 in 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: from='osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:25:20.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:19 smithi204 ceph-mon[43866]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2023-12-14T18:25:20.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:20 smithi031 ceph-mon[42085]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:20.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:20 smithi031 ceph-mon[42085]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:25:20.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:20 smithi031 ceph-mon[42085]: osdmap e75: 12 total, 11 up, 12 in 2023-12-14T18:25:20.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:20.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:20 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:20 smithi176 ceph-mon[43506]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:20 smithi176 ceph-mon[43506]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:25:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:20 smithi176 ceph-mon[43506]: osdmap e75: 12 total, 11 up, 12 in 2023-12-14T18:25:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:20 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:20 smithi204 ceph-mon[43866]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:20 smithi204 ceph-mon[43866]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2023-12-14T18:25:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:20 smithi204 ceph-mon[43866]: osdmap e75: 12 total, 11 up, 12 in 2023-12-14T18:25:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:20 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 -- 172.21.15.31:0/1451093060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 msgr2=0x7fc118101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/1451093060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118101970 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009990 tx=0x7fc10c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 -- 172.21.15.31:0/1451093060 shutdown_connections 2023-12-14T18:25:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/1451093060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc118101eb0 0x7fc118108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/1451093060 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc118100320 0x7fc118100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/1451093060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118101970 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.200+0000 7fc11e1ca700 1 -- 172.21.15.31:0/1451093060 >> 172.21.15.31:0/1451093060 conn(0x7fc1180fb970 msgr2=0x7fc1180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.201+0000 7fc11e1ca700 1 -- 172.21.15.31:0/1451093060 shutdown_connections 2023-12-14T18:25:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.201+0000 7fc11e1ca700 1 -- 172.21.15.31:0/1451093060 wait complete. 2023-12-14T18:25:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.201+0000 7fc11e1ca700 1 Processor -- start 2023-12-14T18:25:21.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.201+0000 7fc11e1ca700 1 -- start start 2023-12-14T18:25:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc11e1ca700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc118100320 0x7fc118074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc11e1ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc11e1ca700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc118101eb0 0x7fc118073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc11e1ca700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118073ba0 con 0x7fc118101520 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc11e1ca700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118073d10 con 0x7fc118100320 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc11e1ca700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181a0080 con 0x7fc118101eb0 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59648/0 (socket says 172.21.15.31:59648) 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 -- 172.21.15.31:0/3441653395 learned_addr learned my addr 172.21.15.31:0/3441653395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc1177fe700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc118100320 0x7fc118074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc117fff700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc118101eb0 0x7fc118073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 -- 172.21.15.31:0/3441653395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc118101eb0 msgr2=0x7fc118073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc118101eb0 0x7fc118073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 -- 172.21.15.31:0/3441653395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc118100320 msgr2=0x7fc118074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc118100320 0x7fc118074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 -- 172.21.15.31:0/3441653395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc118101520 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.202+0000 7fc116ffd700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118072ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc10c000c00 tx=0x7fc10c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:21.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.203+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c01d070 con 0x7fc118101520 2023-12-14T18:25:21.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.203+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11810bc70 con 0x7fc118101520 2023-12-14T18:25:21.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.203+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc10c00fb50 con 0x7fc118101520 2023-12-14T18:25:21.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.203+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0173f0 con 0x7fc118101520 2023-12-14T18:25:21.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.203+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11810bae0 con 0x7fc118101520 2023-12-14T18:25:21.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.204+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc118107920 con 0x7fc118101520 2023-12-14T18:25:21.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.205+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc10c00fcc0 con 0x7fc118101520 2023-12-14T18:25:21.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.206+0000 7fc114ff9700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc1000a1150 0x7fc1000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.206+0000 7fc1177fe700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc1000a1150 0x7fc1000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.206+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc10c0cdb80 con 0x7fc118101520 2023-12-14T18:25:21.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.206+0000 7fc1177fe700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc1000a1150 0x7fc1000a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc118101380 tx=0x7fc10800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:21.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.209+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc10c097760 con 0x7fc118101520 2023-12-14T18:25:21.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.416+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc118061700 con 0x7fc118101520 2023-12-14T18:25:21.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.417+0000 7fc114ff9700 1 -- 172.21.15.31:0/3441653395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fc10c09ad80 con 0x7fc118101520 2023-12-14T18:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:21.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc1000a1150 msgr2=0x7fc1000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc1000a1150 0x7fc1000a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc118101380 tx=0x7fc10800b410 comp rx=0 tx=0).stop 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 msgr2=0x7fc118072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118072ee0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc10c000c00 tx=0x7fc10c00f710 comp rx=0 tx=0).stop 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 shutdown_connections 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc1000a1150 0x7fc1000a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc118101eb0 0x7fc118073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc118100320 0x7fc118074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 --2- 172.21.15.31:0/3441653395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc118101520 0x7fc118072ee0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.419+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 >> 172.21.15.31:0/3441653395 conn(0x7fc1180fb970 msgr2=0x7fc118104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.420+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 shutdown_connections 2023-12-14T18:25:21.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:21.420+0000 7fc11e1ca700 1 -- 172.21.15.31:0/3441653395 wait complete. 2023-12-14T18:25:21.694 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:21 smithi031 ceph-mon[42085]: purged_snaps scrub starts 2023-12-14T18:25:21.694 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:21 smithi031 ceph-mon[42085]: purged_snaps scrub ok 2023-12-14T18:25:21.694 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:21 smithi031 ceph-mon[42085]: osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] boot 2023-12-14T18:25:21.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:21 smithi031 ceph-mon[42085]: osdmap e76: 12 total, 12 up, 12 in 2023-12-14T18:25:21.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:21 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:21.695 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:21 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3441653395' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:25:21.801 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1702578320,"num_in_osds":12,"osd_in_since":1702578304,"num_remapped_pgs":0} 2023-12-14T18:25:21.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-14T18:25:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:21 smithi176 ceph-mon[43506]: purged_snaps scrub starts 2023-12-14T18:25:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:21 smithi176 ceph-mon[43506]: purged_snaps scrub ok 2023-12-14T18:25:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:21 smithi176 ceph-mon[43506]: osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] boot 2023-12-14T18:25:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:21 smithi176 ceph-mon[43506]: osdmap e76: 12 total, 12 up, 12 in 2023-12-14T18:25:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:21 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:21 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3441653395' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:25:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:21 smithi204 ceph-mon[43866]: purged_snaps scrub starts 2023-12-14T18:25:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:21 smithi204 ceph-mon[43866]: purged_snaps scrub ok 2023-12-14T18:25:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:21 smithi204 ceph-mon[43866]: osd.11 [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] boot 2023-12-14T18:25:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:21 smithi204 ceph-mon[43866]: osdmap e76: 12 total, 12 up, 12 in 2023-12-14T18:25:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:21 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:25:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:21 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3441653395' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:25:22.157 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:22 smithi031 ceph-mon[42085]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:22 smithi031 ceph-mon[42085]: osdmap e77: 12 total, 12 up, 12 in 2023-12-14T18:25:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:22 smithi176 ceph-mon[43506]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:22 smithi176 ceph-mon[43506]: osdmap e77: 12 total, 12 up, 12 in 2023-12-14T18:25:23.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:22 smithi204 ceph-mon[43866]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:25:23.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:22 smithi204 ceph-mon[43866]: osdmap e77: 12 total, 12 up, 12 in 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 -- 172.21.15.31:0/2794526156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1890101540 msgr2=0x7f1890101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 --2- 172.21.15.31:0/2794526156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1890101540 0x7f1890101910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1880009990 tx=0x7f1880009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 -- 172.21.15.31:0/2794526156 shutdown_connections 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 --2- 172.21.15.31:0/2794526156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f189010a890 0x7f189010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 --2- 172.21.15.31:0/2794526156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1890101540 0x7f1890101910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 --2- 172.21.15.31:0/2794526156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1890101e50 0x7f189010a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 -- 172.21.15.31:0/2794526156 >> 172.21.15.31:0/2794526156 conn(0x7f18900770a0 msgr2=0x7f18900774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 -- 172.21.15.31:0/2794526156 shutdown_connections 2023-12-14T18:25:23.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.196+0000 7f1894f5b700 1 -- 172.21.15.31:0/2794526156 wait complete. 2023-12-14T18:25:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 Processor -- start 2023-12-14T18:25:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 -- start start 2023-12-14T18:25:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1890101540 0x7f1890196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1890101e50 0x7f1890196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 0x7f189019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019b3e0 con 0x7f189010a890 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019b550 con 0x7f1890101540 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f1894f5b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019b710 con 0x7f1890101e50 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f188ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 0x7f189019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f188ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 0x7f189019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59672/0 (socket says 172.21.15.31:59672) 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f188ed9d700 1 -- 172.21.15.31:0/4242605294 learned_addr learned my addr 172.21.15.31:0/4242605294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.197+0000 7f188dd9b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1890101e50 0x7f1890196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188e59c700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1890101540 0x7f1890196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188ed9d700 1 -- 172.21.15.31:0/4242605294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1890101e50 msgr2=0x7f1890196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188ed9d700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1890101e50 0x7f1890196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188ed9d700 1 -- 172.21.15.31:0/4242605294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1890101540 msgr2=0x7f1890196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188ed9d700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1890101540 0x7f1890196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188ed9d700 1 -- 172.21.15.31:0/4242605294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18800095f0 con 0x7f189010a890 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f188ed9d700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 0x7f189019add0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1888007d70 tx=0x7f188800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800ac90 con 0x7f189010a890 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188800adf0 con 0x7f189010a890 2023-12-14T18:25:23.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18901103e0 con 0x7f189010a890 2023-12-14T18:25:23.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880115e0 con 0x7f189010a890 2023-12-14T18:25:23.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.198+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1890110250 con 0x7f189010a890 2023-12-14T18:25:23.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.199+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1890110570 con 0x7f189010a890 2023-12-14T18:25:23.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.201+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1888012420 con 0x7f189010a890 2023-12-14T18:25:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.201+0000 7f187f7fe700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.201+0000 7f188e59c700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.201+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f18880cca60 con 0x7f189010a890 2023-12-14T18:25:23.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.202+0000 7f188e59c700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f18780a1150 0x7f18780a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f188000b5c0 tx=0x7f1880005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.205+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1888096650 con 0x7f189010a890 2023-12-14T18:25:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.415+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1890066ea0 con 0x7f189010a890 2023-12-14T18:25:23.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.416+0000 7f187f7fe700 1 -- 172.21.15.31:0/4242605294 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18445 (secure 0 0 0) 0x7f1888099c70 con 0x7f189010a890 2023-12-14T18:25:23.418 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:23.418 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:21.618906+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f18780a1150 msgr2=0x7f18780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f18780a1150 0x7f18780a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f188000b5c0 tx=0x7f1880005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 msgr2=0x7f189019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 0x7f189019add0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1888007d70 tx=0x7f188800a340 comp rx=0 tx=0).stop 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 shutdown_connections 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f189010a890 0x7f189019add0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1890101540 0x7f1890196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f18780a1150 0x7f18780a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 --2- 172.21.15.31:0/4242605294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1890101e50 0x7f1890196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 >> 172.21.15.31:0/4242605294 conn(0x7f18900770a0 msgr2=0x7f18900fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 shutdown_connections 2023-12-14T18:25:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:23.419+0000 7f1894f5b700 1 -- 172.21.15.31:0/4242605294 wait complete. 2023-12-14T18:25:23.818 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-14T18:25:23.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-14T18:25:24.170 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:24.323 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:24 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4242605294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:24.323 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:24 smithi031 ceph-mon[42085]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:24 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4242605294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:24 smithi204 ceph-mon[43866]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:24 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4242605294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:24 smithi176 ceph-mon[43506]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 -- 172.21.15.31:0/1831567049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100540 msgr2=0x7fcf00100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/1831567049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100540 0x7fcf00100910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcef00099c0 tx=0x7fcef0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 -- 172.21.15.31:0/1831567049 shutdown_connections 2023-12-14T18:25:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/1831567049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf001055e0 0x7fcf0010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/1831567049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100540 0x7fcf00100910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/1831567049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf00100ee0 0x7fcf00104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 -- 172.21.15.31:0/1831567049 >> 172.21.15.31:0/1831567049 conn(0x7fcf000fbdc0 msgr2=0x7fcf000fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 -- 172.21.15.31:0/1831567049 shutdown_connections 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.193+0000 7fcf06b0d700 1 -- 172.21.15.31:0/1831567049 wait complete. 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 Processor -- start 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 -- start start 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf00100540 0x7fcf00072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 0x7fcf00073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf001055e0 0x7fcf000738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:25.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf001a48b0 con 0x7fcf00100ee0 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf001a4a20 con 0x7fcf00100540 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf06b0d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf001a4be0 con 0x7fcf001055e0 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 0x7fcf00073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 0x7fcf00073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45094/0 (socket says 172.21.15.31:45094) 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 -- 172.21.15.31:0/3490515608 learned_addr learned my addr 172.21.15.31:0/3490515608 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf048a9700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf00100540 0x7fcf00072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcf050aa700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf001055e0 0x7fcf000738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 -- 172.21.15.31:0/3490515608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf001055e0 msgr2=0x7fcf000738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf001055e0 0x7fcf000738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 -- 172.21.15.31:0/3490515608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf00100540 msgr2=0x7fcf00072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf00100540 0x7fcf00072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.194+0000 7fcefffff700 1 -- 172.21.15.31:0/3490515608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00095f0 con 0x7fcf00100ee0 2023-12-14T18:25:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.195+0000 7fcefffff700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 0x7fcf00073330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d980 tx=0x7fcee800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.195+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800f890 con 0x7fcf00100ee0 2023-12-14T18:25:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.195+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcee8010460 con 0x7fcf00100ee0 2023-12-14T18:25:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.195+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf001103f0 con 0x7fcf00100ee0 2023-12-14T18:25:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.195+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee800e610 con 0x7fcf00100ee0 2023-12-14T18:25:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.195+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00110260 con 0x7fcf00100ee0 2023-12-14T18:25:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.196+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf00110580 con 0x7fcf00100ee0 2023-12-14T18:25:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.198+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcee80105d0 con 0x7fcf00100ee0 2023-12-14T18:25:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.198+0000 7fcefdffb700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fceec0a1150 0x7fceec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.199+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcee80cbe10 con 0x7fcf00100ee0 2023-12-14T18:25:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.199+0000 7fcf048a9700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fceec0a1150 0x7fceec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:25.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.199+0000 7fcf048a9700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fceec0a1150 0x7fceec0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006010 tx=0x7fcef0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:25.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.202+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcee8095a00 con 0x7fcf00100ee0 2023-12-14T18:25:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.409+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf001a5650 con 0x7fcf00100ee0 2023-12-14T18:25:25.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.410+0000 7fcefdffb700 1 -- 172.21.15.31:0/3490515608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcee8020020 con 0x7fcf00100ee0 2023-12-14T18:25:25.411 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fceec0a1150 msgr2=0x7fceec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fceec0a1150 0x7fceec0a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006010 tx=0x7fcef0009f10 comp rx=0 tx=0).stop 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 msgr2=0x7fcf00073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 0x7fcf00073330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcee800d980 tx=0x7fcee800dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 shutdown_connections 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fceec0a1150 0x7fceec0a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf001055e0 0x7fcf000738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf00100540 0x7fcf00072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 --2- 172.21.15.31:0/3490515608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf00100ee0 0x7fcf00073330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:25.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 >> 172.21.15.31:0/3490515608 conn(0x7fcf000fbdc0 msgr2=0x7fcf000fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.412+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 shutdown_connections 2023-12-14T18:25:25.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:25.413+0000 7fcf06b0d700 1 -- 172.21.15.31:0/3490515608 wait complete. 2023-12-14T18:25:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:25 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3490515608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:25.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:25 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3490515608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:25.784 INFO:tasks.cephadm:Adding mds.a on smithi031 2023-12-14T18:25:25.784 INFO:tasks.cephadm:Adding mds.d on smithi031 2023-12-14T18:25:25.784 INFO:tasks.cephadm:Adding mds.g on smithi031 2023-12-14T18:25:25.784 INFO:tasks.cephadm:Adding mds.j on smithi031 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.b on smithi176 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.e on smithi176 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.h on smithi176 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.k on smithi176 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.c on smithi204 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.f on smithi204 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.i on smithi204 2023-12-14T18:25:25.785 INFO:tasks.cephadm:Adding mds.l on smithi204 2023-12-14T18:25:25.786 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph orch apply mds all '12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2023-12-14T18:25:25.820 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:25 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3490515608' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:26.108 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:25:26.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:26 smithi204 ceph-mon[43866]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:26.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:26 smithi176 ceph-mon[43506]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:26.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:26 smithi031 ceph-mon[42085]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:27.125 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 -- 172.21.15.204:0/1906648188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c105fb0 msgr2=0x7f3a3c108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:27.125 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 --2- 172.21.15.204:0/1906648188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c105fb0 0x7f3a3c108410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400b0a0 tx=0x7f3a3400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.125 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 -- 172.21.15.204:0/1906648188 shutdown_connections 2023-12-14T18:25:27.125 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 --2- 172.21.15.204:0/1906648188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c105fb0 0x7f3a3c108410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.126 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 --2- 172.21.15.204:0/1906648188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a3c100d70 0x7f3a3c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.126 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 --2- 172.21.15.204:0/1906648188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a3c103690 0x7f3a3c105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.126 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 -- 172.21.15.204:0/1906648188 >> 172.21.15.204:0/1906648188 conn(0x7f3a3c0fa7f0 msgr2=0x7f3a3c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:27.126 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 -- 172.21.15.204:0/1906648188 shutdown_connections 2023-12-14T18:25:27.126 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.123+0000 7f3a439da700 1 -- 172.21.15.204:0/1906648188 wait complete. 2023-12-14T18:25:27.128 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 Processor -- start 2023-12-14T18:25:27.128 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 -- start start 2023-12-14T18:25:27.128 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 0x7f3a3c1960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:27.128 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a3c103690 0x7f3a3c196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:27.128 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a3c105fb0 0x7f3a3c19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c19dc60 con 0x7f3a3c105fb0 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c19dda0 con 0x7f3a3c103690 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a439da700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c19df30 con 0x7f3a3c100d70 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a41776700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 0x7f3a3c1960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a41776700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 0x7f3a3c1960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:45846/0 (socket says 172.21.15.204:45846) 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.124+0000 7f3a41776700 1 -- 172.21.15.204:0/2350759612 learned_addr learned my addr 172.21.15.204:0/2350759612 (peer_addr_for_me v2:172.21.15.204:0/0) 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a40f75700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a3c103690 0x7f3a3c196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41f77700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a3c105fb0 0x7f3a3c19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41776700 1 -- 172.21.15.204:0/2350759612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a3c103690 msgr2=0x7f3a3c196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41776700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a3c103690 0x7f3a3c196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41776700 1 -- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a3c105fb0 msgr2=0x7f3a3c19d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:27.129 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41776700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a3c105fb0 0x7f3a3c19d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41776700 1 -- 172.21.15.204:0/2350759612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34009d00 con 0x7f3a3c100d70 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a41776700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 0x7f3a3c1960d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800d9b0 tx=0x7f3a3800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38004220 con 0x7f3a3c100d70 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a38007640 con 0x7f3a3c100d70 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c10bcb0 con 0x7f3a3c100d70 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38005590 con 0x7f3a3c100d70 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.125+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c10bb20 con 0x7f3a3c100d70 2023-12-14T18:25:27.130 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.126+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c0fc360 con 0x7f3a3c100d70 2023-12-14T18:25:27.131 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.127+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3a38004750 con 0x7f3a3c100d70 2023-12-14T18:25:27.131 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.128+0000 7f3a327fc700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a280a1150 0x7f3a280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:27.131 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.128+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a380cc7f0 con 0x7f3a3c100d70 2023-12-14T18:25:27.131 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.128+0000 7f3a40f75700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a280a1150 0x7f3a280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:27.131 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.129+0000 7f3a40f75700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a280a1150 0x7f3a280a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00abd0 tx=0x7f3a2c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:27.133 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.131+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a38096340 con 0x7f3a3c100d70 2023-12-14T18:25:27.331 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.330+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a3c061c80 con 0x7f3a280a1150 2023-12-14T18:25:27.340 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.338+0000 7f3a327fc700 1 -- 172.21.15.204:0/2350759612 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f3a3c061c80 con 0x7f3a280a1150 2023-12-14T18:25:27.340 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2023-12-14T18:25:27.342 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a280a1150 msgr2=0x7f3a280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:27.342 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a280a1150 0x7f3a280a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00abd0 tx=0x7f3a2c00a380 comp rx=0 tx=0).stop 2023-12-14T18:25:27.342 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 msgr2=0x7f3a3c1960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:27.342 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 0x7f3a3c1960d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800d9b0 tx=0x7f3a3800dcc0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.342 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 shutdown_connections 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a3c105fb0 0x7f3a3c19d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a280a1150 0x7f3a280a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a3c100d70 0x7f3a3c1960d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 --2- 172.21.15.204:0/2350759612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a3c103690 0x7f3a3c196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 >> 172.21.15.204:0/2350759612 conn(0x7f3a3c0fa7f0 msgr2=0x7f3a3c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 shutdown_connections 2023-12-14T18:25:27.343 INFO:teuthology.orchestra.run.smithi204.stderr:2023-12-14T18:25:27.340+0000 7f3a439da700 1 -- 172.21.15.204:0/2350759612 wait complete. 2023-12-14T18:25:27.709 DEBUG:teuthology.orchestra.run.smithi031:mds.a> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.a.service 2023-12-14T18:25:27.712 DEBUG:teuthology.orchestra.run.smithi031:mds.d> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.d.service 2023-12-14T18:25:27.745 INFO:journalctl@ceph.mds.a.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:25:27.747 DEBUG:teuthology.orchestra.run.smithi031:mds.g> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.g.service 2023-12-14T18:25:27.776 INFO:journalctl@ceph.mds.d.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:25:27.777 DEBUG:teuthology.orchestra.run.smithi031:mds.j> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.j.service 2023-12-14T18:25:27.807 INFO:journalctl@ceph.mds.g.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:25:27.808 DEBUG:teuthology.orchestra.run.smithi176:mds.b> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.b.service 2023-12-14T18:25:27.811 DEBUG:teuthology.orchestra.run.smithi176:mds.e> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.e.service 2023-12-14T18:25:27.839 INFO:journalctl@ceph.mds.j.smithi031.stdout:-- Logs begin at Thu 2023-12-14 18:04:35 UTC. -- 2023-12-14T18:25:27.849 INFO:journalctl@ceph.mds.b.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:25:27.850 DEBUG:teuthology.orchestra.run.smithi176:mds.h> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.h.service 2023-12-14T18:25:27.880 INFO:journalctl@ceph.mds.e.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:25:27.881 DEBUG:teuthology.orchestra.run.smithi176:mds.k> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.k.service 2023-12-14T18:25:27.909 INFO:journalctl@ceph.mds.h.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:25:27.910 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.c.service 2023-12-14T18:25:27.913 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.f.service 2023-12-14T18:25:27.937 INFO:journalctl@ceph.mds.k.smithi176.stdout:-- Logs begin at Thu 2023-12-14 18:04:49 UTC. -- 2023-12-14T18:25:27.944 INFO:journalctl@ceph.mds.c.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:25:27.945 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.i.service 2023-12-14T18:25:27.972 INFO:journalctl@ceph.mds.f.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:25:27.973 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.l.service 2023-12-14T18:25:27.999 INFO:journalctl@ceph.mds.i.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:25:28.001 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-14T18:25:28.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:25:28.028 INFO:journalctl@ceph.mds.l.smithi204.stdout:-- Logs begin at Thu 2023-12-14 18:05:57 UTC. -- 2023-12-14T18:25:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.158+0000 7f3511125700 1 -- 172.21.15.31:0/4238981390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c100300 msgr2=0x7f350c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.158+0000 7f3511125700 1 --2- 172.21.15.31:0/4238981390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c100300 0x7f350c100750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3500009990 tx=0x7f3500009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 -- 172.21.15.31:0/4238981390 shutdown_connections 2023-12-14T18:25:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 --2- 172.21.15.31:0/4238981390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100c90 0x7f350c107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 --2- 172.21.15.31:0/4238981390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff110 0x7f350c0ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 --2- 172.21.15.31:0/4238981390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c100300 0x7f350c100750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 -- 172.21.15.31:0/4238981390 >> 172.21.15.31:0/4238981390 conn(0x7f350c0fa6f0 msgr2=0x7f350c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 -- 172.21.15.31:0/4238981390 shutdown_connections 2023-12-14T18:25:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.159+0000 7f3511125700 1 -- 172.21.15.31:0/4238981390 wait complete. 2023-12-14T18:25:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 Processor -- start 2023-12-14T18:25:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 -- start start 2023-12-14T18:25:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 0x7f350c195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100300 0x7f350c196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c100c90 0x7f350c19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19db40 con 0x7f350c0ff110 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19dc80 con 0x7f350c100300 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f3511125700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19de10 con 0x7f350c100c90 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.160+0000 7f350ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 0x7f350c195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 0x7f350c195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45106/0 (socket says 172.21.15.31:45106) 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 -- 172.21.15.31:0/1413132850 learned_addr learned my addr 172.21.15.31:0/1413132850 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350b59e700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c100c90 0x7f350c19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350a59c700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100300 0x7f350c196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 -- 172.21.15.31:0/1413132850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c100c90 msgr2=0x7f350c19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c100c90 0x7f350c19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 -- 172.21.15.31:0/1413132850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100300 msgr2=0x7f350c196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100300 0x7f350c196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 -- 172.21.15.31:0/1413132850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000095f0 con 0x7f350c0ff110 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350a59c700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100300 0x7f350c196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f350ad9d700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 0x7f350c195fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00e970 tx=0x7f34fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc009790 con 0x7f350c0ff110 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc00eec0 con 0x7f350c0ff110 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c10bca0 con 0x7f350c0ff110 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc003cf0 con 0x7f350c0ff110 2023-12-14T18:25:28.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.161+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c10bb10 con 0x7f350c0ff110 2023-12-14T18:25:28.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.162+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c0679f0 con 0x7f350c0ff110 2023-12-14T18:25:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.164+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f34fc00ccd0 con 0x7f350c0ff110 2023-12-14T18:25:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.165+0000 7f34f3fff700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34f40a1150 0x7f34f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.165+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34fc014070 con 0x7f350c0ff110 2023-12-14T18:25:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.165+0000 7f350a59c700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34f40a1150 0x7f34f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.165+0000 7f350a59c700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34f40a1150 0x7f34f40a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3500000c00 tx=0x7f3500005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.169+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f34fc0963f0 con 0x7f350c0ff110 2023-12-14T18:25:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.382+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f350c105f70 con 0x7f350c0ff110 2023-12-14T18:25:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.383+0000 7f34f3fff700 1 -- 172.21.15.31:0/1413132850 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18445 (secure 0 0 0) 0x7f34fc099a10 con 0x7f350c0ff110 2023-12-14T18:25:28.384 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:28.384 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:21.618906+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34f40a1150 msgr2=0x7f34f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34f40a1150 0x7f34f40a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3500000c00 tx=0x7f3500005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 msgr2=0x7f350c195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 0x7f350c195fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00e970 tx=0x7f34fc00ec80 comp rx=0 tx=0).stop 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 shutdown_connections 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34f40a1150 0x7f34f40a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c100c90 0x7f350c19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff110 0x7f350c195fc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.385+0000 7f3511125700 1 --2- 172.21.15.31:0/1413132850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c100300 0x7f350c196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.386+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 >> 172.21.15.31:0/1413132850 conn(0x7f350c0fa6f0 msgr2=0x7f350c0fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.386+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 shutdown_connections 2023-12-14T18:25:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.386+0000 7f3511125700 1 -- 172.21.15.31:0/1413132850 wait complete. 2023-12-14T18:25:28.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-14T18:25:28.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:25:28.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/414259775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40ff160 msgr2=0x7fa3d40ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/414259775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40ff160 0x7fa3d40ff570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009990 tx=0x7fa3d0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/414259775 shutdown_connections 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/414259775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d4100ce0 0x7fa3d41075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/414259775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d40ff160 0x7fa3d40ff570 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/414259775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4100350 0x7fa3d41007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/414259775 >> 172.21.15.31:0/414259775 conn(0x7fa3d40fa780 msgr2=0x7fa3d40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/414259775 shutdown_connections 2023-12-14T18:25:28.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.541+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/414259775 wait complete. 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 Processor -- start 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 -- start start 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d40ff160 0x7fa3d4195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4100350 0x7fa3d41964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 0x7fa3d419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419daf0 con 0x7fa3d4100ce0 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419dc30 con 0x7fa3d4100350 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3dbbdf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d419ddc0 con 0x7fa3d40ff160 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3da17c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 0x7fa3d419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3da17c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 0x7fa3d419d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45124/0 (socket says 172.21.15.31:45124) 2023-12-14T18:25:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.542+0000 7fa3da17c700 1 -- 172.21.15.31:0/2159749321 learned_addr learned my addr 172.21.15.31:0/2159749321 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3d917a700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4100350 0x7fa3d41964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3d997b700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d40ff160 0x7fa3d4195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3da17c700 1 -- 172.21.15.31:0/2159749321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d40ff160 msgr2=0x7fa3d4195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3da17c700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d40ff160 0x7fa3d4195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3da17c700 1 -- 172.21.15.31:0/2159749321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4100350 msgr2=0x7fa3d41964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3da17c700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4100350 0x7fa3d41964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3da17c700 1 -- 172.21.15.31:0/2159749321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00095f0 con 0x7fa3d4100ce0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3da17c700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 0x7fa3d419d520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc007ee0 tx=0x7fa3cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc017070 con 0x7fa3d4100ce0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d410aad0 con 0x7fa3d4100ce0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3cc00f040 con 0x7fa3d4100ce0 2023-12-14T18:25:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc005580 con 0x7fa3d4100ce0 2023-12-14T18:25:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.543+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d410a940 con 0x7fa3d4100ce0 2023-12-14T18:25:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.544+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d419e4b0 con 0x7fa3d4100ce0 2023-12-14T18:25:28.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.545+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa3cc00db20 con 0x7fa3d4100ce0 2023-12-14T18:25:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.546+0000 7fa3c6ffd700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3c00a11d0 0x7fa3c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.546+0000 7fa3d997b700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3c00a11d0 0x7fa3c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.546+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3cc0cc250 con 0x7fa3d4100ce0 2023-12-14T18:25:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.547+0000 7fa3d997b700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3c00a11d0 0x7fa3c00a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000b5c0 tx=0x7fa3d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.550+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa3cc095da0 con 0x7fa3d4100ce0 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:28.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:28.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:28 smithi031 ceph-mon[42085]: Deploying daemon mds.l on smithi204 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:28.720 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:28.721 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:28 smithi204 ceph-mon[43866]: Deploying daemon mds.l on smithi204 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: Saving service mds.all spec with placement smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:28.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:28.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:28.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:28.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:28 smithi176 ceph-mon[43506]: Deploying daemon mds.l on smithi204 2023-12-14T18:25:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.761+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3d419eb50 con 0x7fa3d4100ce0 2023-12-14T18:25:28.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.761+0000 7fa3c6ffd700 1 -- 172.21.15.31:0/2159749321 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3cc0993c0 con 0x7fa3d4100ce0 2023-12-14T18:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:25:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3c00a11d0 msgr2=0x7fa3c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3c00a11d0 0x7fa3c00a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000b5c0 tx=0x7fa3d0005960 comp rx=0 tx=0).stop 2023-12-14T18:25:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 msgr2=0x7fa3d419d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 0x7fa3d419d520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc007ee0 tx=0x7fa3cc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 shutdown_connections 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3d4100ce0 0x7fa3d419d520 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3c00a11d0 0x7fa3c00a3680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d40ff160 0x7fa3d4195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 --2- 172.21.15.31:0/2159749321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3d4100350 0x7fa3d41964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 >> 172.21.15.31:0/2159749321 conn(0x7fa3d40fa780 msgr2=0x7fa3d4068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 shutdown_connections 2023-12-14T18:25:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.764+0000 7fa3dbbdf700 1 -- 172.21.15.31:0/2159749321 wait complete. 2023-12-14T18:25:28.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:28.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 -- 172.21.15.31:0/3148711010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 msgr2=0x7fd34c0ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 --2- 172.21.15.31:0/3148711010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c0ff520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd340009990 tx=0x7fd340009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 -- 172.21.15.31:0/3148711010 shutdown_connections 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 --2- 172.21.15.31:0/3148711010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd34c100c90 0x7fd34c107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 --2- 172.21.15.31:0/3148711010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c0ff520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 --2- 172.21.15.31:0/3148711010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd34c100300 0x7fd34c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 -- 172.21.15.31:0/3148711010 >> 172.21.15.31:0/3148711010 conn(0x7fd34c0fa6f0 msgr2=0x7fd34c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 -- 172.21.15.31:0/3148711010 shutdown_connections 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.914+0000 7fd351a85700 1 -- 172.21.15.31:0/3148711010 wait complete. 2023-12-14T18:25:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 Processor -- start 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 -- start start 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd34c100300 0x7fd34c1016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd34c100c90 0x7fd34c101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c1023c0 con 0x7fd34c0ff110 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c102530 con 0x7fd34c100c90 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd351a85700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c1a00d0 con 0x7fd34c100300 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd34b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd34b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45138/0 (socket says 172.21.15.31:45138) 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd34b7fe700 1 -- 172.21.15.31:0/354884371 learned_addr learned my addr 172.21.15.31:0/354884371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.915+0000 7fd34bfff700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd34c100c90 0x7fd34c101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd34b7fe700 1 -- 172.21.15.31:0/354884371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd34c100300 msgr2=0x7fd34c1016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd34b7fe700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd34c100300 0x7fd34c1016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd34b7fe700 1 -- 172.21.15.31:0/354884371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd34c100c90 msgr2=0x7fd34c101c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd34b7fe700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd34c100c90 0x7fd34c101c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd34b7fe700 1 -- 172.21.15.31:0/354884371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400095f0 con 0x7fd34c0ff110 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd34b7fe700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c1030a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd340004760 tx=0x7fd340009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34001d070 con 0x7fd34c0ff110 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c10ab50 con 0x7fd34c0ff110 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd340004cd0 con 0x7fd34c0ff110 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340005290 con 0x7fd34c0ff110 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.916+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c10a9c0 con 0x7fd34c0ff110 2023-12-14T18:25:28.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.917+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c1a06d0 con 0x7fd34c0ff110 2023-12-14T18:25:28.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.918+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd340003b60 con 0x7fd34c0ff110 2023-12-14T18:25:28.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.919+0000 7fd348ff9700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd3340a1150 0x7fd3340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.919+0000 7fd34affd700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd3340a1150 0x7fd3340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.919+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd3400cd810 con 0x7fd34c0ff110 2023-12-14T18:25:28.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.919+0000 7fd34affd700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd3340a1150 0x7fd3340a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005d90 tx=0x7fd33c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:28.922+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd340097490 con 0x7fd34c0ff110 2023-12-14T18:25:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.178+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd34c1a0db0 con 0x7fd34c0ff110 2023-12-14T18:25:29.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.178+0000 7fd348ff9700 1 -- 172.21.15.31:0/354884371 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fd34009aab0 con 0x7fd34c0ff110 2023-12-14T18:25:29.179 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:29.179 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-14T18:25:29.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.180+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd3340a1150 msgr2=0x7fd3340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:29.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.180+0000 7fd351a85700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd3340a1150 0x7fd3340a3600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd33c005d90 tx=0x7fd33c005d00 comp rx=0 tx=0).stop 2023-12-14T18:25:29.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.180+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 msgr2=0x7fd34c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:29.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.180+0000 7fd351a85700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c1030a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd340004760 tx=0x7fd340009580 comp rx=0 tx=0).stop 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 shutdown_connections 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd34c100c90 0x7fd34c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0ff110 0x7fd34c1030a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd3340a1150 0x7fd3340a3600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 --2- 172.21.15.31:0/354884371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd34c100300 0x7fd34c1016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 >> 172.21.15.31:0/354884371 conn(0x7fd34c0fa6f0 msgr2=0x7fd34c067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 shutdown_connections 2023-12-14T18:25:29.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:29.181+0000 7fd351a85700 1 -- 172.21.15.31:0/354884371 wait complete. 2023-12-14T18:25:29.184 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 1 2023-12-14T18:25:29.339 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:25:29 smithi204 systemd[1]: Starting Ceph mds.l for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1413132850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2159749321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:29.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/354884371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:29.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1413132850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:29.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2159749321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:29.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/354884371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1413132850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2159749321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/354884371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:29.840 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:25:29 smithi204 ceph-mds[68859]: -- 172.21.15.204:0/1230406835 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5559ac00d520 con 0x5559abfe3000 2023-12-14T18:25:29.840 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:25:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-l[68836]: starting mds.l at 2023-12-14T18:25:29.840 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:25:29 smithi204 bash[68821]: 545ab783cb56d9b4c01ade16c4f93acc61ee0c79e56983cbcbb1711a01551362 2023-12-14T18:25:29.840 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:25:29 smithi204 systemd[1]: Started Ceph mds.l for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:30.729 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:30.729 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6832/1411039043,v1:172.21.15.204:6833/1411039043] up:boot 2023-12-14T18:25:30.729 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: fsmap 1 up:standby 2023-12-14T18:25:30.729 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:25:30.729 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.729 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.730 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.730 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:30.730 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:30.730 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:30.730 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:30 smithi176 ceph-mon[43506]: Deploying daemon mds.k on smithi176 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6832/1411039043,v1:172.21.15.204:6833/1411039043] up:boot 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: fsmap 1 up:standby 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:30 smithi204 ceph-mon[43866]: Deploying daemon mds.k on smithi176 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6832/1411039043,v1:172.21.15.204:6833/1411039043] up:boot 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: fsmap 1 up:standby 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:30 smithi031 ceph-mon[42085]: Deploying daemon mds.k on smithi176 2023-12-14T18:25:30.992 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:25:30 smithi176 systemd[1]: Starting Ceph mds.k for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:31.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/2557333217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 msgr2=0x7f8d98101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/2557333217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d98101870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009990 tx=0x7f8d94009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/2557333217 shutdown_connections 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/2557333217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d98101db0 0x7f8d98108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/2557333217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d98100230 0x7f8d98100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/2557333217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d98101870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/2557333217 >> 172.21.15.31:0/2557333217 conn(0x7f8d980fb810 msgr2=0x7f8d980fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/2557333217 shutdown_connections 2023-12-14T18:25:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.346+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/2557333217 wait complete. 2023-12-14T18:25:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 Processor -- start 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 -- start start 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d98100230 0x7f8d98195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d981964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d98101db0 0x7f8d9819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819db10 con 0x7f8d98101420 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819dc50 con 0x7f8d98101db0 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9fb3f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819dde0 con 0x7f8d98100230 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9d0da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d981964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9d0da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d981964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45166/0 (socket says 172.21.15.31:45166) 2023-12-14T18:25:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.347+0000 7f8d9d0da700 1 -- 172.21.15.31:0/1841926562 learned_addr learned my addr 172.21.15.31:0/1841926562 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d8db700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d98100230 0x7f8d98195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9e0dc700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d98101db0 0x7f8d9819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d0da700 1 -- 172.21.15.31:0/1841926562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d98100230 msgr2=0x7f8d98195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d0da700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d98100230 0x7f8d98195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d0da700 1 -- 172.21.15.31:0/1841926562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d98101db0 msgr2=0x7f8d9819d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d0da700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d98101db0 0x7f8d9819d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d0da700 1 -- 172.21.15.31:0/1841926562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d940095f0 con 0x7f8d98101420 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9d0da700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d981964d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d94004760 tx=0x7f8d94004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9401d070 con 0x7f8d98101420 2023-12-14T18:25:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d94004cf0 con 0x7f8d98101420 2023-12-14T18:25:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9810bb90 con 0x7f8d98101420 2023-12-14T18:25:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003d00 con 0x7f8d98101420 2023-12-14T18:25:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.348+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9810ba00 con 0x7f8d98101420 2023-12-14T18:25:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.349+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9819e410 con 0x7f8d98101420 2023-12-14T18:25:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.351+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8d94005000 con 0x7f8d98101420 2023-12-14T18:25:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.351+0000 7f8d8effd700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8d840a1150 0x7f8d840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.352+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d940cd910 con 0x7f8d98101420 2023-12-14T18:25:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.352+0000 7f8d9d8db700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8d840a1150 0x7f8d840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.352+0000 7f8d9d8db700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8d840a1150 0x7f8d840a3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d98101280 tx=0x7f8d8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.355+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8d94097590 con 0x7f8d98101420 2023-12-14T18:25:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.618+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d98107160 con 0x7f8d98101420 2023-12-14T18:25:31.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.619+0000 7f8d8effd700 1 -- 172.21.15.31:0/1841926562 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f8d94095e60 con 0x7f8d98101420 2023-12-14T18:25:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:31.620 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-14T18:25:31.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8d840a1150 msgr2=0x7f8d840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:31.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8d840a1150 0x7f8d840a3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d98101280 tx=0x7f8d8800b410 comp rx=0 tx=0).stop 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 msgr2=0x7f8d981964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d981964d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d94004760 tx=0x7f8d94004790 comp rx=0 tx=0).stop 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 shutdown_connections 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8d98101db0 0x7f8d9819d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8d840a1150 0x7f8d840a3600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d98100230 0x7f8d98195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 --2- 172.21.15.31:0/1841926562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d98101420 0x7f8d981964d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 >> 172.21.15.31:0/1841926562 conn(0x7f8d980fb810 msgr2=0x7f8d980fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 shutdown_connections 2023-12-14T18:25:31.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:31.621+0000 7f8d9fb3f700 1 -- 172.21.15.31:0/1841926562 wait complete. 2023-12-14T18:25:31.624 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2023-12-14T18:25:31.742 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:25:31 smithi176 ceph-mds[67668]: -- 172.21.15.176:0/1448158854 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d785119520 con 0x55d7850eec00 2023-12-14T18:25:31.742 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:25:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-k[67645]: starting mds.k at 2023-12-14T18:25:31.742 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:25:31 smithi176 bash[67630]: 352dcf74b7d75d01160dde1aba76a28852eda9bb3ad42e9938620db7c562b7fc 2023-12-14T18:25:31.742 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:25:31 smithi176 systemd[1]: Started Ceph mds.k for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:32.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: Deploying daemon mds.j on smithi031 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:32.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6832/2002803706,v1:172.21.15.176:6833/2002803706] up:boot 2023-12-14T18:25:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: fsmap 2 up:standby 2023-12-14T18:25:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:25:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:32 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1841926562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:32.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: Deploying daemon mds.j on smithi031 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6832/2002803706,v1:172.21.15.176:6833/2002803706] up:boot 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: fsmap 2 up:standby 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:25:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:32 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1841926562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: Deploying daemon mds.j on smithi031 2023-12-14T18:25:32.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:32.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6832/2002803706,v1:172.21.15.176:6833/2002803706] up:boot 2023-12-14T18:25:32.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: fsmap 2 up:standby 2023-12-14T18:25:32.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:25:32.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:32 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1841926562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:32.913 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:25:32 smithi031 systemd[1]: Starting Ceph mds.j for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:33.464 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:25:33 smithi031 bash[75260]: 444fb1c55d97b42b6b4410638bdc25bb5d658b8cd04a53383cfe7d06a2cc032d 2023-12-14T18:25:33.464 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:25:33 smithi031 ceph-mds[75298]: -- 172.21.15.31:0/3811266477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632723d1520 con 0x5632723a7000 2023-12-14T18:25:33.464 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:25:33 smithi031 systemd[1]: Started Ceph mds.j for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:33.464 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:25:33 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-j[75275]: starting mds.j at 2023-12-14T18:25:33.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:33.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 -- 172.21.15.31:0/3114420996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8105f30 msgr2=0x7f4de8108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 --2- 172.21.15.31:0/3114420996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8105f30 0x7f4de8108330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800b0a0 tx=0x7f4dd800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 -- 172.21.15.31:0/3114420996 shutdown_connections 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 --2- 172.21.15.31:0/3114420996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8105f30 0x7f4de8108330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 --2- 172.21.15.31:0/3114420996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de8100cf0 0x7f4de81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 --2- 172.21.15.31:0/3114420996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8103610 0x7f4de81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 -- 172.21.15.31:0/3114420996 >> 172.21.15.31:0/3114420996 conn(0x7f4de80fa6f0 msgr2=0x7f4de80fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 -- 172.21.15.31:0/3114420996 shutdown_connections 2023-12-14T18:25:33.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.785+0000 7f4df0437700 1 -- 172.21.15.31:0/3114420996 wait complete. 2023-12-14T18:25:33.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 Processor -- start 2023-12-14T18:25:33.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 -- start start 2023-12-14T18:25:33.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 0x7f4de8196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8103610 0x7f4de81965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de8105f30 0x7f4de819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de819dbf0 con 0x7f4de8100cf0 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de819dd30 con 0x7f4de8103610 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.786+0000 7f4df0437700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de819dec0 con 0x7f4de8105f30 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 0x7f4de8196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 0x7f4de8196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45206/0 (socket says 172.21.15.31:45206) 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 -- 172.21.15.31:0/94212545 learned_addr learned my addr 172.21.15.31:0/94212545 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4ded9d2700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8103610 0x7f4de81965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee9d4700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de8105f30 0x7f4de819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 -- 172.21.15.31:0/94212545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de8105f30 msgr2=0x7f4de819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de8105f30 0x7f4de819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 -- 172.21.15.31:0/94212545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8103610 msgr2=0x7f4de81965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8103610 0x7f4de81965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 -- 172.21.15.31:0/94212545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd8009d00 con 0x7f4de8100cf0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4dee1d3700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 0x7f4de8196070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4de000e9a0 tx=0x7f4de000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.787+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0009790 con 0x7f4de8100cf0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.788+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de0010430 con 0x7f4de8100cf0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.788+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de810bbd0 con 0x7f4de8100cf0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.788+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0018680 con 0x7f4de8100cf0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.788+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de810ba40 con 0x7f4de8100cf0 2023-12-14T18:25:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.789+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de80fc260 con 0x7f4de8100cf0 2023-12-14T18:25:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.791+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4de00105a0 con 0x7f4de8100cf0 2023-12-14T18:25:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.792+0000 7f4de77fe700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4dd00a1150 0x7f4dd00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.792+0000 7f4ded9d2700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4dd00a1150 0x7f4dd00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.792+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4de0014070 con 0x7f4de8100cf0 2023-12-14T18:25:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.793+0000 7f4ded9d2700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4dd00a1150 0x7f4dd00a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005950 tx=0x7f4ddc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:33.795+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4de0096410 con 0x7f4de8100cf0 2023-12-14T18:25:34.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.060+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4de802cfe0 con 0x7f4de8100cf0 2023-12-14T18:25:34.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.061+0000 7f4de77fe700 1 -- 172.21.15.31:0/94212545 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2771 (secure 0 0 0) 0x7f4de0026af0 con 0x7f4de8100cf0 2023-12-14T18:25:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:34.063 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4dd00a1150 msgr2=0x7f4dd00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4dd00a1150 0x7f4dd00a3600 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc005950 tx=0x7f4ddc00b410 comp rx=0 tx=0).stop 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 msgr2=0x7f4de8196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 0x7f4de8196070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4de000e9a0 tx=0x7f4de000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 shutdown_connections 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f4dd00a1150 0x7f4dd00a3600 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4de8105f30 0x7f4de819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4de8100cf0 0x7f4de8196070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 --2- 172.21.15.31:0/94212545 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4de8103610 0x7f4de81965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 >> 172.21.15.31:0/94212545 conn(0x7f4de80fa6f0 msgr2=0x7f4de8101950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 shutdown_connections 2023-12-14T18:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:34.063+0000 7f4df0437700 1 -- 172.21.15.31:0/94212545 wait complete. 2023-12-14T18:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 4 2023-12-14T18:25:34.354 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: Deploying daemon mds.d on smithi031 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.31:6834/28036797,v1:172.21.15.31:6835/28036797] up:boot 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: fsmap 3 up:standby 2023-12-14T18:25:34.355 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:25:34.356 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/94212545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:34.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: Deploying daemon mds.d on smithi031 2023-12-14T18:25:34.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:34.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.31:6834/28036797,v1:172.21.15.31:6835/28036797] up:boot 2023-12-14T18:25:34.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: fsmap 3 up:standby 2023-12-14T18:25:34.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:25:34.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:34 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/94212545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: Deploying daemon mds.d on smithi031 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.31:6834/28036797,v1:172.21.15.31:6835/28036797] up:boot 2023-12-14T18:25:34.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: fsmap 3 up:standby 2023-12-14T18:25:34.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:25:34.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:34 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/94212545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:34.714 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:25:34 smithi031 systemd[1]: Starting Ceph mds.d for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:35.214 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-mds[75684]: -- 172.21.15.31:0/722175328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eae597d520 con 0x55eae5953000 2023-12-14T18:25:35.214 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:25:34 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d[75661]: starting mds.d at 2023-12-14T18:25:35.214 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:25:34 smithi031 bash[75646]: 3823d38038dcc72dd14ed49413cfcb4e663a30ce0cbb1ce69317e8f7acbcefbc 2023-12-14T18:25:35.214 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:25:34 smithi031 systemd[1]: Started Ceph mds.d for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:36.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:36.108 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.109 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.109 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.109 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:36.109 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:36.109 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:36.110 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.31:6836/3783760943,v1:172.21.15.31:6837/3783760943] up:boot 2023-12-14T18:25:36.110 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: fsmap 4 up:standby 2023-12-14T18:25:36.110 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:35 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.231+0000 7f5cb6277700 1 -- 172.21.15.31:0/1870523564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb01035b0 msgr2=0x7f5cb0105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.231+0000 7f5cb6277700 1 --2- 172.21.15.31:0/1870523564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb01035b0 0x7f5cb0105990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009990 tx=0x7f5ca4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 -- 172.21.15.31:0/1870523564 shutdown_connections 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 --2- 172.21.15.31:0/1870523564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cb0105ed0 0x7f5cb0108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 --2- 172.21.15.31:0/1870523564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0100c90 0x7f5cb0103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 --2- 172.21.15.31:0/1870523564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb01035b0 0x7f5cb0105990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 -- 172.21.15.31:0/1870523564 >> 172.21.15.31:0/1870523564 conn(0x7f5cb00fa6f0 msgr2=0x7f5cb00fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 -- 172.21.15.31:0/1870523564 shutdown_connections 2023-12-14T18:25:36.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 -- 172.21.15.31:0/1870523564 wait complete. 2023-12-14T18:25:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 Processor -- start 2023-12-14T18:25:36.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.232+0000 7f5cb6277700 1 -- start start 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb6277700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 0x7f5cb0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb6277700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cb01035b0 0x7f5cb0196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb6277700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0105ed0 0x7f5cb019d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb6277700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb019dab0 con 0x7f5cb0100c90 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb6277700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb019dbf0 con 0x7f5cb01035b0 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb6277700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb019dd80 con 0x7f5cb0105ed0 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 0x7f5cb0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 0x7f5cb0195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55160/0 (socket says 172.21.15.31:55160) 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 -- 172.21.15.31:0/3063928026 learned_addr learned my addr 172.21.15.31:0/3063928026 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caf7fe700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cb01035b0 0x7f5cb0196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5cb4814700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0105ed0 0x7f5cb019d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 -- 172.21.15.31:0/3063928026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0105ed0 msgr2=0x7f5cb019d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0105ed0 0x7f5cb019d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 -- 172.21.15.31:0/3063928026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cb01035b0 msgr2=0x7f5cb0196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cb01035b0 0x7f5cb0196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.233+0000 7f5caffff700 1 -- 172.21.15.31:0/3063928026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40095f0 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.234+0000 7f5caffff700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 0x7f5cb0195f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000e9a0 tx=0x7f5ca000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.234+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0009790 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.234+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb010bbd0 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.234+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ca000cbd0 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.234+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0018680 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.234+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb010ba40 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.235+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb00679f0 con 0x7f5cb0100c90 2023-12-14T18:25:36.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.237+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ca000cd40 con 0x7f5cb0100c90 2023-12-14T18:25:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.237+0000 7f5cad7fa700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5c980a1150 0x7f5c980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.237+0000 7f5caf7fe700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5c980a1150 0x7f5c980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.237+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5ca0014070 con 0x7f5cb0100c90 2023-12-14T18:25:36.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.238+0000 7f5caf7fe700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5c980a1150 0x7f5c980a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b2e0 tx=0x7f5ca4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:36.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.241+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ca0096590 con 0x7f5cb0100c90 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.31:6836/3783760943,v1:172.21.15.31:6837/3783760943] up:boot 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: fsmap 4 up:standby 2023-12-14T18:25:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:35 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:25:36.437 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:25:36 smithi176 systemd[1]: Starting Ceph mds.e for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:36.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.31:6836/3783760943,v1:172.21.15.31:6837/3783760943] up:boot 2023-12-14T18:25:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: fsmap 4 up:standby 2023-12-14T18:25:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:35 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:25:36.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.496+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cb019e890 con 0x7f5cb0100c90 2023-12-14T18:25:36.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.496+0000 7f5cad7fa700 1 -- 172.21.15.31:0/3063928026 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7f5ca0026de0 con 0x7f5cb0100c90 2023-12-14T18:25:36.500 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:36.500 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2023-12-14T18:25:36.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5c980a1150 msgr2=0x7f5c980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5c980a1150 0x7f5c980a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b2e0 tx=0x7f5ca4005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 msgr2=0x7f5cb0195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 0x7f5cb0195f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000e9a0 tx=0x7f5ca000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 shutdown_connections 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5c980a1150 0x7f5c980a3600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cb0105ed0 0x7f5cb019d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5cb0100c90 0x7f5cb0195f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 --2- 172.21.15.31:0/3063928026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5cb01035b0 0x7f5cb0196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 >> 172.21.15.31:0/3063928026 conn(0x7f5cb00fa6f0 msgr2=0x7f5cb0102100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.498+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 shutdown_connections 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:36.499+0000 7f5cb6277700 1 -- 172.21.15.31:0/3063928026 wait complete. 2023-12-14T18:25:36.502 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 5 2023-12-14T18:25:36.977 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mds[68009]: -- 172.21.15.176:0/3664399238 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563e9e841520 con 0x563e9e816c00 2023-12-14T18:25:36.977 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-e[67986]: starting mds.e at 2023-12-14T18:25:36.977 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:25:36 smithi176 bash[67972]: bba9e0b45c36c5947f9de520521f99cb915ae99b497a15b356a80821ef81fae7 2023-12-14T18:25:36.978 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:25:36 smithi176 systemd[1]: Started Ceph mds.e for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: Deploying daemon mds.e on smithi176 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3063928026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:37.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:36 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:37.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: Deploying daemon mds.e on smithi176 2023-12-14T18:25:37.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:37.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3063928026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:37.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.322 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.322 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.322 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:37.322 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:37.322 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:36 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:37.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: Deploying daemon mds.e on smithi176 2023-12-14T18:25:37.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:37.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3063928026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:37.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:37.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:37.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:37.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:36 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:37.949 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:25:37 smithi204 systemd[1]: Starting Ceph mds.f for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:38.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:37 smithi204 ceph-mon[43866]: Deploying daemon mds.f on smithi204 2023-12-14T18:25:38.211 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:37 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6834/1697669779,v1:172.21.15.176:6835/1697669779] up:boot 2023-12-14T18:25:38.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:37 smithi204 ceph-mon[43866]: fsmap 5 up:standby 2023-12-14T18:25:38.212 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:37 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:25:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:37 smithi176 ceph-mon[43506]: Deploying daemon mds.f on smithi204 2023-12-14T18:25:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:37 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6834/1697669779,v1:172.21.15.176:6835/1697669779] up:boot 2023-12-14T18:25:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:37 smithi176 ceph-mon[43506]: fsmap 5 up:standby 2023-12-14T18:25:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:37 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:25:38.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:37 smithi031 ceph-mon[42085]: Deploying daemon mds.f on smithi204 2023-12-14T18:25:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:37 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6834/1697669779,v1:172.21.15.176:6835/1697669779] up:boot 2023-12-14T18:25:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:37 smithi031 ceph-mon[42085]: fsmap 5 up:standby 2023-12-14T18:25:38.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:37 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:25:38.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:38.589 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mds[69201]: -- 172.21.15.204:0/1275817777 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558108e13520 con 0x558108de8c00 2023-12-14T18:25:38.589 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f[69178]: starting mds.f at 2023-12-14T18:25:38.589 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:25:38 smithi204 bash[69163]: 59206bc6f397f3fc3522459db551a4bba4c3208ae0ea53ffd7568fec11be70ea 2023-12-14T18:25:38.589 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:25:38 smithi204 systemd[1]: Started Ceph mds.f for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/1800267667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24103610 msgr2=0x7fbd241059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/1800267667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24103610 0x7fbd241059f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009990 tx=0x7fbd14009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/1800267667 shutdown_connections 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/1800267667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd24105f30 0x7fbd24108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/1800267667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd24100cf0 0x7fbd241030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/1800267667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24103610 0x7fbd241059f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/1800267667 >> 172.21.15.31:0/1800267667 conn(0x7fbd240fa6f0 msgr2=0x7fbd240fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/1800267667 shutdown_connections 2023-12-14T18:25:38.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.657+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/1800267667 wait complete. 2023-12-14T18:25:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 Processor -- start 2023-12-14T18:25:38.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 -- start start 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 0x7fbd24195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd24103610 0x7fbd241964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd24105f30 0x7fbd2419d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419daf0 con 0x7fbd24100cf0 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419dc30 con 0x7fbd24105f30 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd2b2f6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2419ddc0 con 0x7fbd24103610 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 0x7fbd24195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 0x7fbd24195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55180/0 (socket says 172.21.15.31:55180) 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 -- 172.21.15.31:0/106569188 learned_addr learned my addr 172.21.15.31:0/106569188 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29893700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd24105f30 0x7fbd2419d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd28891700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd24103610 0x7fbd241964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 -- 172.21.15.31:0/106569188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd24103610 msgr2=0x7fbd241964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd24103610 0x7fbd241964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 -- 172.21.15.31:0/106569188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd24105f30 msgr2=0x7fbd2419d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd24105f30 0x7fbd2419d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.658+0000 7fbd29092700 1 -- 172.21.15.31:0/106569188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd140095f0 con 0x7fbd24100cf0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.659+0000 7fbd29092700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 0x7fbd24195f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000d980 tx=0x7fbd2000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.659+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20004220 con 0x7fbd24100cf0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.659+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd20007640 con 0x7fbd24100cf0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.659+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2410bbd0 con 0x7fbd24100cf0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.659+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20005690 con 0x7fbd24100cf0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.659+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2410ba40 con 0x7fbd24100cf0 2023-12-14T18:25:38.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.660+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd240679f0 con 0x7fbd24100cf0 2023-12-14T18:25:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.661+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbd20004750 con 0x7fbd24100cf0 2023-12-14T18:25:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.662+0000 7fbd1a7fc700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbd100a1150 0x7fbd100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.662+0000 7fbd28891700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbd100a1150 0x7fbd100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.662+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbd200cc7d0 con 0x7fbd24100cf0 2023-12-14T18:25:38.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.662+0000 7fbd28891700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbd100a1150 0x7fbd100a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400b5c0 tx=0x7fbd14005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:38.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.665+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd200963c0 con 0x7fbd24100cf0 2023-12-14T18:25:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.919+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd2419e8d0 con 0x7fbd24100cf0 2023-12-14T18:25:38.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.919+0000 7fbd1a7fc700 1 -- 172.21.15.31:0/106569188 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4288 (secure 0 0 0) 0x7fbd200999e0 con 0x7fbd24100cf0 2023-12-14T18:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2023-12-14T18:25:38.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbd100a1150 msgr2=0x7fbd100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbd100a1150 0x7fbd100a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400b5c0 tx=0x7fbd14005fd0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 msgr2=0x7fbd24195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 0x7fbd24195f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000d980 tx=0x7fbd2000dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:38.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 shutdown_connections 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbd24105f30 0x7fbd2419d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd24100cf0 0x7fbd24195f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbd100a1150 0x7fbd100a3600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 --2- 172.21.15.31:0/106569188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd24103610 0x7fbd241964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 >> 172.21.15.31:0/106569188 conn(0x7fbd240fa6f0 msgr2=0x7fbd24102150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 shutdown_connections 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:38.922+0000 7fbd2b2f6700 1 -- 172.21.15.31:0/106569188 wait complete. 2023-12-14T18:25:38.925 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 6 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:39.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/106569188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:39.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:38 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/106569188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:39.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:38 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/106569188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:39.982 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:25:39 smithi204 systemd[1]: Starting Ceph mds.c for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:39 smithi176 ceph-mon[43506]: Deploying daemon mds.c on smithi204 2023-12-14T18:25:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:39 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6834/3595337412,v1:172.21.15.204:6835/3595337412] up:boot 2023-12-14T18:25:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:39 smithi176 ceph-mon[43506]: fsmap 6 up:standby 2023-12-14T18:25:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:39 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:39 smithi204 ceph-mon[43866]: Deploying daemon mds.c on smithi204 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:39 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6834/3595337412,v1:172.21.15.204:6835/3595337412] up:boot 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:39 smithi204 ceph-mon[43866]: fsmap 6 up:standby 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:39 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mds[69543]: -- 172.21.15.204:0/400457980 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be42647520 con 0x55be4261c800 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c[69520]: starting mds.c at 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:25:40 smithi204 bash[69505]: 314c41ce30d18d334688fbc43b1f44031ece626ac0cfde12aeeff479abbb5b41 2023-12-14T18:25:40.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:25:40 smithi204 systemd[1]: Started Ceph mds.c for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:39 smithi031 ceph-mon[42085]: Deploying daemon mds.c on smithi204 2023-12-14T18:25:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:39 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6834/3595337412,v1:172.21.15.204:6835/3595337412] up:boot 2023-12-14T18:25:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:39 smithi031 ceph-mon[42085]: fsmap 6 up:standby 2023-12-14T18:25:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:39 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:25:40.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:41.071 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:40 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 -- 172.21.15.31:0/3363067580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100300 msgr2=0x7fd7a0100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3363067580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100300 0x7fd7a0100750 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009990 tx=0x7fd78c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 -- 172.21.15.31:0/3363067580 shutdown_connections 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3363067580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100c90 0x7fd7a0107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3363067580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a00ff110 0x7fd7a00ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3363067580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100300 0x7fd7a0100750 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 -- 172.21.15.31:0/3363067580 >> 172.21.15.31:0/3363067580 conn(0x7fd7a00fa6f0 msgr2=0x7fd7a00fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 -- 172.21.15.31:0/3363067580 shutdown_connections 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.085+0000 7fd7a7165700 1 -- 172.21.15.31:0/3363067580 wait complete. 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 Processor -- start 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 -- start start 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a00ff110 0x7fd7a01030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100300 0x7fd7a01016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 0x7fd7a0101c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a01023c0 con 0x7fd7a0100c90 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0102530 con 0x7fd7a0100300 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a7165700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a01a0050 con 0x7fd7a00ff110 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 0x7fd7a0101c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 0x7fd7a0101c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55194/0 (socket says 172.21.15.31:55194) 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 -- 172.21.15.31:0/3941751000 learned_addr learned my addr 172.21.15.31:0/3941751000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd79ffff700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100300 0x7fd7a01016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a4f01700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a00ff110 0x7fd7a01030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 -- 172.21.15.31:0/3941751000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a00ff110 msgr2=0x7fd7a01030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a00ff110 0x7fd7a01030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 -- 172.21.15.31:0/3941751000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100300 msgr2=0x7fd7a01016e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100300 0x7fd7a01016e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd7a5702700 1 -- 172.21.15.31:0/3941751000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0095f0 con 0x7fd7a0100c90 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.086+0000 7fd79ffff700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100300 0x7fd7a01016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.087+0000 7fd7a5702700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 0x7fd7a0101c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd79800ef30 tx=0x7fd79800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.087+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798018070 con 0x7fd7a0100c90 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.087+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a010aac0 con 0x7fd7a0100c90 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.087+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd79800f040 con 0x7fd7a0100c90 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.087+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79800cdb0 con 0x7fd7a0100c90 2023-12-14T18:25:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.087+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a010a930 con 0x7fd7a0100c90 2023-12-14T18:25:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.088+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a01a06e0 con 0x7fd7a0100c90 2023-12-14T18:25:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.089+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd798003a00 con 0x7fd7a0100c90 2023-12-14T18:25:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.090+0000 7fd79dffb700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd7900a1150 0x7fd7900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.090+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7980cc0a0 con 0x7fd7a0100c90 2023-12-14T18:25:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.090+0000 7fd7a4f01700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd7900a1150 0x7fd7900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:41.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.090+0000 7fd7a4f01700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd7900a1150 0x7fd7900a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd794005950 tx=0x7fd7940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:41.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.094+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd798095d20 con 0x7fd7a0100c90 2023-12-14T18:25:41.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:41.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:41.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:41.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:40 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:41.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.357+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7a01a0c90 con 0x7fd7a0100c90 2023-12-14T18:25:41.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.358+0000 7fd79dffb700 1 -- 172.21.15.31:0/3941751000 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5808 (secure 0 0 0) 0x7fd798099340 con 0x7fd7a0100c90 2023-12-14T18:25:41.360 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:41.360 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-12-14T18:25:41.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd7900a1150 msgr2=0x7fd7900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd7900a1150 0x7fd7900a3600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd794005950 tx=0x7fd7940058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 msgr2=0x7fd7a0101c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 0x7fd7a0101c20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd79800ef30 tx=0x7fd79800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 shutdown_connections 2023-12-14T18:25:41.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7a0100c90 0x7fd7a0101c20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd7900a1150 0x7fd7900a3600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a00ff110 0x7fd7a01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 --2- 172.21.15.31:0/3941751000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd7a0100300 0x7fd7a01016e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 >> 172.21.15.31:0/3941751000 conn(0x7fd7a00fa6f0 msgr2=0x7fd7a0067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:41.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 shutdown_connections 2023-12-14T18:25:41.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:41.360+0000 7fd7a7165700 1 -- 172.21.15.31:0/3941751000 wait complete. 2023-12-14T18:25:41.364 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-12-14T18:25:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:41.375 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:41.376 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:40 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:41.742 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:25:41 smithi176 systemd[1]: Starting Ceph mds.h for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: Deploying daemon mds.h on smithi176 2023-12-14T18:25:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] up:boot 2023-12-14T18:25:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: fsmap 7 up:standby 2023-12-14T18:25:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:25:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3941751000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:42.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:42.243 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-mds[68353]: -- 172.21.15.176:0/532995099 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d6698d9520 con 0x55d6698af000 2023-12-14T18:25:42.243 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:25:41 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h[68329]: starting mds.h at 2023-12-14T18:25:42.243 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:25:41 smithi176 bash[68314]: 7836a5e6387e321d62372ad2dc41b90430d890a6c728bff35ea2f761293dd9c7 2023-12-14T18:25:42.243 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:25:41 smithi176 systemd[1]: Started Ceph mds.h for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: Deploying daemon mds.h on smithi176 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] up:boot 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: fsmap 7 up:standby 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3941751000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:41 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: Deploying daemon mds.h on smithi176 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] up:boot 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: fsmap 7 up:standby 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3941751000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:41 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:43.195 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6836/2569700606,v1:172.21.15.176:6837/2569700606] up:boot 2023-12-14T18:25:43.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: fsmap 8 up:standby 2023-12-14T18:25:43.196 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:42 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6836/2569700606,v1:172.21.15.176:6837/2569700606] up:boot 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: fsmap 8 up:standby 2023-12-14T18:25:43.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:42 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:25:43.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6836/2569700606,v1:172.21.15.176:6837/2569700606] up:boot 2023-12-14T18:25:43.409 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: fsmap 8 up:standby 2023-12-14T18:25:43.410 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:42 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:25:43.518 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:25:43 smithi204 systemd[1]: Starting Ceph mds.i for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:43.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 -- 172.21.15.31:0/1896693703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 msgr2=0x7f26800ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:43.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 --2- 172.21.15.31:0/1896693703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f26800ff520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2670009990 tx=0x7f2670009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 -- 172.21.15.31:0/1896693703 shutdown_connections 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 --2- 172.21.15.31:0/1896693703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2680100c90 0x7f2680107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 --2- 172.21.15.31:0/1896693703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f26800ff520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 --2- 172.21.15.31:0/1896693703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2680100300 0x7f2680100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 -- 172.21.15.31:0/1896693703 >> 172.21.15.31:0/1896693703 conn(0x7f26800fa6f0 msgr2=0x7f26800fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 -- 172.21.15.31:0/1896693703 shutdown_connections 2023-12-14T18:25:43.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.521+0000 7f2684b73700 1 -- 172.21.15.31:0/1896693703 wait complete. 2023-12-14T18:25:43.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 Processor -- start 2023-12-14T18:25:43.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 -- start start 2023-12-14T18:25:43.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f2680196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2680100300 0x7f2680196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2680100c90 0x7f268019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268019db90 con 0x7f26800ff110 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268019dcd0 con 0x7f2680100c90 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.522+0000 7f2684b73700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268019de60 con 0x7f2680100300 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f2680196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f2680196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55218/0 (socket says 172.21.15.31:55218) 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 -- 172.21.15.31:0/3901068885 learned_addr learned my addr 172.21.15.31:0/3901068885 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f2675d9b700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2680100300 0x7f2680196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267ed9d700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2680100c90 0x7f268019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 -- 172.21.15.31:0/3901068885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2680100300 msgr2=0x7f2680196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2680100300 0x7f2680196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 -- 172.21.15.31:0/3901068885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2680100c90 msgr2=0x7f268019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2680100c90 0x7f268019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 -- 172.21.15.31:0/3901068885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700095f0 con 0x7f26800ff110 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f267e59c700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f2680196010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2670004760 tx=0x7f2670004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267001d070 con 0x7f26800ff110 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2670021410 con 0x7f26800ff110 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268010bbe0 con 0x7f26800ff110 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670017650 con 0x7f26800ff110 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.523+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268010ba50 con 0x7f26800ff110 2023-12-14T18:25:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.524+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2680106720 con 0x7f26800ff110 2023-12-14T18:25:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.526+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2670021580 con 0x7f26800ff110 2023-12-14T18:25:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.526+0000 7f2677fff700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f266c0a1150 0x7f266c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.526+0000 7f2675d9b700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f266c0a1150 0x7f266c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.526+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26700cdcb0 con 0x7f26800ff110 2023-12-14T18:25:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.527+0000 7f2675d9b700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f266c0a1150 0x7f266c0a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2668009c60 tx=0x7f2668009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:43.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.530+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2670097930 con 0x7f26800ff110 2023-12-14T18:25:43.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.784+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f268002cfe0 con 0x7f26800ff110 2023-12-14T18:25:43.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.785+0000 7f2677fff700 1 -- 172.21.15.31:0/3901068885 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6568 (secure 0 0 0) 0x7f267009af50 con 0x7f26800ff110 2023-12-14T18:25:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:43.787 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f266c0a1150 msgr2=0x7f266c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f266c0a1150 0x7f266c0a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2668009c60 tx=0x7f2668009260 comp rx=0 tx=0).stop 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 msgr2=0x7f2680196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f2680196010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2670004760 tx=0x7f2670004790 comp rx=0 tx=0).stop 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 shutdown_connections 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2680100c90 0x7f268019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26800ff110 0x7f2680196010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f266c0a1150 0x7f266c0a3600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 --2- 172.21.15.31:0/3901068885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2680100300 0x7f2680196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.787+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 >> 172.21.15.31:0/3901068885 conn(0x7f26800fa6f0 msgr2=0x7f2680103860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.788+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 shutdown_connections 2023-12-14T18:25:43.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:43.788+0000 7f2684b73700 1 -- 172.21.15.31:0/3901068885 wait complete. 2023-12-14T18:25:43.791 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 9 2023-12-14T18:25:43.792 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mds[69884]: -- 172.21.15.204:0/20164870 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556ac7e33520 con 0x556ac7e09000 2023-12-14T18:25:43.792 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: starting mds.i at 2023-12-14T18:25:44.089 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:25:43 smithi204 bash[69846]: 14669656c1c179056a315d3401b82cb17c37cd18837ac3ca2b10a90931bf62f7 2023-12-14T18:25:44.089 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:25:43 smithi204 systemd[1]: Started Ceph mds.i for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:44.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: Deploying daemon mds.i on smithi204 2023-12-14T18:25:44.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3901068885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:44.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:44.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:44.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:43 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: Deploying daemon mds.i on smithi204 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3901068885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:43 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: Deploying daemon mds.i on smithi204 2023-12-14T18:25:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3901068885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:44.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:43 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:45.141 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:44 smithi031 ceph-mon[42085]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:45.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:44 smithi031 ceph-mon[42085]: Deploying daemon mds.g on smithi031 2023-12-14T18:25:45.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:44 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] up:boot 2023-12-14T18:25:45.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:44 smithi031 ceph-mon[42085]: fsmap 9 up:standby 2023-12-14T18:25:45.142 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:44 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:25:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:44 smithi176 ceph-mon[43506]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:44 smithi176 ceph-mon[43506]: Deploying daemon mds.g on smithi031 2023-12-14T18:25:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:44 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] up:boot 2023-12-14T18:25:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:44 smithi176 ceph-mon[43506]: fsmap 9 up:standby 2023-12-14T18:25:45.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:44 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:25:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:44 smithi204 ceph-mon[43866]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:44 smithi204 ceph-mon[43866]: Deploying daemon mds.g on smithi031 2023-12-14T18:25:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:44 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] up:boot 2023-12-14T18:25:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:44 smithi204 ceph-mon[43866]: fsmap 9 up:standby 2023-12-14T18:25:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:44 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:25:45.464 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:25:45 smithi031 systemd[1]: Starting Ceph mds.g for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:45.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 -- 172.21.15.31:0/938902605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac0734d0 msgr2=0x7fa9ac073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 --2- 172.21.15.31:0/938902605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac073940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009990 tx=0x7fa9a0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 -- 172.21.15.31:0/938902605 shutdown_connections 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 --2- 172.21.15.31:0/938902605 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac108100 0x7fa9ac10a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 --2- 172.21.15.31:0/938902605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 --2- 172.21.15.31:0/938902605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac073940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 -- 172.21.15.31:0/938902605 >> 172.21.15.31:0/938902605 conn(0x7fa9ac0fb830 msgr2=0x7fa9ac0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 -- 172.21.15.31:0/938902605 shutdown_connections 2023-12-14T18:25:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.950+0000 7fa9b1965700 1 -- 172.21.15.31:0/938902605 wait complete. 2023-12-14T18:25:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.951+0000 7fa9b1965700 1 Processor -- start 2023-12-14T18:25:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 -- start start 2023-12-14T18:25:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ac108100 0x7fa9ac19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac19fd10 con 0x7fa9ac074ab0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac19fe50 con 0x7fa9ac0734d0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9b1965700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac19ffe0 con 0x7fa9ac108100 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58636/0 (socket says 172.21.15.31:58636) 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 -- 172.21.15.31:0/2301122711 learned_addr learned my addr 172.21.15.31:0/2301122711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aaffd700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9ab7fe700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ac108100 0x7fa9ac19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 -- 172.21.15.31:0/2301122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ac108100 msgr2=0x7fa9ac19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ac108100 0x7fa9ac19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 -- 172.21.15.31:0/2301122711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac0734d0 msgr2=0x7fa9ac198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aa7fc700 1 -- 172.21.15.31:0/2301122711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00095f0 con 0x7fa9ac074ab0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.952+0000 7fa9aaffd700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.953+0000 7fa9aa7fc700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac1986d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0004730 tx=0x7fa9a0004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.953+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a001d070 con 0x7fa9ac074ab0 2023-12-14T18:25:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.953+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a0004c90 con 0x7fa9ac074ab0 2023-12-14T18:25:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.953+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac10ddf0 con 0x7fa9ac074ab0 2023-12-14T18:25:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.953+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00054b0 con 0x7fa9ac074ab0 2023-12-14T18:25:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.953+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac10dc60 con 0x7fa9ac074ab0 2023-12-14T18:25:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.954+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac106840 con 0x7fa9ac074ab0 2023-12-14T18:25:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.957+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa9a000fbc0 con 0x7fa9ac074ab0 2023-12-14T18:25:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.958+0000 7fa9b0963700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9940a11d0 0x7fa9940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:45.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.958+0000 7fa9aaffd700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9940a11d0 0x7fa9940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.958+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa9a00cdab0 con 0x7fa9ac074ab0 2023-12-14T18:25:45.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.958+0000 7fa9aaffd700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9940a11d0 0x7fa9940a3680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa99c005950 tx=0x7fa99c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:45.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:45.960+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa9a0097730 con 0x7fa9ac074ab0 2023-12-14T18:25:45.963 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:25:45 smithi031 ceph-mds[76213]: -- 172.21.15.31:0/2367391908 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557f26d37520 con 0x557f26d0c800 2023-12-14T18:25:45.964 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:25:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-g[76190]: starting mds.g at 2023-12-14T18:25:45.964 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:25:45 smithi031 bash[76175]: 1f7b22ca15dbf2ace0bb410bae985c842bb83357ef62510e15236760a66d00d4 2023-12-14T18:25:45.964 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:25:45 smithi031 systemd[1]: Started Ceph mds.g for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:46.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.235+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9ac02cfe0 con 0x7fa9ac074ab0 2023-12-14T18:25:46.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.236+0000 7fa9b0963700 1 -- 172.21.15.31:0/2301122711 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8088 (secure 0 0 0) 0x7fa9a009ad50 con 0x7fa9ac074ab0 2023-12-14T18:25:46.238 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:46.238 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.238+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9940a11d0 msgr2=0x7fa9940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.238+0000 7fa9b1965700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9940a11d0 0x7fa9940a3680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa99c005950 tx=0x7fa99c00b410 comp rx=0 tx=0).stop 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.238+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 msgr2=0x7fa9ac1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.238+0000 7fa9b1965700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac1986d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0004730 tx=0x7fa9a0004760 comp rx=0 tx=0).stop 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 shutdown_connections 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9940a11d0 0x7fa9940a3680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9ac108100 0x7fa9ac19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9ac0734d0 0x7fa9ac198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 --2- 172.21.15.31:0/2301122711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac074ab0 0x7fa9ac1986d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:46.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 >> 172.21.15.31:0/2301122711 conn(0x7fa9ac0fb830 msgr2=0x7fa9ac0fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 shutdown_connections 2023-12-14T18:25:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:46.239+0000 7fa9b1965700 1 -- 172.21.15.31:0/2301122711 wait complete. 2023-12-14T18:25:46.242 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 11 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: Deploying daemon mds.a on smithi031 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.31:6838/2398776190,v1:172.21.15.31:6839/2398776190] up:boot 2023-12-14T18:25:46.807 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: fsmap 10 up:standby 2023-12-14T18:25:46.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:25:46.808 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:46 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2301122711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: Deploying daemon mds.a on smithi031 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.31:6838/2398776190,v1:172.21.15.31:6839/2398776190] up:boot 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: fsmap 10 up:standby 2023-12-14T18:25:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:25:46.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:46 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2301122711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:47.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:47.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: Deploying daemon mds.a on smithi031 2023-12-14T18:25:47.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.31:6838/2398776190,v1:172.21.15.31:6839/2398776190] up:boot 2023-12-14T18:25:47.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: fsmap 10 up:standby 2023-12-14T18:25:47.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:25:47.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:46 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2301122711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:47.964 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:25:47 smithi031 systemd[1]: Started Ceph mds.a for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:47.964 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:25:47 smithi031 ceph-mds[76602]: -- 172.21.15.31:0/3532564767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5614aeced520 con 0x5614aecc2c00 2023-12-14T18:25:47.964 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:25:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-a[76579]: starting mds.a at 2023-12-14T18:25:48.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1436519730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0ffd80 msgr2=0x7f78cc1001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1436519730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0ffd80 0x7f78cc1001f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b0a0 tx=0x7f78c400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1436519730 shutdown_connections 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1436519730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0ffd80 0x7f78cc1001f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1436519730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc0fea80 0x7f78cc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1436519730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78cc0ff3d0 0x7f78cc0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1436519730 >> 172.21.15.31:0/1436519730 conn(0x7f78cc0fa6f0 msgr2=0x7f78cc0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1436519730 shutdown_connections 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1436519730 wait complete. 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.407+0000 7f78d3bf7700 1 Processor -- start 2023-12-14T18:25:48.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 -- start start 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 0x7f78cc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc0ff3d0 0x7f78cc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78cc0ffd80 0x7f78cc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc19fdb0 con 0x7f78cc0fea80 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc19fef0 con 0x7f78cc0ffd80 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d3bf7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1a0080 con 0x7f78cc0ff3d0 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 0x7f78cc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 0x7f78cc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58678/0 (socket says 172.21.15.31:58678) 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 -- 172.21.15.31:0/1343732577 learned_addr learned my addr 172.21.15.31:0/1343732577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1192700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc0ff3d0 0x7f78cc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d2194700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78cc0ffd80 0x7f78cc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 -- 172.21.15.31:0/1343732577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc0ff3d0 msgr2=0x7f78cc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc0ff3d0 0x7f78cc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 -- 172.21.15.31:0/1343732577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78cc0ffd80 msgr2=0x7f78cc19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78cc0ffd80 0x7f78cc19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 -- 172.21.15.31:0/1343732577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c4009d00 con 0x7f78cc0fea80 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78d1993700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 0x7f78cc198230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d950 tx=0x7f78c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.408+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800f890 con 0x7f78cc0fea80 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.409+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78c8010460 con 0x7f78cc0fea80 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.409+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc10dd60 con 0x7f78cc0fea80 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.409+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800e610 con 0x7f78cc0fea80 2023-12-14T18:25:48.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.409+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc10dbd0 con 0x7f78cc0fea80 2023-12-14T18:25:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.410+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc109860 con 0x7f78cc0fea80 2023-12-14T18:25:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.411+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f78c80108d0 con 0x7f78cc0fea80 2023-12-14T18:25:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.412+0000 7f78beffd700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f78b80a11d0 0x7f78b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.412+0000 7f78d1192700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f78b80a11d0 0x7f78b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.412+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f78c80cc1b0 con 0x7f78cc0fea80 2023-12-14T18:25:48.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.412+0000 7f78d1192700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f78b80a11d0 0x7f78b80a3680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78c0005950 tx=0x7f78c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:48.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.417+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78c8095da0 con 0x7f78cc0fea80 2023-12-14T18:25:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.680+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78cc061940 con 0x7f78cc0fea80 2023-12-14T18:25:48.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.680+0000 7f78beffd700 1 -- 172.21.15.31:0/1343732577 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8846 (secure 0 0 0) 0x7f78c80993c0 con 0x7f78cc0fea80 2023-12-14T18:25:48.683 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:48.683 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-14T18:25:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f78b80a11d0 msgr2=0x7f78b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f78b80a11d0 0x7f78b80a3680 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78c0005950 tx=0x7f78c000b410 comp rx=0 tx=0).stop 2023-12-14T18:25:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 msgr2=0x7f78cc198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 0x7f78cc198230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78c800d950 tx=0x7f78c800dc60 comp rx=0 tx=0).stop 2023-12-14T18:25:48.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 shutdown_connections 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f78cc0ffd80 0x7f78cc19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78cc0fea80 0x7f78cc198230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f78b80a11d0 0x7f78b80a3680 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 --2- 172.21.15.31:0/1343732577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc0ff3d0 0x7f78cc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 >> 172.21.15.31:0/1343732577 conn(0x7f78cc0fa6f0 msgr2=0x7f78cc0fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 shutdown_connections 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:48.683+0000 7f78d3bf7700 1 -- 172.21.15.31:0/1343732577 wait complete. 2023-12-14T18:25:48.686 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 12 2023-12-14T18:25:48.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:48.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:48.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:48.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:48.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:48.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:48.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:48.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: Deploying daemon mds.b on smithi176 2023-12-14T18:25:48.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.31:6840/3456971189,v1:172.21.15.31:6841/3456971189] up:boot 2023-12-14T18:25:48.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: fsmap 11 up:standby 2023-12-14T18:25:48.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: Deploying daemon mds.b on smithi176 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.31:6840/3456971189,v1:172.21.15.31:6841/3456971189] up:boot 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: fsmap 11 up:standby 2023-12-14T18:25:48.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:25:49.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:49.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: Deploying daemon mds.b on smithi176 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.31:6840/3456971189,v1:172.21.15.31:6841/3456971189] up:boot 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: fsmap 11 up:standby 2023-12-14T18:25:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:25:49.151 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:25:48 smithi176 systemd[1]: Starting Ceph mds.b for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:25:49.644 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mds[68698]: -- 172.21.15.176:0/2790019354 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d96e3ed520 con 0x55d96e3c2800 2023-12-14T18:25:49.644 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b[68675]: starting mds.b at 2023-12-14T18:25:49.644 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:25:49 smithi176 bash[68660]: d46cb1843272f62a75ab82988936152a53cc708d47b98419e28194e61f145e45 2023-12-14T18:25:49.644 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:25:49 smithi176 systemd[1]: Started Ceph mds.b for 37e25396-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1343732577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:49 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1343732577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:49 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1343732577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:50.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:49 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:50.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 -- 172.21.15.31:0/3639040299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fe290 msgr2=0x7f50880fe700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 --2- 172.21.15.31:0/3639040299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fe290 0x7f50880fe700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f507c0098f0 tx=0x7f507c009c00 comp rx=0 tx=0).stop 2023-12-14T18:25:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 -- 172.21.15.31:0/3639040299 shutdown_connections 2023-12-14T18:25:50.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 --2- 172.21.15.31:0/3639040299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fec40 0x7f50880ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 --2- 172.21.15.31:0/3639040299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f5088100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 --2- 172.21.15.31:0/3639040299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fe290 0x7f50880fe700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:50.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.868+0000 7f508e688700 1 -- 172.21.15.31:0/3639040299 >> 172.21.15.31:0/3639040299 conn(0x7f50880f9e80 msgr2=0x7f50880fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.872+0000 7f508e688700 1 -- 172.21.15.31:0/3639040299 shutdown_connections 2023-12-14T18:25:50.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.872+0000 7f508e688700 1 -- 172.21.15.31:0/3639040299 wait complete. 2023-12-14T18:25:50.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 Processor -- start 2023-12-14T18:25:50.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 -- start start 2023-12-14T18:25:50.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 0x7f5088198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:50.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fec40 0x7f5088198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f508819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508819fdc0 con 0x7f50880fe290 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508819ff00 con 0x7f50881007a0 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.873+0000 7f508e688700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50881a0090 con 0x7f50880fec40 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f5087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 0x7f5088198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f50877fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fec40 0x7f5088198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f508819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f508819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:38922/0 (socket says 172.21.15.31:38922) 2023-12-14T18:25:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f5087fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 0x7f5088198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58706/0 (socket says 172.21.15.31:58706) 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f50877fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fec40 0x7f5088198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47158/0 (socket says 172.21.15.31:47158) 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 -- 172.21.15.31:0/2207893530 learned_addr learned my addr 172.21.15.31:0/2207893530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 -- 172.21.15.31:0/2207893530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fec40 msgr2=0x7f5088198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fec40 0x7f5088198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 -- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 msgr2=0x7f5088198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 0x7f5088198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 -- 172.21.15.31:0/2207893530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5080009d00 con 0x7f50881007a0 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f5087fff700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 0x7f5088198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508cc25700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f508819f7f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f508000e3c0 tx=0x7f508000e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50800090d0 con 0x7f50881007a0 2023-12-14T18:25:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f508000f040 con 0x7f50881007a0 2023-12-14T18:25:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c009520 con 0x7f50881007a0 2023-12-14T18:25:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.874+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508810de60 con 0x7f50881007a0 2023-12-14T18:25:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.875+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080007c20 con 0x7f50881007a0 2023-12-14T18:25:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.876+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5080004030 con 0x7f50881007a0 2023-12-14T18:25:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.877+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508810a140 con 0x7f50881007a0 2023-12-14T18:25:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.877+0000 7f50857fa700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f50700a11d0 0x7f50700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.878+0000 7f5087fff700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f50700a11d0 0x7f50700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.878+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f50800cd170 con 0x7f50881007a0 2023-12-14T18:25:50.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.878+0000 7f5087fff700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f50700a11d0 0x7f50700a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5078011490 tx=0x7f5078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:50.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:50.882+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5080096d50 con 0x7f50881007a0 2023-12-14T18:25:50.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:50 smithi031 ceph-mon[42085]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:50.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:50 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] up:boot 2023-12-14T18:25:50.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:50 smithi031 ceph-mon[42085]: fsmap 12 up:standby 2023-12-14T18:25:50.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:50 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:25:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:50 smithi176 ceph-mon[43506]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:50 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] up:boot 2023-12-14T18:25:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:50 smithi176 ceph-mon[43506]: fsmap 12 up:standby 2023-12-14T18:25:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:50 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:25:51.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:50 smithi204 ceph-mon[43866]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:51.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:50 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] up:boot 2023-12-14T18:25:51.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:50 smithi204 ceph-mon[43866]: fsmap 12 up:standby 2023-12-14T18:25:51.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:50 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:25:51.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.161+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50880617a0 con 0x7f50881007a0 2023-12-14T18:25:51.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.162+0000 7f50857fa700 1 -- 172.21.15.31:0/2207893530 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9607 (secure 0 0 0) 0x7f508009a370 con 0x7f50881007a0 2023-12-14T18:25:51.164 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:51.164 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f50700a11d0 msgr2=0x7f50700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f50700a11d0 0x7f50700a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5078011490 tx=0x7f5078009260 comp rx=0 tx=0).stop 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 msgr2=0x7f508819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f508819f7f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f508000e3c0 tx=0x7f508000e6d0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 shutdown_connections 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f50881007a0 0x7f508819f7f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50880fe290 0x7f5088198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f50700a11d0 0x7f50700a3680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 --2- 172.21.15.31:0/2207893530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50880fec40 0x7f5088198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 >> 172.21.15.31:0/2207893530 conn(0x7f50880f9e80 msgr2=0x7f50880fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 shutdown_connections 2023-12-14T18:25:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.164+0000 7f508e688700 1 -- 172.21.15.31:0/2207893530 wait complete. 2023-12-14T18:25:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 13 2023-12-14T18:25:51.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:25:51.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 -- 172.21.15.31:0/2912733771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101dd0 msgr2=0x7f2538108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 --2- 172.21.15.31:0/2912733771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101dd0 0x7f2538108690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f253000b0a0 tx=0x7f253000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 -- 172.21.15.31:0/2912733771 shutdown_connections 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 --2- 172.21.15.31:0/2912733771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101dd0 0x7f2538108690 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 --2- 172.21.15.31:0/2912733771 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538100250 0x7f2538100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 --2- 172.21.15.31:0/2912733771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538101440 0x7f2538101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 -- 172.21.15.31:0/2912733771 >> 172.21.15.31:0/2912733771 conn(0x7f25380fb850 msgr2=0x7f25380fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 -- 172.21.15.31:0/2912733771 shutdown_connections 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.318+0000 7f253d3a9700 1 -- 172.21.15.31:0/2912733771 wait complete. 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.319+0000 7f253d3a9700 1 Processor -- start 2023-12-14T18:25:51.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.319+0000 7f253d3a9700 1 -- start start 2023-12-14T18:25:51.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f253d3a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 0x7f25381960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f253d3a9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101440 0x7f25381965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f253d3a9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538101dd0 0x7f253819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f253d3a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253819dc20 con 0x7f2538100250 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f253d3a9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253819dd60 con 0x7f2538101440 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f253d3a9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253819def0 con 0x7f2538101dd0 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 0x7f25381960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 0x7f25381960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58714/0 (socket says 172.21.15.31:58714) 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 -- 172.21.15.31:0/3087104954 learned_addr learned my addr 172.21.15.31:0/3087104954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f25367fc700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101440 0x7f25381965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f25377fe700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538101dd0 0x7f253819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 -- 172.21.15.31:0/3087104954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538101dd0 msgr2=0x7f253819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538101dd0 0x7f253819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 -- 172.21.15.31:0/3087104954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101440 msgr2=0x7f25381965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101440 0x7f25381965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 -- 172.21.15.31:0/3087104954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009d00 con 0x7f2538100250 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.320+0000 7f2536ffd700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 0x7f25381960a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f252800e970 tx=0x7f252800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.321+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528009790 con 0x7f2538100250 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.321+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253810bc30 con 0x7f2538100250 2023-12-14T18:25:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.321+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f252800c7e0 con 0x7f2538100250 2023-12-14T18:25:51.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.321+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528003c70 con 0x7f2538100250 2023-12-14T18:25:51.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.321+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253810baa0 con 0x7f2538100250 2023-12-14T18:25:51.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.322+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2538067e40 con 0x7f2538100250 2023-12-14T18:25:51.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.323+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f252800cc50 con 0x7f2538100250 2023-12-14T18:25:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.324+0000 7f251ffff700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25200a1150 0x7f25200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.324+0000 7f25367fc700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25200a1150 0x7f25200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.324+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2528014070 con 0x7f2538100250 2023-12-14T18:25:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.325+0000 7f25367fc700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25200a1150 0x7f25200a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f252c005950 tx=0x7f252c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:51.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.328+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2528096370 con 0x7f2538100250 2023-12-14T18:25:51.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.560+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25381070a0 con 0x7f2538100250 2023-12-14T18:25:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.562+0000 7f251ffff700 1 -- 172.21.15.31:0/3087104954 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18445 (secure 0 0 0) 0x7f2528099990 con 0x7f2538100250 2023-12-14T18:25:51.564 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:51.564 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":77,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:21.618906+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25200a1150 msgr2=0x7f25200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25200a1150 0x7f25200a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f252c005950 tx=0x7f252c00b410 comp rx=0 tx=0).stop 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 msgr2=0x7f25381960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 0x7f25381960a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f252800e970 tx=0x7f252800ec80 comp rx=0 tx=0).stop 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 shutdown_connections 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25200a1150 0x7f25200a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538101dd0 0x7f253819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538100250 0x7f25381960a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 --2- 172.21.15.31:0/3087104954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101440 0x7f25381965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 >> 172.21.15.31:0/3087104954 conn(0x7f25380fb850 msgr2=0x7f25380fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 shutdown_connections 2023-12-14T18:25:51.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.564+0000 7f253d3a9700 1 -- 172.21.15.31:0/3087104954 wait complete. 2023-12-14T18:25:51.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-14T18:25:51.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:25:51.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 -- 172.21.15.31:0/3175606811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520100d60 msgr2=0x7fd520103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 --2- 172.21.15.31:0/3175606811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520100d60 0x7fd520103140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd510009990 tx=0x7fd510009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 -- 172.21.15.31:0/3175606811 shutdown_connections 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 --2- 172.21.15.31:0/3175606811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd520105fa0 0x7fd520108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 --2- 172.21.15.31:0/3175606811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520100d60 0x7fd520103140 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 --2- 172.21.15.31:0/3175606811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520103680 0x7fd520105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 -- 172.21.15.31:0/3175606811 >> 172.21.15.31:0/3175606811 conn(0x7fd5200fa740 msgr2=0x7fd5200fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 -- 172.21.15.31:0/3175606811 shutdown_connections 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.723+0000 7fd525f7f700 1 -- 172.21.15.31:0/3175606811 wait complete. 2023-12-14T18:25:51.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 Processor -- start 2023-12-14T18:25:51.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 -- start start 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520100d60 0x7fd5201006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 0x7fd5200fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd520105fa0 0x7fd5200ff250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ff9f0 con 0x7fd520103680 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5200ffb60 con 0x7fd520105fa0 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd525f7f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52019ffe0 con 0x7fd520100d60 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd51effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 0x7fd5200fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd51effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 0x7fd5200fed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58740/0 (socket says 172.21.15.31:58740) 2023-12-14T18:25:51.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd51effd700 1 -- 172.21.15.31:0/2842368599 learned_addr learned my addr 172.21.15.31:0/2842368599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd51ffff700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd520105fa0 0x7fd5200ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.724+0000 7fd51f7fe700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520100d60 0x7fd5201006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51effd700 1 -- 172.21.15.31:0/2842368599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520100d60 msgr2=0x7fd5201006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51effd700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520100d60 0x7fd5201006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51effd700 1 -- 172.21.15.31:0/2842368599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd520105fa0 msgr2=0x7fd5200ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51effd700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd520105fa0 0x7fd5200ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51effd700 1 -- 172.21.15.31:0/2842368599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd514009520 con 0x7fd520103680 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51effd700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 0x7fd5200fed10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd51400eb30 tx=0x7fd51400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514009880 con 0x7fd520103680 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5140044e0 con 0x7fd520103680 2023-12-14T18:25:51.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5100095f0 con 0x7fd520103680 2023-12-14T18:25:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514003d80 con 0x7fd520103680 2023-12-14T18:25:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.725+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52010bba0 con 0x7fd520103680 2023-12-14T18:25:51.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.726+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520067160 con 0x7fd520103680 2023-12-14T18:25:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.727+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd514003f20 con 0x7fd520103680 2023-12-14T18:25:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.728+0000 7fd51cff9700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd5080a1150 0x7fd5080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.728+0000 7fd51f7fe700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd5080a1150 0x7fd5080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.728+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd514014070 con 0x7fd520103680 2023-12-14T18:25:51.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.729+0000 7fd51f7fe700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd5080a1150 0x7fd5080a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b680 tx=0x7fd510005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:51.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.733+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd514096590 con 0x7fd520103680 2023-12-14T18:25:51.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2207893530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:51.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:51 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3087104954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.973+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5201a0b50 con 0x7fd520103680 2023-12-14T18:25:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.974+0000 7fd51cff9700 1 -- 172.21.15.31:0/2842368599 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd514094e60 con 0x7fd520103680 2023-12-14T18:25:51.975 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd5080a1150 msgr2=0x7fd5080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd5080a1150 0x7fd5080a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b680 tx=0x7fd510005fd0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 msgr2=0x7fd5200fed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 0x7fd5200fed10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd51400eb30 tx=0x7fd51400ee40 comp rx=0 tx=0).stop 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 shutdown_connections 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd520105fa0 0x7fd5200ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd5080a1150 0x7fd5080a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd520100d60 0x7fd5201006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 --2- 172.21.15.31:0/2842368599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd520103680 0x7fd5200fed10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 >> 172.21.15.31:0/2842368599 conn(0x7fd5200fa740 msgr2=0x7fd5201021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 shutdown_connections 2023-12-14T18:25:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:51.976+0000 7fd525f7f700 1 -- 172.21.15.31:0/2842368599 wait complete. 2023-12-14T18:25:51.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2207893530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:52.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:51 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3087104954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 -- 172.21.15.31:0/264458544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210105ed0 msgr2=0x7f9210108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 --2- 172.21.15.31:0/264458544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210105ed0 0x7f9210108330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f920800b0a0 tx=0x7f920800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 -- 172.21.15.31:0/264458544 shutdown_connections 2023-12-14T18:25:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 --2- 172.21.15.31:0/264458544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210105ed0 0x7f9210108330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 --2- 172.21.15.31:0/264458544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210100c90 0x7f9210103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 --2- 172.21.15.31:0/264458544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92101035b0 0x7f9210105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 -- 172.21.15.31:0/264458544 >> 172.21.15.31:0/264458544 conn(0x7f92100fa6f0 msgr2=0x7f92100fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 -- 172.21.15.31:0/264458544 shutdown_connections 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.139+0000 7f9216dd0700 1 -- 172.21.15.31:0/264458544 wait complete. 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.140+0000 7f9216dd0700 1 Processor -- start 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.140+0000 7f9216dd0700 1 -- start start 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9216dd0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9210100c90 0x7f92101960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9216dd0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 0x7f92101965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9216dd0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210105ed0 0x7f921019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9216dd0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921019dc20 con 0x7f92101035b0 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9216dd0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921019dd60 con 0x7f9210100c90 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9216dd0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921019def0 con 0x7f9210105ed0 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f920ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 0x7f92101965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9210100c90 0x7f92101960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f920ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 0x7f92101965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58766/0 (socket says 172.21.15.31:58766) 2023-12-14T18:25:52.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f920ffff700 1 -- 172.21.15.31:0/26348647 learned_addr learned my addr 172.21.15.31:0/26348647 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:52.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f921536d700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210105ed0 0x7f921019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 -- 172.21.15.31:0/26348647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210105ed0 msgr2=0x7f921019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210105ed0 0x7f921019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 -- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 msgr2=0x7f92101965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 0x7f92101965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 -- 172.21.15.31:0/26348647 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9208009d00 con 0x7f9210100c90 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f920ffff700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 0x7f92101965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f921536d700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210105ed0 0x7f921019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.141+0000 7f9214b6c700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9210100c90 0x7f92101960a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f920400e9f0 tx=0x7f920400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.142+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204004220 con 0x7f9210100c90 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.142+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9204004cf0 con 0x7f9210100c90 2023-12-14T18:25:52.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.142+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921010bbe0 con 0x7f9210100c90 2023-12-14T18:25:52.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.142+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400ce70 con 0x7f9210100c90 2023-12-14T18:25:52.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.142+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921010ba50 con 0x7f9210100c90 2023-12-14T18:25:52.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.143+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92100fc260 con 0x7f9210100c90 2023-12-14T18:25:52.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.145+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9204004750 con 0x7f9210100c90 2023-12-14T18:25:52.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.146+0000 7f920dffb700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f92000a11d0 0x7f92000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.146+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9204014070 con 0x7f9210100c90 2023-12-14T18:25:52.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.146+0000 7f920ffff700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f92000a11d0 0x7f92000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.146+0000 7f920ffff700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f92000a11d0 0x7f92000a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91fc011550 tx=0x7f91fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.150+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9204095e90 con 0x7f9210100c90 2023-12-14T18:25:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2207893530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:52.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:52.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:52.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:52.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:52.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:51 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3087104954' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:52.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.381+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f921002cfe0 con 0x7f9210100c90 2023-12-14T18:25:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.381+0000 7f920dffb700 1 -- 172.21.15.31:0/26348647 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9204004e60 con 0x7f9210100c90 2023-12-14T18:25:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:25:52.383 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-12-14T18:25:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:25:52.383 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-12-14T18:25:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f92000a11d0 msgr2=0x7f92000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f92000a11d0 0x7f92000a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91fc011550 tx=0x7f91fc009260 comp rx=0 tx=0).stop 2023-12-14T18:25:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9210100c90 msgr2=0x7f92101960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9210100c90 0x7f92101960a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f920400e9f0 tx=0x7f920400ed00 comp rx=0 tx=0).stop 2023-12-14T18:25:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 shutdown_connections 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f92000a11d0 0x7f92000a3680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9210105ed0 0x7f921019d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9210100c90 0x7f92101960a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 --2- 172.21.15.31:0/26348647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92101035b0 0x7f92101965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 >> 172.21.15.31:0/26348647 conn(0x7f92100fa6f0 msgr2=0x7f9210101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 shutdown_connections 2023-12-14T18:25:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.384+0000 7f9216dd0700 1 -- 172.21.15.31:0/26348647 wait complete. 2023-12-14T18:25:52.401 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-14T18:25:52.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-14T18:25:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 -- 172.21.15.31:0/2107686483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ff440 msgr2=0x7f81780ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 --2- 172.21.15.31:0/2107686483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ff440 0x7f81780ff8b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f816c009990 tx=0x7f816c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 -- 172.21.15.31:0/2107686483 shutdown_connections 2023-12-14T18:25:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 --2- 172.21.15.31:0/2107686483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ffdf0 0x7f8178100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 --2- 172.21.15.31:0/2107686483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81780feaf0 0x7f81780fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 --2- 172.21.15.31:0/2107686483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ff440 0x7f81780ff8b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 -- 172.21.15.31:0/2107686483 >> 172.21.15.31:0/2107686483 conn(0x7f81780fa740 msgr2=0x7f81780fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 -- 172.21.15.31:0/2107686483 shutdown_connections 2023-12-14T18:25:52.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.553+0000 7f817cee6700 1 -- 172.21.15.31:0/2107686483 wait complete. 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 Processor -- start 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 -- start start 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81780feaf0 0x7f8178198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 0x7f8178198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ffdf0 0x7f817819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817819fcd0 con 0x7f81780ff440 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817819fe10 con 0x7f81780ffdf0 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.554+0000 7f817cee6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817819ffa0 con 0x7f81780feaf0 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 0x7f8178198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 0x7f8178198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58784/0 (socket says 172.21.15.31:58784) 2023-12-14T18:25:52.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8176d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ffdf0 0x7f817819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 -- 172.21.15.31:0/3831313065 learned_addr learned my addr 172.21.15.31:0/3831313065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f817659c700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81780feaf0 0x7f8178198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 -- 172.21.15.31:0/3831313065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81780feaf0 msgr2=0x7f8178198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81780feaf0 0x7f8178198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 -- 172.21.15.31:0/3831313065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ffdf0 msgr2=0x7f817819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ffdf0 0x7f817819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 -- 172.21.15.31:0/3831313065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0095f0 con 0x7f81780ff440 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f8175d9b700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 0x7f8178198690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f816c0059d0 tx=0x7f816c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c01d070 con 0x7f81780ff440 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817810dd80 con 0x7f81780ff440 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816c00fb90 con 0x7f81780ff440 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816c0173f0 con 0x7f81780ff440 2023-12-14T18:25:52.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.555+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817810dbf0 con 0x7f81780ff440 2023-12-14T18:25:52.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.557+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81781098d0 con 0x7f81780ff440 2023-12-14T18:25:52.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.558+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f816c00fd00 con 0x7f81780ff440 2023-12-14T18:25:52.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.559+0000 7f81677fe700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81600a1150 0x7f81600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.559+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f816c0cd8e0 con 0x7f81780ff440 2023-12-14T18:25:52.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.559+0000 7f817659c700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81600a1150 0x7f81600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.559+0000 7f817659c700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81600a1150 0x7f81600a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8168009c60 tx=0x7f8168009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.564+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f816c05c730 con 0x7f81780ff440 2023-12-14T18:25:52.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:52.826+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8178061700 con 0x7f81780ff440 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2842368599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/26348647' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:52 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:53.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2842368599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/26348647' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:52 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2842368599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/26348647' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:52 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.831+0000 7f81677fe700 1 -- 172.21.15.31:0/3831313065 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f816c0974c0 con 0x7f81780ff440 2023-12-14T18:25:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81600a1150 msgr2=0x7f81600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81600a1150 0x7f81600a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8168009c60 tx=0x7f8168009260 comp rx=0 tx=0).stop 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 msgr2=0x7f8178198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 0x7f8178198690 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f816c0059d0 tx=0x7f816c004730 comp rx=0 tx=0).stop 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 shutdown_connections 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81780ffdf0 0x7f817819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81600a1150 0x7f81600a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81780feaf0 0x7f8178198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 --2- 172.21.15.31:0/3831313065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81780ff440 0x7f8178198690 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 >> 172.21.15.31:0/3831313065 conn(0x7f81780fa740 msgr2=0x7f81780fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.837+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 shutdown_connections 2023-12-14T18:25:53.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:53.838+0000 7f817cee6700 1 -- 172.21.15.31:0/3831313065 wait complete. 2023-12-14T18:25:53.841 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-12-14T18:25:53.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 -- 172.21.15.31:0/2833581914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 msgr2=0x7f0ee8100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/2833581914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8100760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009990 tx=0x7f0ed8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 -- 172.21.15.31:0/2833581914 shutdown_connections 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/2833581914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 0x7f0ee8107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/2833581914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 0x7f0ee80ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/2833581914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8100760 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 -- 172.21.15.31:0/2833581914 >> 172.21.15.31:0/2833581914 conn(0x7f0ee80fa740 msgr2=0x7f0ee80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 -- 172.21.15.31:0/2833581914 shutdown_connections 2023-12-14T18:25:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 -- 172.21.15.31:0/2833581914 wait complete. 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.002+0000 7f0ef01f7700 1 Processor -- start 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 -- start start 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 0x7f0ee8196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 0x7f0ee819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819db90 con 0x7f0ee8100310 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819dcd0 con 0x7f0ee8100ca0 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0ef01f7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819de60 con 0x7f0ee80ff120 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58802/0 (socket says 172.21.15.31:58802) 2023-12-14T18:25:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 -- 172.21.15.31:0/1728191074 learned_addr learned my addr 172.21.15.31:0/1728191074 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eee794700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 0x7f0ee819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eedf93700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 0x7f0ee8196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 -- 172.21.15.31:0/1728191074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 msgr2=0x7f0ee8196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 0x7f0ee8196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 -- 172.21.15.31:0/1728191074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 msgr2=0x7f0ee819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 0x7f0ee819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 -- 172.21.15.31:0/1728191074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80095f0 con 0x7f0ee8100310 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eedf93700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 0x7f0ee8196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eee794700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 0x7f0ee819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.003+0000 7f0eed792700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8196550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800b620 tx=0x7f0ed8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.004+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed801d070 con 0x7f0ee8100310 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.004+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ed80047e0 con 0x7f0ee8100310 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.004+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee810aa80 con 0x7f0ee8100310 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.004+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8005560 con 0x7f0ee8100310 2023-12-14T18:25:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.004+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee810a8f0 con 0x7f0ee8100310 2023-12-14T18:25:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.005+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee819e460 con 0x7f0ee8100310 2023-12-14T18:25:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.006+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0ed8004950 con 0x7f0ee8100310 2023-12-14T18:25:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.007+0000 7f0edeffd700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0ed40a11d0 0x7f0ed40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.007+0000 7f0eedf93700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0ed40a11d0 0x7f0ed40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.007+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f0ed80cda90 con 0x7f0ee8100310 2023-12-14T18:25:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.007+0000 7f0eedf93700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0ed40a11d0 0x7f0ed40a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8100170 tx=0x7f0ee4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.011+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0ed80975a0 con 0x7f0ee8100310 2023-12-14T18:25:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:53 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3831313065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:25:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:53 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3831313065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:25:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.219+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f0ee819ea10 con 0x7f0ee8100310 2023-12-14T18:25:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:53 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3831313065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:25:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:25:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.837+0000 7f0edeffd700 1 -- 172.21.15.31:0/1728191074 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) v1 ==== 171+0+0 (secure 0 0 0) 0x7f0ed809abc0 con 0x7f0ee8100310 2023-12-14T18:25:54.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.850+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0ed40a11d0 msgr2=0x7f0ed40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.850+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0ed40a11d0 0x7f0ed40a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8100170 tx=0x7f0ee4009260 comp rx=0 tx=0).stop 2023-12-14T18:25:54.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.850+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 msgr2=0x7f0ee8196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.850+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8196550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800b620 tx=0x7f0ed8005ef0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.855+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 shutdown_connections 2023-12-14T18:25:54.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.855+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0ee8100ca0 0x7f0ee819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.855+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0ed40a11d0 0x7f0ed40a3680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.855+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ee80ff120 0x7f0ee8196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.855+0000 7f0ef01f7700 1 --2- 172.21.15.31:0/1728191074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ee8100310 0x7f0ee8196550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.855+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 >> 172.21.15.31:0/1728191074 conn(0x7f0ee80fa740 msgr2=0x7f0ee8067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.857+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 shutdown_connections 2023-12-14T18:25:54.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:54.857+0000 7f0ef01f7700 1 -- 172.21.15.31:0/1728191074 wait complete. 2023-12-14T18:25:54.863 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-12-14T18:25:54.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-14T18:25:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 -- 172.21.15.31:0/1246384887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb00ff160 msgr2=0x7f1cb00ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/1246384887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb00ff160 0x7f1cb00ff570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009990 tx=0x7f1c98009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 -- 172.21.15.31:0/1246384887 shutdown_connections 2023-12-14T18:25:55.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/1246384887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cb0100ce0 0x7f1cb01075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/1246384887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb00ff160 0x7f1cb00ff570 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/1246384887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb0100350 0x7f1cb01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 -- 172.21.15.31:0/1246384887 >> 172.21.15.31:0/1246384887 conn(0x7f1cb00fa780 msgr2=0x7f1cb00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 -- 172.21.15.31:0/1246384887 shutdown_connections 2023-12-14T18:25:55.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.017+0000 7f1cb4b01700 1 -- 172.21.15.31:0/1246384887 wait complete. 2023-12-14T18:25:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 Processor -- start 2023-12-14T18:25:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 -- start start 2023-12-14T18:25:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb00ff160 0x7f1cb0195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cb0100350 0x7f1cb01964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 0x7f1cb019d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019db20 con 0x7f1cb0100ce0 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019dc60 con 0x7f1cb0100350 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cb4b01700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019ddf0 con 0x7f1cb00ff160 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1caed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 0x7f1cb019d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1caed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 0x7f1cb019d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34364/0 (socket says 172.21.15.31:34364) 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1caed9d700 1 -- 172.21.15.31:0/3311807566 learned_addr learned my addr 172.21.15.31:0/3311807566 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cadd9b700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cb0100350 0x7f1cb01964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.018+0000 7f1cae59c700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb00ff160 0x7f1cb0195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1caed9d700 1 -- 172.21.15.31:0/3311807566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb00ff160 msgr2=0x7f1cb0195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1caed9d700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb00ff160 0x7f1cb0195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1caed9d700 1 -- 172.21.15.31:0/3311807566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cb0100350 msgr2=0x7f1cb01964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1caed9d700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cb0100350 0x7f1cb01964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1caed9d700 1 -- 172.21.15.31:0/3311807566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980095f0 con 0x7f1cb0100ce0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1caed9d700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 0x7f1cb019d550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800ef30 tx=0x7f1ca800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8018070 con 0x7f1cb0100ce0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb010ab10 con 0x7f1cb0100ce0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca800f040 con 0x7f1cb0100ce0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800cd00 con 0x7f1cb0100ce0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.019+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb010a980 con 0x7f1cb0100ce0 2023-12-14T18:25:55.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.020+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb019e480 con 0x7f1cb0100ce0 2023-12-14T18:25:55.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.021+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1ca8003a00 con 0x7f1cb0100ce0 2023-12-14T18:25:55.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.022+0000 7f1ca77fe700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1c9c0a1150 0x7f1c9c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.022+0000 7f1cae59c700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1c9c0a1150 0x7f1c9c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.022+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f1ca80ccbf0 con 0x7f1cb0100ce0 2023-12-14T18:25:55.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.023+0000 7f1cae59c700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1c9c0a1150 0x7f1c9c0a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800b2e0 tx=0x7f1c98005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.026+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1ca8096580 con 0x7f1cb0100ce0 2023-12-14T18:25:55.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3831313065' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: osdmap e78: 12 total, 12 up, 12 in 2023-12-14T18:25:55.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:54 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1728191074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3831313065' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: osdmap e78: 12 total, 12 up, 12 in 2023-12-14T18:25:55.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:54 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1728191074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-14T18:25:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.284+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1cb019ea30 con 0x7f1cb0100ce0 2023-12-14T18:25:55.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.293+0000 7f1ca77fe700 1 -- 172.21.15.31:0/3311807566 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1ca8099ba0 con 0x7f1cb0100ce0 2023-12-14T18:25:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.297+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1c9c0a1150 msgr2=0x7f1c9c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.297+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1c9c0a1150 0x7f1c9c0a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800b2e0 tx=0x7f1c98005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.297+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 msgr2=0x7f1cb019d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.297+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 0x7f1cb019d550 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800ef30 tx=0x7f1ca800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.307+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 shutdown_connections 2023-12-14T18:25:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.307+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb0100ce0 0x7f1cb019d550 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.307+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1c9c0a1150 0x7f1c9c0a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.307+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1cb00ff160 0x7f1cb0195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.307+0000 7f1cb4b01700 1 --2- 172.21.15.31:0/3311807566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1cb0100350 0x7f1cb01964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.307+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 >> 172.21.15.31:0/3311807566 conn(0x7f1cb00fa780 msgr2=0x7f1cb0068d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.309+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 shutdown_connections 2023-12-14T18:25:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.309+0000 7f1cb4b01700 1 -- 172.21.15.31:0/3311807566 wait complete. 2023-12-14T18:25:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2023-12-14T18:25:55.335 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-14T18:25:55.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3831313065' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: osdmap e78: 12 total, 12 up, 12 in 2023-12-14T18:25:55.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:54 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1728191074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-14T18:25:55.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 -- 172.21.15.31:0/4241652099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9444100310 msgr2=0x7f9444100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 --2- 172.21.15.31:0/4241652099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9444100310 0x7f9444100760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9438009990 tx=0x7f9438009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 -- 172.21.15.31:0/4241652099 shutdown_connections 2023-12-14T18:25:55.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 --2- 172.21.15.31:0/4241652099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9444100ca0 0x7f9444107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 --2- 172.21.15.31:0/4241652099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94440ff120 0x7f94440ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 --2- 172.21.15.31:0/4241652099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9444100310 0x7f9444100760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.475+0000 7f9449fc9700 1 -- 172.21.15.31:0/4241652099 >> 172.21.15.31:0/4241652099 conn(0x7f94440fa740 msgr2=0x7f94440fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.476+0000 7f9449fc9700 1 -- 172.21.15.31:0/4241652099 shutdown_connections 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.476+0000 7f9449fc9700 1 -- 172.21.15.31:0/4241652099 wait complete. 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.476+0000 7f9449fc9700 1 Processor -- start 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.476+0000 7f9449fc9700 1 -- start start 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9449fc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 0x7f9444195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9449fc9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9444100310 0x7f94441964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9449fc9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9444100ca0 0x7f944419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9449fc9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944419db20 con 0x7f94440ff120 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9449fc9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944419dc60 con 0x7f9444100ca0 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9449fc9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944419ddf0 con 0x7f9444100310 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 0x7f9444195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 0x7f9444195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34372/0 (socket says 172.21.15.31:34372) 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 -- 172.21.15.31:0/1320480653 learned_addr learned my addr 172.21.15.31:0/1320480653 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9442ffd700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9444100310 0x7f94441964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9443fff700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9444100ca0 0x7f944419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 -- 172.21.15.31:0/1320480653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9444100310 msgr2=0x7f94441964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9444100310 0x7f94441964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 -- 172.21.15.31:0/1320480653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9444100ca0 msgr2=0x7f944419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9444100ca0 0x7f944419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 -- 172.21.15.31:0/1320480653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380095f0 con 0x7f94440ff120 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f94437fe700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 0x7f9444195fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f943400d980 tx=0x7f943400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.477+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943400f890 con 0x7f94440ff120 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.478+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9434010460 con 0x7f94440ff120 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.478+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944410bbf0 con 0x7f94440ff120 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.478+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340195c0 con 0x7f94440ff120 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.478+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944410ba60 con 0x7f94440ff120 2023-12-14T18:25:55.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.479+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94441060a0 con 0x7f94440ff120 2023-12-14T18:25:55.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.480+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f94340105d0 con 0x7f94440ff120 2023-12-14T18:25:55.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.480+0000 7f9440ff9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f942c0a11d0 0x7f942c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.481+0000 7f9442ffd700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f942c0a11d0 0x7f942c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.481+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f94340cc4a0 con 0x7f94440ff120 2023-12-14T18:25:55.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.481+0000 7f9442ffd700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f942c0a11d0 0x7f942c0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f943800b5c0 tx=0x7f9438005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.485+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9434095d40 con 0x7f94440ff120 2023-12-14T18:25:55.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:55.708+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f944402cfe0 con 0x7f94440ff120 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1728191074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: osdmap e79: 12 total, 12 up, 12 in 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3311807566' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: osdmap e80: 12 total, 12 up, 12 in 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3311807566' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-12-14T18:25:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:55 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1320480653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1728191074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: osdmap e79: 12 total, 12 up, 12 in 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3311807566' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: osdmap e80: 12 total, 12 up, 12 in 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3311807566' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-12-14T18:25:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:55 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1320480653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-14T18:25:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.297+0000 7f9440ff9700 1 -- 172.21.15.31:0/1320480653 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) v1 ==== 157+0+0 (secure 0 0 0) 0x7f9434099360 con 0x7f94440ff120 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f942c0a11d0 msgr2=0x7f942c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f942c0a11d0 0x7f942c0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f943800b5c0 tx=0x7f9438005960 comp rx=0 tx=0).stop 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 msgr2=0x7f9444195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 0x7f9444195fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f943400d980 tx=0x7f943400dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 shutdown_connections 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9444100ca0 0x7f944419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94440ff120 0x7f9444195fa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f942c0a11d0 0x7f942c0a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 --2- 172.21.15.31:0/1320480653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9444100310 0x7f94441964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 >> 172.21.15.31:0/1320480653 conn(0x7f94440fa740 msgr2=0x7f94440fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 shutdown_connections 2023-12-14T18:25:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.301+0000 7f9449fc9700 1 -- 172.21.15.31:0/1320480653 wait complete. 2023-12-14T18:25:56.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1728191074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: osdmap e79: 12 total, 12 up, 12 in 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3311807566' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: daemon mds.i assigned to filesystem cephfs as rank 0 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: osdmap e80: 12 total, 12 up, 12 in 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3311807566' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2023-12-14T18:25:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: daemon mds.i is now active in filesystem cephfs as rank 0 2023-12-14T18:25:56.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:55 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1320480653' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-14T18:25:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 -- 172.21.15.31:0/910283596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 msgr2=0x7fb3a8106ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 --2- 172.21.15.31:0/910283596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a8106ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000c210 tx=0x7fb3a000c520 comp rx=0 tx=0).stop 2023-12-14T18:25:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 -- 172.21.15.31:0/910283596 shutdown_connections 2023-12-14T18:25:56.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 --2- 172.21.15.31:0/910283596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a8106ce0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 --2- 172.21.15.31:0/910283596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 0x7fb3a80fecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 --2- 172.21.15.31:0/910283596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3a80ffa90 0x7fb3a80ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 -- 172.21.15.31:0/910283596 >> 172.21.15.31:0/910283596 conn(0x7fb3a80f9ee0 msgr2=0x7fb3a80fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 -- 172.21.15.31:0/910283596 shutdown_connections 2023-12-14T18:25:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.468+0000 7fb3adc77700 1 -- 172.21.15.31:0/910283596 wait complete. 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.469+0000 7fb3adc77700 1 Processor -- start 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.469+0000 7fb3adc77700 1 -- start start 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3adc77700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 0x7fb3a8196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3adc77700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3a80ffa90 0x7fb3a81965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3adc77700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3adc77700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a819dbf0 con 0x7fb3a8100420 2023-12-14T18:25:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3adc77700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a819dd30 con 0x7fb3a80fe8a0 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3adc77700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a819dec0 con 0x7fb3a80ffa90 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34400/0 (socket says 172.21.15.31:34400) 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 -- 172.21.15.31:0/805011657 learned_addr learned my addr 172.21.15.31:0/805011657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a77fe700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 0x7fb3a8196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a6ffd700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3a80ffa90 0x7fb3a81965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 -- 172.21.15.31:0/805011657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3a80ffa90 msgr2=0x7fb3a81965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3a80ffa90 0x7fb3a81965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 -- 172.21.15.31:0/805011657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 msgr2=0x7fb3a8196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 0x7fb3a8196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 -- 172.21.15.31:0/805011657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a000ae10 con 0x7fb3a8100420 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a77fe700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 0x7fb3a8196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.470+0000 7fb3a7fff700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a819d620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000a040 tx=0x7fb3a0008410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.471+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000f040 con 0x7fb3a8100420 2023-12-14T18:25:56.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.471+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a8109980 con 0x7fb3a8100420 2023-12-14T18:25:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.471+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3a0005030 con 0x7fb3a8100420 2023-12-14T18:25:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.471+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0007700 con 0x7fb3a8100420 2023-12-14T18:25:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.471+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a81097f0 con 0x7fb3a8100420 2023-12-14T18:25:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.472+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a8067160 con 0x7fb3a8100420 2023-12-14T18:25:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.473+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb3a00051a0 con 0x7fb3a8100420 2023-12-14T18:25:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.474+0000 7fb3a4ff9700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb3900a1150 0x7fb3900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.474+0000 7fb3a77fe700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb3900a1150 0x7fb3900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.474+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fb3a00cdef0 con 0x7fb3a8100420 2023-12-14T18:25:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.474+0000 7fb3a77fe700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb3900a1150 0x7fb3900a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80ff8f0 tx=0x7fb39800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.477+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb3a0097780 con 0x7fb3a8100420 2023-12-14T18:25:56.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:56.690+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb3a81056e0 con 0x7fb3a8100420 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: pgmap v224: 97 pgs: 12 creating+peering, 2 active+clean, 83 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] up:active 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1320480653' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: osdmap e81: 12 total, 12 up, 12 in 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:25:57.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:56 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/805011657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: pgmap v224: 97 pgs: 12 creating+peering, 2 active+clean, 83 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] up:active 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1320480653' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: osdmap e81: 12 total, 12 up, 12 in 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:25:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:56 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/805011657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: pgmap v224: 97 pgs: 12 creating+peering, 2 active+clean, 83 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] up:active 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1320480653' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: osdmap e81: 12 total, 12 up, 12 in 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:25:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:56 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/805011657' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-14T18:25:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.318+0000 7fb3a4ff9700 1 -- 172.21.15.31:0/805011657 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) v1 ==== 228+0+0 (secure 0 0 0) 0x7fb3a009ada0 con 0x7fb3a8100420 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb3900a1150 msgr2=0x7fb3900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb3900a1150 0x7fb3900a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80ff8f0 tx=0x7fb39800b670 comp rx=0 tx=0).stop 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 msgr2=0x7fb3a819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a819d620 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000a040 tx=0x7fb3a0008410 comp rx=0 tx=0).stop 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 shutdown_connections 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3a8100420 0x7fb3a819d620 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3a80fe8a0 0x7fb3a8196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb3900a1150 0x7fb3900a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 --2- 172.21.15.31:0/805011657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3a80ffa90 0x7fb3a81965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 >> 172.21.15.31:0/805011657 conn(0x7fb3a80f9ee0 msgr2=0x7fb3a80fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.325+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 shutdown_connections 2023-12-14T18:25:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.326+0000 7fb3adc77700 1 -- 172.21.15.31:0/805011657 wait complete. 2023-12-14T18:25:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data_ec' created 2023-12-14T18:25:58.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-14T18:25:58.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 -- 172.21.15.31:0/2092728143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c40ff120 msgr2=0x7fa8c40ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/2092728143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c40ff120 0x7fa8c40ff530 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac009990 tx=0x7fa8ac009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 -- 172.21.15.31:0/2092728143 shutdown_connections 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/2092728143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8c4100ca0 0x7fa8c4107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/2092728143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c40ff120 0x7fa8c40ff530 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/2092728143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8c4100310 0x7fa8c4100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 -- 172.21.15.31:0/2092728143 >> 172.21.15.31:0/2092728143 conn(0x7fa8c40fa740 msgr2=0x7fa8c40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 -- 172.21.15.31:0/2092728143 shutdown_connections 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.488+0000 7fa8c90d6700 1 -- 172.21.15.31:0/2092728143 wait complete. 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 Processor -- start 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 -- start start 2023-12-14T18:25:58.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8c40ff120 0x7fa8c41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 0x7fa8c41016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8c4100ca0 0x7fa8c4101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41023d0 con 0x7fa8c4100310 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4102540 con 0x7fa8c40ff120 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c90d6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41a0120 con 0x7fa8c4100ca0 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 0x7fa8c41016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 0x7fa8c41016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34408/0 (socket says 172.21.15.31:34408) 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 -- 172.21.15.31:0/482102856 learned_addr learned my addr 172.21.15.31:0/482102856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:58.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c359e700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8c4100ca0 0x7fa8c4101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c2d9d700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8c40ff120 0x7fa8c41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 -- 172.21.15.31:0/482102856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8c4100ca0 msgr2=0x7fa8c4101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8c4100ca0 0x7fa8c4101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 -- 172.21.15.31:0/482102856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8c40ff120 msgr2=0x7fa8c41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.489+0000 7fa8c259c700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8c40ff120 0x7fa8c41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8c259c700 1 -- 172.21.15.31:0/482102856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0095f0 con 0x7fa8c4100310 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8c259c700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 0x7fa8c41016f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400e9a0 tx=0x7fa8b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4009790 con 0x7fa8c4100310 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8b400cbd0 con 0x7fa8c4100310 2023-12-14T18:25:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c410aae0 con 0x7fa8c4100310 2023-12-14T18:25:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4018680 con 0x7fa8c4100310 2023-12-14T18:25:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.490+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c410a950 con 0x7fa8c4100310 2023-12-14T18:25:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.491+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c41a0810 con 0x7fa8c4100310 2023-12-14T18:25:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.492+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa8b400cd40 con 0x7fa8c4100310 2023-12-14T18:25:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.493+0000 7fa8bbfff700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8b00a1150 0x7fa8b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.493+0000 7fa8c2d9d700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8b00a1150 0x7fa8b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.493+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fa8b4014070 con 0x7fa8c4100310 2023-12-14T18:25:58.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.494+0000 7fa8c2d9d700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8b00a1150 0x7fa8b00a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00b2e0 tx=0x7fa8ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:58.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.497+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa8b4096590 con 0x7fa8c4100310 2023-12-14T18:25:58.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:58 smithi204 ceph-mon[43866]: pgmap v227: 97 pgs: 12 creating+peering, 53 active+clean, 32 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 8 op/s 2023-12-14T18:25:58.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:58 smithi204 ceph-mon[43866]: osdmap e82: 12 total, 12 up, 12 in 2023-12-14T18:25:58.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:58.704+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fa8c41a0dc0 con 0x7fa8c4100310 2023-12-14T18:25:58.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:58 smithi031 ceph-mon[42085]: pgmap v227: 97 pgs: 12 creating+peering, 53 active+clean, 32 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 8 op/s 2023-12-14T18:25:58.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:58 smithi031 ceph-mon[42085]: osdmap e82: 12 total, 12 up, 12 in 2023-12-14T18:25:58.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:58 smithi176 ceph-mon[43506]: pgmap v227: 97 pgs: 12 creating+peering, 53 active+clean, 32 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 8 op/s 2023-12-14T18:25:58.745 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:58 smithi176 ceph-mon[43506]: osdmap e82: 12 total, 12 up, 12 in 2023-12-14T18:25:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.321+0000 7fa8bbfff700 1 -- 172.21.15.31:0/482102856 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa8b4094e60 con 0x7fa8c4100310 2023-12-14T18:25:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.329+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8b00a1150 msgr2=0x7fa8b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.329+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8b00a1150 0x7fa8b00a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00b2e0 tx=0x7fa8ac005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.329+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 msgr2=0x7fa8c41016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.329+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 0x7fa8c41016f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400e9a0 tx=0x7fa8b400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.331+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 shutdown_connections 2023-12-14T18:25:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.331+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8b00a1150 0x7fa8b00a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.331+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8c4100ca0 0x7fa8c4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.331+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8c40ff120 0x7fa8c41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.331+0000 7fa8c90d6700 1 --2- 172.21.15.31:0/482102856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8c4100310 0x7fa8c41016f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.331+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 >> 172.21.15.31:0/482102856 conn(0x7fa8c40fa740 msgr2=0x7fa8c4067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:59.335 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:25:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: 2023-12-14T18:25:59.029+0000 7fdf8b999700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-14T18:25:59.335 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:59 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/805011657' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-14T18:25:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:59 smithi204 ceph-mon[43866]: osdmap e83: 12 total, 12 up, 12 in 2023-12-14T18:25:59.336 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:25:59 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/482102856' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-14T18:25:59.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.338+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 shutdown_connections 2023-12-14T18:25:59.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.344+0000 7fa8c90d6700 1 -- 172.21.15.31:0/482102856 wait complete. 2023-12-14T18:25:59.350 INFO:teuthology.orchestra.run.smithi031.stderr:set pool 4 allow_ec_overwrites to true 2023-12-14T18:25:59.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-14T18:25:59.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 -- 172.21.15.31:0/3884452332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100cd0 msgr2=0x7f93d8107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 --2- 172.21.15.31:0/3884452332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100cd0 0x7f93d8107590 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f93d000b0a0 tx=0x7f93d000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 -- 172.21.15.31:0/3884452332 shutdown_connections 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 --2- 172.21.15.31:0/3884452332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100cd0 0x7f93d8107590 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 --2- 172.21.15.31:0/3884452332 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93d80ff150 0x7f93d80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 --2- 172.21.15.31:0/3884452332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d8100340 0x7f93d8100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 -- 172.21.15.31:0/3884452332 >> 172.21.15.31:0/3884452332 conn(0x7f93d80fa770 msgr2=0x7f93d80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 -- 172.21.15.31:0/3884452332 shutdown_connections 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.504+0000 7f93def8f700 1 -- 172.21.15.31:0/3884452332 wait complete. 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 Processor -- start 2023-12-14T18:25:59.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 -- start start 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93d80ff150 0x7f93d8194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 0x7f93d8195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d8100cd0 0x7f93d819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d819cb40 con 0x7f93d8100340 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d819cc80 con 0x7f93d80ff150 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93def8f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d819ce10 con 0x7f93d8100cd0 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.505+0000 7f93d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 0x7f93d8195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 0x7f93d8195500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34430/0 (socket says 172.21.15.31:34430) 2023-12-14T18:25:59.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 -- 172.21.15.31:0/3399189434 learned_addr learned my addr 172.21.15.31:0/3399189434 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93dd52c700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d8100cd0 0x7f93d819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93dcd2b700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93d80ff150 0x7f93d8194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 -- 172.21.15.31:0/3399189434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d8100cd0 msgr2=0x7f93d819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d8100cd0 0x7f93d819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 -- 172.21.15.31:0/3399189434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93d80ff150 msgr2=0x7f93d8194fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93d80ff150 0x7f93d8194fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 -- 172.21.15.31:0/3399189434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d0009d00 con 0x7f93d8100340 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d7fff700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 0x7f93d8195500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93cc007b50 tx=0x7f93cc007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc011070 con 0x7f93d8100340 2023-12-14T18:25:59.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93cc00c2e0 con 0x7f93d8100340 2023-12-14T18:25:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00e500 con 0x7f93d8100340 2023-12-14T18:25:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.506+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d810ab20 con 0x7f93d8100340 2023-12-14T18:25:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.507+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d810a990 con 0x7f93d8100340 2023-12-14T18:25:59.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.507+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d80691a0 con 0x7f93d8100340 2023-12-14T18:25:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.509+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f93cc004750 con 0x7f93d8100340 2023-12-14T18:25:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.509+0000 7f93d5ffb700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f93c00a1150 0x7f93c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.509+0000 7f93dcd2b700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f93c00a1150 0x7f93c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.509+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f93cc0cdda0 con 0x7f93d8100340 2023-12-14T18:25:59.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.510+0000 7f93dcd2b700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f93c00a1150 0x7f93c00a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005950 tx=0x7f93c80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:59.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.514+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f93cc0973b0 con 0x7f93d8100340 2023-12-14T18:25:59.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:59 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/805011657' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-14T18:25:59.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:59 smithi031 ceph-mon[42085]: osdmap e83: 12 total, 12 up, 12 in 2023-12-14T18:25:59.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:25:59 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/482102856' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-14T18:25:59.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:59 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/805011657' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-14T18:25:59.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:59 smithi176 ceph-mon[43506]: osdmap e83: 12 total, 12 up, 12 in 2023-12-14T18:25:59.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:25:59 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/482102856' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-14T18:25:59.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.777+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f93d8069c60 con 0x7f93d8100340 2023-12-14T18:25:59.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.783+0000 7f93d5ffb700 1 -- 172.21.15.31:0/3399189434 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f93cc09a9d0 con 0x7f93d8100340 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f93c00a1150 msgr2=0x7f93c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f93c00a1150 0x7f93c00a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005950 tx=0x7f93c80058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 msgr2=0x7f93d8195500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 0x7f93d8195500 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93cc007b50 tx=0x7f93cc007e60 comp rx=0 tx=0).stop 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 shutdown_connections 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f93c00a1150 0x7f93c00a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93d8100cd0 0x7f93d819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f93d80ff150 0x7f93d8194fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 --2- 172.21.15.31:0/3399189434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8100340 0x7f93d8195500 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.785+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 >> 172.21.15.31:0/3399189434 conn(0x7f93d80fa770 msgr2=0x7f93d80fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.786+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 shutdown_connections 2023-12-14T18:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.786+0000 7f93def8f700 1 -- 172.21.15.31:0/3399189434 wait complete. 2023-12-14T18:25:59.789 INFO:teuthology.orchestra.run.smithi031.stderr:added data pool 4 to fsmap 2023-12-14T18:25:59.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.936+0000 7fa8870e5700 1 -- 172.21.15.31:0/3739157836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880100c90 msgr2=0x7fa880103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.936+0000 7fa8870e5700 1 --2- 172.21.15.31:0/3739157836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880100c90 0x7fa880103070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa874009990 tx=0x7fa874009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- 172.21.15.31:0/3739157836 shutdown_connections 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 --2- 172.21.15.31:0/3739157836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa880105ed0 0x7fa880108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 --2- 172.21.15.31:0/3739157836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880100c90 0x7fa880103070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 --2- 172.21.15.31:0/3739157836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 0x7fa880105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- 172.21.15.31:0/3739157836 >> 172.21.15.31:0/3739157836 conn(0x7fa8800fa6f0 msgr2=0x7fa8800fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- 172.21.15.31:0/3739157836 shutdown_connections 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- 172.21.15.31:0/3739157836 wait complete. 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 Processor -- start 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- start start 2023-12-14T18:25:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa880100c90 0x7fa880195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 0x7fa880196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 0x7fa88019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88019db70 con 0x7fa880105ed0 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88019dcb0 con 0x7fa880100c90 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.937+0000 7fa8870e5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88019de40 con 0x7fa8801035b0 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 0x7fa88019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 0x7fa88019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34450/0 (socket says 172.21.15.31:34450) 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 -- 172.21.15.31:0/160596089 learned_addr learned my addr 172.21.15.31:0/160596089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa87ffff700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 0x7fa880196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa884e81700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa880100c90 0x7fa880195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 -- 172.21.15.31:0/160596089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 msgr2=0x7fa880196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 0x7fa880196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 -- 172.21.15.31:0/160596089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa880100c90 msgr2=0x7fa880195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa880100c90 0x7fa880195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 -- 172.21.15.31:0/160596089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8740095f0 con 0x7fa880105ed0 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa87ffff700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 0x7fa880196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa885682700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 0x7fa88019d5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa878007ee0 tx=0x7fa87800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878017070 con 0x7fa880105ed0 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87800f040 con 0x7fa880105ed0 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88010bbc0 con 0x7fa880105ed0 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.938+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878005740 con 0x7fa880105ed0 2023-12-14T18:25:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.939+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88010ba30 con 0x7fa880105ed0 2023-12-14T18:25:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.940+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8800fc260 con 0x7fa880105ed0 2023-12-14T18:25:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.941+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa8780058a0 con 0x7fa880105ed0 2023-12-14T18:25:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.941+0000 7fa87dffb700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8700a1150 0x7fa8700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.941+0000 7fa884e81700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8700a1150 0x7fa8700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.942+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa8780cc3c0 con 0x7fa880105ed0 2023-12-14T18:25:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.942+0000 7fa884e81700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8700a1150 0x7fa8700a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b5c0 tx=0x7fa874005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:59.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:25:59.946+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa878095920 con 0x7fa880105ed0 2023-12-14T18:26:00.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.201+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa880061970 con 0x7fa880105ed0 2023-12-14T18:26:00.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.202+0000 7fa87dffb700 1 -- 172.21.15.31:0/160596089 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10753 (secure 0 0 0) 0x7fa878098f40 con 0x7fa880105ed0 2023-12-14T18:26:00.204 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:00.204 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:25:59.779686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.204+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8700a1150 msgr2=0x7fa8700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.204+0000 7fa8870e5700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8700a1150 0x7fa8700a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa87400b5c0 tx=0x7fa874005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.204+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 msgr2=0x7fa88019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.204+0000 7fa8870e5700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 0x7fa88019d5a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa878007ee0 tx=0x7fa87800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 shutdown_connections 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa880105ed0 0x7fa88019d5a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa880100c90 0x7fa880195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa8700a1150 0x7fa8700a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 --2- 172.21.15.31:0/160596089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8801035b0 0x7fa880196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 >> 172.21.15.31:0/160596089 conn(0x7fa8800fa6f0 msgr2=0x7fa8801018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 shutdown_connections 2023-12-14T18:26:00.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.205+0000 7fa8870e5700 1 -- 172.21.15.31:0/160596089 wait complete. 2023-12-14T18:26:00.208 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-12-14T18:26:00.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 -- 172.21.15.31:0/4199775759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 msgr2=0x7f43641030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 --2- 172.21.15.31:0/4199775759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641030b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4358009990 tx=0x7f4358009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 -- 172.21.15.31:0/4199775759 shutdown_connections 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 --2- 172.21.15.31:0/4199775759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4364105f10 0x7f4364108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 --2- 172.21.15.31:0/4199775759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641030b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 --2- 172.21.15.31:0/4199775759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43641035f0 0x7f43641059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 -- 172.21.15.31:0/4199775759 >> 172.21.15.31:0/4199775759 conn(0x7f43640fa750 msgr2=0x7f43640fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 -- 172.21.15.31:0/4199775759 shutdown_connections 2023-12-14T18:26:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 -- 172.21.15.31:0/4199775759 wait complete. 2023-12-14T18:26:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.361+0000 7f43693e7700 1 Processor -- start 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 -- start start 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43641035f0 0x7f4364198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4364105f10 0x7f436419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436419fd60 con 0x7f4364100cd0 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436419fea0 con 0x7f43641035f0 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43693e7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43641a0030 con 0x7f4364105f10 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34462/0 (socket says 172.21.15.31:34462) 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 -- 172.21.15.31:0/3037153616 learned_addr learned my addr 172.21.15.31:0/3037153616 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f43627fc700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43641035f0 0x7f4364198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 -- 172.21.15.31:0/3037153616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4364105f10 msgr2=0x7f436419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4364105f10 0x7f436419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 -- 172.21.15.31:0/3037153616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43641035f0 msgr2=0x7f4364198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43641035f0 0x7f4364198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 -- 172.21.15.31:0/3037153616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580095f0 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.362+0000 7f4362ffd700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641981e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4358004740 tx=0x7f4358009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.363+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435801d070 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.363+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4358004830 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.363+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436410bc20 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.363+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358005480 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.363+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436410ba90 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.364+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4364107ac0 con 0x7f4364100cd0 2023-12-14T18:26:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.365+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f435800fb40 con 0x7f4364100cd0 2023-12-14T18:26:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.366+0000 7f434bfff700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f434c0a1150 0x7f434c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.366+0000 7f43627fc700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f434c0a1150 0x7f434c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.366+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f43580ce220 con 0x7f4364100cd0 2023-12-14T18:26:00.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.367+0000 7f43627fc700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f434c0a1150 0x7f434c0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4354009b30 tx=0x7f4354009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:00.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.371+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4358097830 con 0x7f4364100cd0 2023-12-14T18:26:00.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.578+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f436402cfe0 con 0x7f4364100cd0 2023-12-14T18:26:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.580+0000 7f434bfff700 1 -- 172.21.15.31:0/3037153616 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f435809ae50 con 0x7f4364100cd0 2023-12-14T18:26:00.583 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:00.583 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f434c0a1150 msgr2=0x7f434c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f434c0a1150 0x7f434c0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4354009b30 tx=0x7f4354009260 comp rx=0 tx=0).stop 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 msgr2=0x7f43641981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641981e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4358004740 tx=0x7f4358009580 comp rx=0 tx=0).stop 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 shutdown_connections 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f434c0a1150 0x7f434c0a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4364105f10 0x7f436419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4364100cd0 0x7f43641981e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 --2- 172.21.15.31:0/3037153616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43641035f0 0x7f4364198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.582+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 >> 172.21.15.31:0/3037153616 conn(0x7f43640fa750 msgr2=0x7f43640ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.583+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 shutdown_connections 2023-12-14T18:26:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.583+0000 7f43693e7700 1 -- 172.21.15.31:0/3037153616 wait complete. 2023-12-14T18:26:00.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/482102856' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-14T18:26:00.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: osdmap e84: 12 total, 12 up, 12 in 2023-12-14T18:26:00.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: pgmap v230: 129 pgs: 10 creating+peering, 86 active+clean, 33 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-12-14T18:26:00.597 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3399189434' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-14T18:26:00.598 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: osdmap e85: 12 total, 12 up, 12 in 2023-12-14T18:26:00.598 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3399189434' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-14T18:26:00.598 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:00.598 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/160596089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:00.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 -- 172.21.15.31:0/1377954760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 msgr2=0x7ff0b0101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 --2- 172.21.15.31:0/1377954760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0101870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009990 tx=0x7ff0a0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 -- 172.21.15.31:0/1377954760 shutdown_connections 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 --2- 172.21.15.31:0/1377954760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b0101db0 0x7ff0b0108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 --2- 172.21.15.31:0/1377954760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b0100230 0x7ff0b0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 --2- 172.21.15.31:0/1377954760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0101870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 -- 172.21.15.31:0/1377954760 >> 172.21.15.31:0/1377954760 conn(0x7ff0b00fb810 msgr2=0x7ff0b00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 -- 172.21.15.31:0/1377954760 shutdown_connections 2023-12-14T18:26:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.735+0000 7ff0b7934700 1 -- 172.21.15.31:0/1377954760 wait complete. 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 Processor -- start 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 -- start start 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b0100230 0x7ff0b00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b0101db0 0x7ff0b0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0073bc0 con 0x7ff0b0101420 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0073d30 con 0x7ff0b0101db0 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b7934700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b019ffd0 con 0x7ff0b0100230 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b4ecf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b4ecf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34466/0 (socket says 172.21.15.31:34466) 2023-12-14T18:26:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.736+0000 7ff0b4ecf700 1 -- 172.21.15.31:0/3947449843 learned_addr learned my addr 172.21.15.31:0/3947449843 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b56d0700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b0100230 0x7ff0b00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b5ed1700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b0101db0 0x7ff0b0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b4ecf700 1 -- 172.21.15.31:0/3947449843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b0100230 msgr2=0x7ff0b00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b4ecf700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b0100230 0x7ff0b00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b4ecf700 1 -- 172.21.15.31:0/3947449843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b0101db0 msgr2=0x7ff0b0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b4ecf700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b0101db0 0x7ff0b0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b4ecf700 1 -- 172.21.15.31:0/3947449843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00095f0 con 0x7ff0b0101420 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b4ecf700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0072ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0004760 tx=0x7ff0a0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a001d070 con 0x7ff0b0101420 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0a0004cf0 con 0x7ff0b0101420 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b010bb90 con 0x7ff0b0101420 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0005280 con 0x7ff0b0101420 2023-12-14T18:26:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.737+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b010ba00 con 0x7ff0b0101420 2023-12-14T18:26:00.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/482102856' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-14T18:26:00.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: osdmap e84: 12 total, 12 up, 12 in 2023-12-14T18:26:00.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: pgmap v230: 129 pgs: 10 creating+peering, 86 active+clean, 33 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-12-14T18:26:00.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3399189434' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-14T18:26:00.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: osdmap e85: 12 total, 12 up, 12 in 2023-12-14T18:26:00.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3399189434' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-14T18:26:00.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:00.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/160596089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.738+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b0107180 con 0x7ff0b0101420 2023-12-14T18:26:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.739+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff0a00053e0 con 0x7ff0b0101420 2023-12-14T18:26:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.740+0000 7ff0a67fc700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff09c0a1150 0x7ff09c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.740+0000 7ff0b56d0700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff09c0a1150 0x7ff09c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.740+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff0a00cdbf0 con 0x7ff0b0101420 2023-12-14T18:26:00.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.742+0000 7ff0b56d0700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff09c0a1150 0x7ff09c0a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0101280 tx=0x7ff0ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:00.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.745+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff0a005b730 con 0x7ff0b0101420 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/482102856' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: osdmap e84: 12 total, 12 up, 12 in 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: pgmap v230: 129 pgs: 10 creating+peering, 86 active+clean, 33 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3399189434' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: osdmap e85: 12 total, 12 up, 12 in 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3399189434' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:00.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/160596089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:00.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.957+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0b002cfe0 con 0x7ff0b0101420 2023-12-14T18:26:00.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.959+0000 7ff0a67fc700 1 -- 172.21.15.31:0/3947449843 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7ff0a0097180 con 0x7ff0b0101420 2023-12-14T18:26:00.962 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:00.962 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:00.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff09c0a1150 msgr2=0x7ff09c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff09c0a1150 0x7ff09c0a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0101280 tx=0x7ff0ac00b410 comp rx=0 tx=0).stop 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 msgr2=0x7ff0b0072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0072ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0004760 tx=0x7ff0a0004790 comp rx=0 tx=0).stop 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 shutdown_connections 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff0b0101db0 0x7ff0b0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff09c0a1150 0x7ff09c0a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0b0100230 0x7ff0b00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 --2- 172.21.15.31:0/3947449843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b0101420 0x7ff0b0072ee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.961+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 >> 172.21.15.31:0/3947449843 conn(0x7ff0b00fb810 msgr2=0x7ff0b00fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.962+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 shutdown_connections 2023-12-14T18:26:00.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:00.962+0000 7ff0b7934700 1 -- 172.21.15.31:0/3947449843 wait complete. 2023-12-14T18:26:00.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.126+0000 7fe982110700 1 -- 172.21.15.31:0/1070719175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c100ce0 msgr2=0x7fe97c1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.126+0000 7fe982110700 1 --2- 172.21.15.31:0/1070719175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c100ce0 0x7fe97c1030c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009990 tx=0x7fe96c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 -- 172.21.15.31:0/1070719175 shutdown_connections 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 --2- 172.21.15.31:0/1070719175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe97c105f20 0x7fe97c108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 --2- 172.21.15.31:0/1070719175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c100ce0 0x7fe97c1030c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 --2- 172.21.15.31:0/1070719175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c103600 0x7fe97c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 -- 172.21.15.31:0/1070719175 >> 172.21.15.31:0/1070719175 conn(0x7fe97c0fa740 msgr2=0x7fe97c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 -- 172.21.15.31:0/1070719175 shutdown_connections 2023-12-14T18:26:01.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 -- 172.21.15.31:0/1070719175 wait complete. 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 Processor -- start 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.127+0000 7fe982110700 1 -- start start 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe982110700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c100ce0 0x7fe97c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe982110700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe97c103600 0x7fe97c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe982110700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 0x7fe97c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe982110700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c19db60 con 0x7fe97c105f20 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe982110700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c19dca0 con 0x7fe97c100ce0 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe982110700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c19de30 con 0x7fe97c103600 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 0x7fe97c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 0x7fe97c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34494/0 (socket says 172.21.15.31:34494) 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 -- 172.21.15.31:0/1396341002 learned_addr learned my addr 172.21.15.31:0/1396341002 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97b7fe700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c100ce0 0x7fe97c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97affd700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe97c103600 0x7fe97c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 -- 172.21.15.31:0/1396341002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe97c103600 msgr2=0x7fe97c196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe97c103600 0x7fe97c196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 -- 172.21.15.31:0/1396341002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c100ce0 msgr2=0x7fe97c195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c100ce0 0x7fe97c195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 -- 172.21.15.31:0/1396341002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0095f0 con 0x7fe97c105f20 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97b7fe700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c100ce0 0x7fe97c195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe97bfff700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 0x7fe97c19d590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe97400b970 tx=0x7fe97400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.128+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97400c9b0 con 0x7fe97c105f20 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.129+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe974014440 con 0x7fe97c105f20 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.129+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe97c10bc20 con 0x7fe97c105f20 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.129+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740075d0 con 0x7fe97c105f20 2023-12-14T18:26:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.129+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c10ba90 con 0x7fe97c105f20 2023-12-14T18:26:01.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.130+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c0fc2b0 con 0x7fe97c105f20 2023-12-14T18:26:01.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.131+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe9740145b0 con 0x7fe97c105f20 2023-12-14T18:26:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.131+0000 7fe978ff9700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe9680a1150 0x7fe9680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.132+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe974006e40 con 0x7fe97c105f20 2023-12-14T18:26:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.132+0000 7fe97b7fe700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe9680a1150 0x7fe9680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.132+0000 7fe97b7fe700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe9680a1150 0x7fe9680a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006010 tx=0x7fe96c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.136+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe97404e730 con 0x7fe97c105f20 2023-12-14T18:26:01.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.393+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe97c02cfe0 con 0x7fe97c105f20 2023-12-14T18:26:01.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.394+0000 7fe978ff9700 1 -- 172.21.15.31:0/1396341002 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10753 (secure 0 0 0) 0x7fe974095820 con 0x7fe97c105f20 2023-12-14T18:26:01.395 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:01.395 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:25:59.779686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-14T18:26:01.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe9680a1150 msgr2=0x7fe9680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe9680a1150 0x7fe9680a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006010 tx=0x7fe96c00b540 comp rx=0 tx=0).stop 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 msgr2=0x7fe97c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 0x7fe97c19d590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe97400b970 tx=0x7fe97400bd30 comp rx=0 tx=0).stop 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 shutdown_connections 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe97c105f20 0x7fe97c19d590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe97c100ce0 0x7fe97c195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe9680a1150 0x7fe9680a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 --2- 172.21.15.31:0/1396341002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe97c103600 0x7fe97c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.396+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 >> 172.21.15.31:0/1396341002 conn(0x7fe97c0fa740 msgr2=0x7fe97c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.397+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 shutdown_connections 2023-12-14T18:26:01.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.397+0000 7fe982110700 1 -- 172.21.15.31:0/1396341002 wait complete. 2023-12-14T18:26:01.399 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-12-14T18:26:01.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 -- 172.21.15.31:0/3136240673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 msgr2=0x7f9da0108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3136240673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0108680 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009990 tx=0x7f9d98009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 -- 172.21.15.31:0/3136240673 shutdown_connections 2023-12-14T18:26:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3136240673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0108680 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3136240673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da0100240 0x7f9da0100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3136240673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9da0101430 0x7f9da0101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 -- 172.21.15.31:0/3136240673 >> 172.21.15.31:0/3136240673 conn(0x7f9da00fb820 msgr2=0x7f9da00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 -- 172.21.15.31:0/3136240673 shutdown_connections 2023-12-14T18:26:01.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.547+0000 7f9da48c4700 1 -- 172.21.15.31:0/3136240673 wait complete. 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 Processor -- start 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 -- start start 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9da0100240 0x7f9da00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da0101430 0x7f9da0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0073bc0 con 0x7f9da0101dc0 2023-12-14T18:26:01.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da0073d30 con 0x7f9da0100240 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9da48c4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da019ffb0 con 0x7f9da0101430 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9d9ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9d9ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34512/0 (socket says 172.21.15.31:34512) 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.548+0000 7f9d9ed9d700 1 -- 172.21.15.31:0/3633804768 learned_addr learned my addr 172.21.15.31:0/3633804768 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9dd9b700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da0101430 0x7f9da0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9e59c700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9da0100240 0x7f9da00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9ed9d700 1 -- 172.21.15.31:0/3633804768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da0101430 msgr2=0x7f9da0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9ed9d700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da0101430 0x7f9da0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9ed9d700 1 -- 172.21.15.31:0/3633804768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9da0100240 msgr2=0x7f9da00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9ed9d700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9da0100240 0x7f9da00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9ed9d700 1 -- 172.21.15.31:0/3633804768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d980095f0 con 0x7f9da0101dc0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d9ed9d700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0073420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9d98004730 tx=0x7f9d98004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801d070 con 0x7f9da0101dc0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d98004c90 con 0x7f9da0101dc0 2023-12-14T18:26:01.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da010bc20 con 0x7f9da0101dc0 2023-12-14T18:26:01.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98003d00 con 0x7f9da0101dc0 2023-12-14T18:26:01.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.549+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da010ba90 con 0x7f9da0101dc0 2023-12-14T18:26:01.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.550+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da01070d0 con 0x7f9da0101dc0 2023-12-14T18:26:01.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.551+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9d980047d0 con 0x7f9da0101dc0 2023-12-14T18:26:01.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.552+0000 7f9d977fe700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d8c0a1150 0x7f9d8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.552+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9d980cda30 con 0x7f9da0101dc0 2023-12-14T18:26:01.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.552+0000 7f9d9e59c700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d8c0a1150 0x7f9d8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.552+0000 7f9d9e59c700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d8c0a1150 0x7f9d8c0a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9da0101290 tx=0x7f9d90009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.556+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d98097020 con 0x7f9da0101dc0 2023-12-14T18:26:01.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:01 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3037153616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:01.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:01 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3947449843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:01.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:01 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3037153616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:01.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:01 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3947449843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:01.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.770+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da002cfe0 con 0x7f9da0101dc0 2023-12-14T18:26:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.771+0000 7f9d977fe700 1 -- 172.21.15.31:0/3633804768 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f9d9809a640 con 0x7f9da0101dc0 2023-12-14T18:26:01.773 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:01.773 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d8c0a1150 msgr2=0x7f9d8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d8c0a1150 0x7f9d8c0a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9da0101290 tx=0x7f9d90009040 comp rx=0 tx=0).stop 2023-12-14T18:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 msgr2=0x7f9da0073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0073420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9d98004730 tx=0x7f9d98004760 comp rx=0 tx=0).stop 2023-12-14T18:26:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 shutdown_connections 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9da0101dc0 0x7f9da0073420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9da0100240 0x7f9da00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d8c0a1150 0x7f9d8c0a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 --2- 172.21.15.31:0/3633804768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da0101430 0x7f9da0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 >> 172.21.15.31:0/3633804768 conn(0x7f9da00fb820 msgr2=0x7f9da00fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 shutdown_connections 2023-12-14T18:26:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.774+0000 7f9da48c4700 1 -- 172.21.15.31:0/3633804768 wait complete. 2023-12-14T18:26:01.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-14T18:25:55.286836+0000', 'modified': '2023-12-14T18:25:59.779686+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24538}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.204:6839/1902825357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1902825357}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1902825357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-14T18:26:01.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-14T18:26:01.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-14T18:26:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:01 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3037153616' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:01 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3947449843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 -- 172.21.15.31:0/3063508466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c300ff150 msgr2=0x7f9c300ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 --2- 172.21.15.31:0/3063508466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c300ff150 0x7f9c300ff560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009990 tx=0x7f9c20009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 -- 172.21.15.31:0/3063508466 shutdown_connections 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 --2- 172.21.15.31:0/3063508466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c30100cd0 0x7f9c30107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 --2- 172.21.15.31:0/3063508466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c300ff150 0x7f9c300ff560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 --2- 172.21.15.31:0/3063508466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c30100340 0x7f9c30100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 -- 172.21.15.31:0/3063508466 >> 172.21.15.31:0/3063508466 conn(0x7f9c300fa750 msgr2=0x7f9c300fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 -- 172.21.15.31:0/3063508466 shutdown_connections 2023-12-14T18:26:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.927+0000 7f9c34842700 1 -- 172.21.15.31:0/3063508466 wait complete. 2023-12-14T18:26:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 Processor -- start 2023-12-14T18:26:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 -- start start 2023-12-14T18:26:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c300ff150 0x7f9c301981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 0x7f9c301986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c30100cd0 0x7f9c3019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3019fd30 con 0x7f9c30100340 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3019fe70 con 0x7f9c30100cd0 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c34842700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301a0000 con 0x7f9c300ff150 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 0x7f9c301986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 0x7f9c301986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34528/0 (socket says 172.21.15.31:34528) 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/2598250644 learned_addr learned my addr 172.21.15.31:0/2598250644 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c30100cd0 0x7f9c3019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c300ff150 0x7f9c301981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/2598250644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c300ff150 msgr2=0x7f9c301981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c300ff150 0x7f9c301981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/2598250644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c30100cd0 msgr2=0x7f9c3019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c30100cd0 0x7f9c3019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.928+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/2598250644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c200095f0 con 0x7f9c30100340 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.929+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 0x7f9c301986f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400d980 tx=0x7f9c2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.929+0000 7f9c1f7fe700 1 -- 172.21.15.31:0/2598250644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24004220 con 0x7f9c30100340 2023-12-14T18:26:01.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.929+0000 7f9c1f7fe700 1 -- 172.21.15.31:0/2598250644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c24007640 con 0x7f9c30100340 2023-12-14T18:26:01.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.929+0000 7f9c1f7fe700 1 -- 172.21.15.31:0/2598250644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24005690 con 0x7f9c30100340 2023-12-14T18:26:01.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.929+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3010aac0 con 0x7f9c30100340 2023-12-14T18:26:01.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.929+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3010a930 con 0x7f9c30100340 2023-12-14T18:26:01.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.931+0000 7f9c1f7fe700 1 -- 172.21.15.31:0/2598250644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9c24004750 con 0x7f9c30100340 2023-12-14T18:26:01.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.932+0000 7f9c1f7fe700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c180a11d0 0x7f9c180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.932+0000 7f9c1f7fe700 1 -- 172.21.15.31:0/2598250644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c240cc900 con 0x7f9c30100340 2023-12-14T18:26:01.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.932+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c180a11d0 0x7f9c180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.932+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c180a11d0 0x7f9c180a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000b5c0 tx=0x7f9c20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 Processor -- start 2023-12-14T18:26:01.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 -- start start 2023-12-14T18:26:01.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb34b0 0x55a32aeb38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aeb4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 0x55a32aeb5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ad9c590 con 0x55a32aeb4cb0 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ae0b820 con 0x55a32aeb3e00 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ae04cb0 con 0x55a32aeb34b0 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c2ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 0x55a32aeb5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c2ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 0x55a32aeb5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34532/0 (socket says 172.21.15.31:34532) 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c2ed9d700 1 -- 172.21.15.31:0/2218699875 learned_addr learned my addr 172.21.15.31:0/2218699875 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.933+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aeb4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb34b0 0x55a32aeb38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2ed9d700 1 -- 172.21.15.31:0/2218699875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb34b0 msgr2=0x55a32aeb38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb34b0 0x55a32aeb38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2ed9d700 1 -- 172.21.15.31:0/2218699875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 msgr2=0x55a32aeb4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aeb4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2ed9d700 1 -- 172.21.15.31:0/2218699875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a32ac86360 con 0x55a32aeb4cb0 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aeb4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:01.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 0x55a32aeb5120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c28003fa0 tx=0x7f9c28009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614a48faa2d2f3f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c1cff9700 1 -- 172.21.15.31:0/2218699875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28019070 con 0x55a32aeb4cb0 2023-12-14T18:26:01.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c1cff9700 1 -- 172.21.15.31:0/2218699875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2800f040 con 0x55a32aeb4cb0 2023-12-14T18:26:01.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.934+0000 7f9c1cff9700 1 -- 172.21.15.31:0/2218699875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c28007a00 con 0x55a32aeb4cb0 2023-12-14T18:26:01.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2218699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 msgr2=0x55a32aeb5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 0x55a32aeb5120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9c28003fa0 tx=0x7f9c28009ba0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2218699875 shutdown_connections 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb4cb0 0x55a32aeb5120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb34b0 0x55a32aeb38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2218699875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aeb4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2218699875 >> 172.21.15.31:0/2218699875 conn(0x55a32aea8c50 msgr2=0x55a32aeae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2218699875 shutdown_connections 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2218699875 wait complete. 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.935+0000 7f9c409d0b80 1 -- start start 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 Processor -- start 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aebd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aebd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ada0f40 con 0x55a32aeb34b0 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ada0be0 con 0x55a32aeb3e00 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ac88650 con 0x55a32aeb4cb0 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c409d0b80 10 client.0 _ll_register_callbacks cb 0x55a32adc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aebd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aebd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34544/0 (socket says 172.21.15.31:34544) 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 -- 172.21.15.31:0/2555581918 learned_addr learned my addr 172.21.15.31:0/2555581918 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aebd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 -- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 msgr2=0x55a32aec4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 -- 172.21.15.31:0/2555581918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 msgr2=0x55a32aebd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aebd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 -- 172.21.15.31:0/2555581918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c28003bd0 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aebd0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005b40 tx=0x7f9c20009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89636cb2975f1b91 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.936+0000 7f9c15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c15ffb700 1 -- 172.21.15.31:0/2555581918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20003fc0 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a32aea6260 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c15ffb700 1 -- 172.21.15.31:0/2555581918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c20004120 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c15ffb700 1 -- 172.21.15.31:0/2555581918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c20004500 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 10 client.14721 fetch_fsmap learned FSMap version 17 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55a32adcb120 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c15ffb700 1 -- 172.21.15.31:0/2555581918 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a32aea6260 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c15ffb700 1 -- 172.21.15.31:0/2555581918 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9c2002f070 con 0x55a32aeb34b0 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 10 client.14721 fetch_fsmap finished waiting for FSMap version 17 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 10 client.14721 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 20 client.14721 populate_metadata read hostname 'smithi031' 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32aec83a0 0x55a32aec7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 4 client.14721 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.937+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a32adfdc90 con 0x55a32aec83a0 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.938+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32aec83a0 0x55a32aec7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.938+0000 7f9c15ffb700 10 client.14721 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.938+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32aec83a0 0x55a32aec7780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e8fa9ec1994025 server_cookie=89df837f3ae858ad in_seq=0 out_seq=0 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c15ffb700 1 -- 172.21.15.31:0/2555581918 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a32adfdc90 con 0x55a32aec83a0 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c15ffb700 10 client.14721 handle_command_reply: tid=0 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 client.14721 shutdown 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 msgr2=0x55a32aebd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aebd0b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005b40 tx=0x7f9c20009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 shutdown_connections 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32aec83a0 0x55a32aec7780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aebd0b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c15ffb700 0 client.14721 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/2555581918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aebd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 >> 172.21.15.31:0/2555581918 conn(0x55a32aea8c50 msgr2=0x55a32aeacee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 shutdown_connections 2023-12-14T18:26:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 1 -- 172.21.15.31:0/2555581918 wait complete. 2023-12-14T18:26:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.939+0000 7f9c409d0b80 20 client.14721 trim_cache size 0 max 16384 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 Processor -- start 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 -- start start 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aec7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aec7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32adc3290 con 0x55a32aeb34b0 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32ad10c50 con 0x55a32aeb3e00 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32aea8820 con 0x55a32aeb4cb0 2023-12-14T18:26:01.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aec7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aec7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34552/0 (socket says 172.21.15.31:34552) 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2e59c700 1 -- 172.21.15.31:0/226946943 learned_addr learned my addr 172.21.15.31:0/226946943 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aec7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2e59c700 1 -- 172.21.15.31:0/226946943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 msgr2=0x55a32aec45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.958+0000 7f9c2e59c700 1 -- 172.21.15.31:0/226946943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 msgr2=0x55a32aec7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aec7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c2e59c700 1 -- 172.21.15.31:0/226946943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a32aeb5350 con 0x55a32aeb34b0 2023-12-14T18:26:01.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aec7780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c20006010 tx=0x7f9c20009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12990574ae0685e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c1d7fa700 1 -- 172.21.15.31:0/226946943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000f460 con 0x55a32aeb34b0 2023-12-14T18:26:01.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c1d7fa700 1 -- 172.21.15.31:0/226946943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c20028070 con 0x55a32aeb34b0 2023-12-14T18:26:01.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c1d7fa700 1 -- 172.21.15.31:0/226946943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004cb0 con 0x55a32aeb34b0 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 -- 172.21.15.31:0/226946943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 msgr2=0x55a32aec7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aec7780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c20006010 tx=0x7f9c20009290 comp rx=0 tx=0).stop 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 -- 172.21.15.31:0/226946943 shutdown_connections 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aec45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb34b0 0x55a32aec7780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/226946943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb3e00 0x55a32aec7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.959+0000 7f9c409d0b80 1 -- 172.21.15.31:0/226946943 >> 172.21.15.31:0/226946943 conn(0x55a32aea8c50 msgr2=0x55a32aebb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 -- 172.21.15.31:0/226946943 shutdown_connections 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 -- 172.21.15.31:0/226946943 wait complete. 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 -- start start 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 Processor -- start 2023-12-14T18:26:01.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb34b0 0x55a32aebeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 0x55a32aec4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aebf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32aebfac0 con 0x55a32aeb3e00 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32af1c260 con 0x55a32aeb34b0 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a32af1c3a0 con 0x55a32aeb4cb0 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c409d0b80 10 client.0 _ll_register_callbacks cb 0x55a32adc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c2dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 0x55a32aec4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c2dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 0x55a32aec4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34560/0 (socket says 172.21.15.31:34560) 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/3024125850 learned_addr learned my addr 172.21.15.31:0/3024125850 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c2ed9d700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aebf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.960+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb34b0 0x55a32aebeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 msgr2=0x55a32aebf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aebf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/3024125850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb34b0 msgr2=0x55a32aebeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb34b0 0x55a32aebeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2dd9b700 1 -- 172.21.15.31:0/3024125850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a32adcb120 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb34b0 0x55a32aebeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c2dd9b700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 0x55a32aec4410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9c24000c00 tx=0x7f9c24095f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5887cc0d14ad042a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3024125850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2409daf0 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3024125850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c240a1f10 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c2800f040 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3024125850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c24004380 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c409d0b80 10 client.14727 fetch_fsmap learned FSMap version 17 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55a32ac32b10 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3024125850 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2407c020 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.961+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3024125850 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9c240a1a80 con 0x55a32aeb3e00 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 10 client.14727 fetch_fsmap finished waiting for FSMap version 17 2023-12-14T18:26:01.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 10 client.14727 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 20 client.14727 populate_metadata read hostname 'smithi031' 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32af21030 0x55a32af1da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 4 client.14727 mds_command: new command op to 24538 tid=0[{"prefix": "status"}] 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a32adfd9b0 con 0x55a32af21030 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32af21030 0x55a32af1da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c2e59c700 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32af21030 0x55a32af1da90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6810b0249ef74b0a server_cookie=654921fa33887067 in_seq=0 out_seq=0 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c15ffb700 10 client.14727 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c15ffb700 1 -- 172.21.15.31:0/3024125850 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55a32adfd9b0 con 0x55a32af21030 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c15ffb700 10 client.14727 handle_command_reply: tid=0 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.962+0000 7f9c409d0b80 1 client.14727 shutdown 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 msgr2=0x55a32aec4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 0x55a32aec4410 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9c24000c00 tx=0x7f9c24095f30 comp rx=0 tx=0).stop 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 shutdown_connections 2023-12-14T18:26:01.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a32af21030 0x55a32af1da90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a32aeb4cb0 0x55a32aebf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a32aeb34b0 0x55a32aebeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c15ffb700 0 client.14727 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 --2- 172.21.15.31:0/3024125850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a32aeb3e00 0x55a32aec4410 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 >> 172.21.15.31:0/3024125850 conn(0x55a32aea8c50 msgr2=0x55a32aec7780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 shutdown_connections 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 1 -- 172.21.15.31:0/3024125850 wait complete. 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c409d0b80 20 client.14727 trim_cache size 0 max 16384 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24538, 2023-12-14T18:26:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 6.6730701979999996, 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 17, 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 85, 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 18.256141943999999 2023-12-14T18:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c180a11d0 msgr2=0x7f9c180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c180a11d0 0x7f9c180a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000b5c0 tx=0x7f9c20005960 comp rx=0 tx=0).stop 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 msgr2=0x7f9c301986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 0x7f9c301986f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400d980 tx=0x7f9c2400dc90 comp rx=0 tx=0).stop 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 shutdown_connections 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c30100cd0 0x7f9c3019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c180a11d0 0x7f9c180a3680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c300ff150 0x7f9c301981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 --2- 172.21.15.31:0/2598250644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c30100340 0x7f9c301986f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 >> 172.21.15.31:0/2598250644 conn(0x7f9c300fa750 msgr2=0x7f9c30068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.963+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 shutdown_connections 2023-12-14T18:26:01.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:01.964+0000 7f9c34842700 1 -- 172.21.15.31:0/2598250644 wait complete. 2023-12-14T18:26:01.978 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-14T18:26:01.978 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:01.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph fs ls 2023-12-14T18:26:02.320 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:02.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:02 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1396341002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:02.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:02 smithi031 ceph-mon[42085]: pgmap v232: 129 pgs: 2 creating+activating, 1 creating+peering, 112 active+clean, 14 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 4 op/s 2023-12-14T18:26:02.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:02 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3633804768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:02.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:02 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1396341002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:02.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:02 smithi176 ceph-mon[43506]: pgmap v232: 129 pgs: 2 creating+activating, 1 creating+peering, 112 active+clean, 14 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 4 op/s 2023-12-14T18:26:02.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:02 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3633804768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:02.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:02 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1396341002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:02.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:02 smithi204 ceph-mon[43866]: pgmap v232: 129 pgs: 2 creating+activating, 1 creating+peering, 112 active+clean, 14 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 4 op/s 2023-12-14T18:26:02.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:02 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3633804768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:03.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 -- 172.21.15.31:0/179040209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c104500 msgr2=0x7fa92c10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 --2- 172.21.15.31:0/179040209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c104500 0x7fa92c10c080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa92400b0d0 tx=0x7fa92400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 -- 172.21.15.31:0/179040209 shutdown_connections 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 --2- 172.21.15.31:0/179040209 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c104500 0x7fa92c10c080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 --2- 172.21.15.31:0/179040209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa92c0ff460 0x7fa92c0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 --2- 172.21.15.31:0/179040209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c0ffe00 0x7fa92c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 -- 172.21.15.31:0/179040209 >> 172.21.15.31:0/179040209 conn(0x7fa92c0770a0 msgr2=0x7fa92c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 -- 172.21.15.31:0/179040209 shutdown_connections 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.458+0000 7fa9336f5700 1 -- 172.21.15.31:0/179040209 wait complete. 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 Processor -- start 2023-12-14T18:26:03.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 -- start start 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa92c0ff460 0x7fa92c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 0x7fa92c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c104500 0x7fa92c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19b590 con 0x7fa92c0ffe00 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19b700 con 0x7fa92c0ff460 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa9336f5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c19b8c0 con 0x7fa92c104500 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa930c90700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 0x7fa92c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa930c90700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 0x7fa92c196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34576/0 (socket says 172.21.15.31:34576) 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa930c90700 1 -- 172.21.15.31:0/2258520944 learned_addr learned my addr 172.21.15.31:0/2258520944 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa931491700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa92c0ff460 0x7fa92c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.459+0000 7fa931c92700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c104500 0x7fa92c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa930c90700 1 -- 172.21.15.31:0/2258520944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c104500 msgr2=0x7fa92c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:03.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa930c90700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c104500 0x7fa92c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa930c90700 1 -- 172.21.15.31:0/2258520944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa92c0ff460 msgr2=0x7fa92c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa930c90700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa92c0ff460 0x7fa92c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa930c90700 1 -- 172.21.15.31:0/2258520944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa924009d00 con 0x7fa92c0ffe00 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa930c90700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 0x7fa92c196b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00d980 tx=0x7fa91c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00f890 con 0x7fa92c0ffe00 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa91c010460 con 0x7fa92c0ffe00 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c10f290 con 0x7fa92c0ffe00 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00e610 con 0x7fa92c0ffe00 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.460+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c10f100 con 0x7fa92c0ffe00 2023-12-14T18:26:03.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.461+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92c10f420 con 0x7fa92c0ffe00 2023-12-14T18:26:03.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.463+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa91c0105d0 con 0x7fa92c0ffe00 2023-12-14T18:26:03.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.463+0000 7fa9227fc700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9180a1150 0x7fa9180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:03.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.464+0000 7fa931491700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9180a1150 0x7fa9180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:03.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.464+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa91c0cc4b0 con 0x7fa92c0ffe00 2023-12-14T18:26:03.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.464+0000 7fa931491700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9180a1150 0x7fa9180a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa928005950 tx=0x7fa92800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:03.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.467+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa91c0959b0 con 0x7fa92c0ffe00 2023-12-14T18:26:03.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.720+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa92c066ea0 con 0x7fa92c0ffe00 2023-12-14T18:26:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.720+0000 7fa9227fc700 1 -- 172.21.15.31:0/2258520944 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa91c020070 con 0x7fa92c0ffe00 2023-12-14T18:26:03.721 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-14T18:26:03.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9180a1150 msgr2=0x7fa9180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:03.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9180a1150 0x7fa9180a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa928005950 tx=0x7fa92800b410 comp rx=0 tx=0).stop 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 msgr2=0x7fa92c196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 0x7fa92c196b10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00d980 tx=0x7fa91c00dc90 comp rx=0 tx=0).stop 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 shutdown_connections 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9180a1150 0x7fa9180a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa92c104500 0x7fa92c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa92c0ff460 0x7fa92c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 --2- 172.21.15.31:0/2258520944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa92c0ffe00 0x7fa92c196b10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.722+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 >> 172.21.15.31:0/2258520944 conn(0x7fa92c0770a0 msgr2=0x7fa92c068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.723+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 shutdown_connections 2023-12-14T18:26:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:03.723+0000 7fa9336f5700 1 -- 172.21.15.31:0/2258520944 wait complete. 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-14T18:26:04.162 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-14T18:26:04.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.162 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-12-14T18:26:04.185 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-14T18:26:04.185 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41931sec preferred_lft 41931sec 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-12-14T18:26:04.186 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:26:04.187 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:26:04.187 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:04.187 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-12-14T18:26:04.187 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-14T18:26:04.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.187 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:26:04.187 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:26:04.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-12-14T18:26:04.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-12-14T18:26:04.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-12-14T18:26:04.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-14T18:26:04.188 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:26:04.343 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:04 smithi031 ceph-mon[42085]: pgmap v233: 129 pgs: 2 creating+activating, 1 creating+peering, 126 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-12-14T18:26:04.343 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:04 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2258520944' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:26:04.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.390 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-14T18:26:04.421 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-12-14T18:26:04.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.423 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-12-14T18:26:04.498 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-12-14T18:26:04.498 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-14T18:26:04.499 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-14T18:26:04.499 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-14T18:26:04.499 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-14T18:26:04.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.500 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:26:04.500 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:26:04.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-14T18:26:04.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-14T18:26:04.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-14T18:26:04.500 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:26:04.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:04 smithi204 ceph-mon[43866]: pgmap v233: 129 pgs: 2 creating+activating, 1 creating+peering, 126 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-12-14T18:26:04.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:04 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2258520944' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:26:04.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:04 smithi176 ceph-mon[43506]: pgmap v233: 129 pgs: 2 creating+activating, 1 creating+peering, 126 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2023-12-14T18:26:04.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:04 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2258520944' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:26:04.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.802 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-12-14T18:26:04.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.823 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-12-14T18:26:04.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.885 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:26:04.885 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:26:04.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-14T18:26:04.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-14T18:26:04.886 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:26:04.997 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-14T18:26:04.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-14T18:26:04.997 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:26:05.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:05.232 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:26:05.232 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:26:05.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-12-14T18:26:05.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-12-14T18:26:05.232 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:26:05.346 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-14T18:26:05.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:26:05.346 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:05.365 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-14T18:26:05.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:26:05.366 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:05.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2023-12-14T18:26:05.501 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-14T18:26:05.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-14T18:26:05.577 INFO:teuthology.orchestra.run.smithi031.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-14T18:26:05.578 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-14T18:26:05.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:05.579 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2023-12-14T18:26:05.641 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-14T18:26:05.641 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-14T18:26:05.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2023-12-14T18:26:05.719 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-14T18:26:05.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-14T18:26:05.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:2023-12-14T18:26:05.791+0000 7f67f55445c0 -1 init, newargv = 0x561094840320 newargc=15 2023-12-14T18:26:05.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[79007]: starting ceph client 2023-12-14T18:26:05.793 INFO:teuthology.orchestra.run.smithi031.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-14T18:26:05.795 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-14T18:26:05.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:05.796 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2023-12-14T18:26:05.800 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[79007]: starting fuse 2023-12-14T18:26:05.819 INFO:teuthology.orchestra.run.smithi031.stdout:124 2023-12-14T18:26:05.819 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-12-14T18:26:05.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-14T18:26:05.819 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2023-12-14T18:26:05.819 DEBUG:teuthology.orchestra.run.smithi031:> import re 2023-12-14T18:26:05.819 DEBUG:teuthology.orchestra.run.smithi031:> import os 2023-12-14T18:26:05.819 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2023-12-14T18:26:05.819 DEBUG:teuthology.orchestra.run.smithi031:> 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> return f 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-14T18:26:05.820 DEBUG:teuthology.orchestra.run.smithi031:> 2023-12-14T18:26:05.821 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.admin")) 2023-12-14T18:26:05.821 DEBUG:teuthology.orchestra.run.smithi031:> ' 2023-12-14T18:26:05.957 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.admin.79007.asok 2023-12-14T18:26:06.139 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.79007.asok 2023-12-14T18:26:06.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:06.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.79007.asok status 2023-12-14T18:26:06.318 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:26:06.318 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2023-12-14T18:26:06.318 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "1faa353368ad4f5aeb56d7107b638336f58659ed", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "admin", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "79007", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 14739, 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 14739 2023-12-14T18:26:06.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2186471350 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2186471350 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.14739 192.168.0.1:0/2186471350", 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2186471350", 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 17, 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 85, 2023-12-14T18:26:06.320 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2023-12-14T18:26:06.321 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2023-12-14T18:26:06.321 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2023-12-14T18:26:06.321 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:26:06.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:06.330 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2023-12-14T18:26:06.330 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:06.349 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2023-12-14T18:26:06.349 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:06.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:06.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:06.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:06.424 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-14T18:26:06.544 INFO:teuthology.orchestra.run.smithi031.stdout:# file: . 2023-12-14T18:26:06.544 INFO:teuthology.orchestra.run.smithi031.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-14T18:26:06.544 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:06.545 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:26:06.545 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:26:06.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:06.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:06 smithi031 ceph-mon[42085]: pgmap v234: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 429 B/s wr, 1 op/s 2023-12-14T18:26:06.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 -- 172.21.15.31:0/1074390534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 msgr2=0x7f0770107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 --2- 172.21.15.31:0/1074390534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f0770107590 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f076800b0a0 tx=0x7f076800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 -- 172.21.15.31:0/1074390534 shutdown_connections 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 --2- 172.21.15.31:0/1074390534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f0770107590 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 --2- 172.21.15.31:0/1074390534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07700ff150 0x7f07700ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 --2- 172.21.15.31:0/1074390534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0770100340 0x7f0770100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 -- 172.21.15.31:0/1074390534 >> 172.21.15.31:0/1074390534 conn(0x7f07700fa730 msgr2=0x7f07700fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 -- 172.21.15.31:0/1074390534 shutdown_connections 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.752+0000 7f0774f72700 1 -- 172.21.15.31:0/1074390534 wait complete. 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 Processor -- start 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 -- start start 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07700ff150 0x7f0770196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0770100340 0x7f07701965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f077019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019dbe0 con 0x7f0770100cd0 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019dd20 con 0x7f07700ff150 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f0774f72700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077019deb0 con 0x7f0770100340 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f076ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f077019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f076ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f077019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37602/0 (socket says 172.21.15.31:37602) 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f076ed9d700 1 -- 172.21.15.31:0/59891797 learned_addr learned my addr 172.21.15.31:0/59891797 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f076e59c700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07700ff150 0x7f0770196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.753+0000 7f076dd9b700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0770100340 0x7f07701965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f076ed9d700 1 -- 172.21.15.31:0/59891797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0770100340 msgr2=0x7f07701965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f076ed9d700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0770100340 0x7f07701965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f076ed9d700 1 -- 172.21.15.31:0/59891797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07700ff150 msgr2=0x7f0770196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f076ed9d700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07700ff150 0x7f0770196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f076ed9d700 1 -- 172.21.15.31:0/59891797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768009d00 con 0x7f0770100cd0 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f076ed9d700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f077019d610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f076800b780 tx=0x7f076800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076800e040 con 0x7f0770100cd0 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077010aac0 con 0x7f0770100cd0 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0768004c60 con 0x7f0770100cd0 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076801b6b0 con 0x7f0770100cd0 2023-12-14T18:26:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.754+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077010a930 con 0x7f0770100cd0 2023-12-14T18:26:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.755+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0770067160 con 0x7f0770100cd0 2023-12-14T18:26:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.756+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f076801d480 con 0x7f0770100cd0 2023-12-14T18:26:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.757+0000 7f075f7fe700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f07580a1150 0x7f07580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.757+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f07680ce8a0 con 0x7f0770100cd0 2023-12-14T18:26:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.757+0000 7f076e59c700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f07580a1150 0x7f07580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.758+0000 7f076e59c700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f07580a1150 0x7f07580a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07701001a0 tx=0x7f0760009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.761+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0768097e30 con 0x7f0770100cd0 2023-12-14T18:26:06.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:06 smithi204 ceph-mon[43866]: pgmap v234: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 429 B/s wr, 1 op/s 2023-12-14T18:26:06.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.970+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f077019e9c0 con 0x7f0770100cd0 2023-12-14T18:26:06.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.972+0000 7f075f7fe700 1 -- 172.21.15.31:0/59891797 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f076809b450 con 0x7f0770100cd0 2023-12-14T18:26:06.973 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:06.973 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f07580a1150 msgr2=0x7f07580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f07580a1150 0x7f07580a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07701001a0 tx=0x7f0760009260 comp rx=0 tx=0).stop 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 msgr2=0x7f077019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f077019d610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f076800b780 tx=0x7f076800bd00 comp rx=0 tx=0).stop 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 shutdown_connections 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0770100cd0 0x7f077019d610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07700ff150 0x7f0770196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f07580a1150 0x7f07580a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 --2- 172.21.15.31:0/59891797 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0770100340 0x7f07701965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 >> 172.21.15.31:0/59891797 conn(0x7f07700fa730 msgr2=0x7f0770068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:06.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 shutdown_connections 2023-12-14T18:26:06.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:06.974+0000 7f0774f72700 1 -- 172.21.15.31:0/59891797 wait complete. 2023-12-14T18:26:06.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:26:06.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:26:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:06 smithi176 ceph-mon[43506]: pgmap v234: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 429 B/s wr, 1 op/s 2023-12-14T18:26:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 -- 172.21.15.31:0/4175344105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 msgr2=0x7f3ce0108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/4175344105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce0108670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b0a0 tx=0x7f3cd800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 -- 172.21.15.31:0/4175344105 shutdown_connections 2023-12-14T18:26:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/4175344105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce0108670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/4175344105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ce0100230 0x7f3ce0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/4175344105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ce0101420 0x7f3ce0101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 -- 172.21.15.31:0/4175344105 >> 172.21.15.31:0/4175344105 conn(0x7f3ce00fb850 msgr2=0x7f3ce00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:07.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 -- 172.21.15.31:0/4175344105 shutdown_connections 2023-12-14T18:26:07.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 -- 172.21.15.31:0/4175344105 wait complete. 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.130+0000 7f3ce6a79700 1 Processor -- start 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 -- start start 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ce0100230 0x7f3ce0196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ce0101420 0x7f3ce01965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce019dc00 con 0x7f3ce0101db0 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce019dd40 con 0x7f3ce0100230 2023-12-14T18:26:07.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce6a79700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce019ded0 con 0x7f3ce0101420 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37620/0 (socket says 172.21.15.31:37620) 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 -- 172.21.15.31:0/898950979 learned_addr learned my addr 172.21.15.31:0/898950979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce4815700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ce0100230 0x7f3ce0196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3cdffff700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ce0101420 0x7f3ce01965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 -- 172.21.15.31:0/898950979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ce0101420 msgr2=0x7f3ce01965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ce0101420 0x7f3ce01965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 -- 172.21.15.31:0/898950979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ce0100230 msgr2=0x7f3ce0196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ce0100230 0x7f3ce0196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.131+0000 7f3ce5016700 1 -- 172.21.15.31:0/898950979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8009d00 con 0x7f3ce0101db0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.132+0000 7f3ce5016700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce019d630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b780 tx=0x7f3cd800bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.132+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800e040 con 0x7f3ce0101db0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.132+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cd8004c60 con 0x7f3ce0101db0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.132+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce010bba0 con 0x7f3ce0101db0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.132+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd801b6b0 con 0x7f3ce0101db0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.132+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce010ba10 con 0x7f3ce0101db0 2023-12-14T18:26:07.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.133+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce019e520 con 0x7f3ce0101db0 2023-12-14T18:26:07.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.134+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3cd801d480 con 0x7f3ce0101db0 2023-12-14T18:26:07.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.135+0000 7f3cddffb700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3cd00a11d0 0x7f3cd00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.135+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3cd80ce850 con 0x7f3ce0101db0 2023-12-14T18:26:07.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.135+0000 7f3ce4815700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3cd00a11d0 0x7f3cd00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.135+0000 7f3ce4815700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3cd00a11d0 0x7f3cd00a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0101280 tx=0x7f3cd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:07.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.138+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3cd8097e60 con 0x7f3ce0101db0 2023-12-14T18:26:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.350+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ce0107080 con 0x7f3ce0101db0 2023-12-14T18:26:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.351+0000 7f3cddffb700 1 -- 172.21.15.31:0/898950979 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cd80210a0 con 0x7f3ce0101db0 2023-12-14T18:26:07.352 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:26:07.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3cd00a11d0 msgr2=0x7f3cd00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3cd00a11d0 0x7f3cd00a3680 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0101280 tx=0x7f3cd4009260 comp rx=0 tx=0).stop 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 msgr2=0x7f3ce019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce019d630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800b780 tx=0x7f3cd800bd00 comp rx=0 tx=0).stop 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 shutdown_connections 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ce0101db0 0x7f3ce019d630 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ce0100230 0x7f3ce0196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3cd00a11d0 0x7f3cd00a3680 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 --2- 172.21.15.31:0/898950979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ce0101420 0x7f3ce01965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 >> 172.21.15.31:0/898950979 conn(0x7f3ce00fb850 msgr2=0x7f3ce00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 shutdown_connections 2023-12-14T18:26:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.353+0000 7f3ce6a79700 1 -- 172.21.15.31:0/898950979 wait complete. 2023-12-14T18:26:07.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:26:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 -- 172.21.15.31:0/808043794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 msgr2=0x7f2ab0100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 --2- 172.21.15.31:0/808043794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0100760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009990 tx=0x7f2aac009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 -- 172.21.15.31:0/808043794 shutdown_connections 2023-12-14T18:26:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 --2- 172.21.15.31:0/808043794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ab0100ca0 0x7f2ab0107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 --2- 172.21.15.31:0/808043794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab00ff120 0x7f2ab00ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 --2- 172.21.15.31:0/808043794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0100760 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 -- 172.21.15.31:0/808043794 >> 172.21.15.31:0/808043794 conn(0x7f2ab00fa740 msgr2=0x7f2ab00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 -- 172.21.15.31:0/808043794 shutdown_connections 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.507+0000 7f2ab8176700 1 -- 172.21.15.31:0/808043794 wait complete. 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 Processor -- start 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 -- start start 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab00ff120 0x7f2ab01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ab0100ca0 0x7f2ab019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab019fd50 con 0x7f2ab0100310 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab019fe90 con 0x7f2ab0100ca0 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab8176700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab01a0020 con 0x7f2ab00ff120 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab5711700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab5711700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37646/0 (socket says 172.21.15.31:37646) 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.508+0000 7f2ab5711700 1 -- 172.21.15.31:0/1657628647 learned_addr learned my addr 172.21.15.31:0/1657628647 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5f12700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab00ff120 0x7f2ab01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab6713700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ab0100ca0 0x7f2ab019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5711700 1 -- 172.21.15.31:0/1657628647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab00ff120 msgr2=0x7f2ab01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5711700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab00ff120 0x7f2ab01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5711700 1 -- 172.21.15.31:0/1657628647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ab0100ca0 msgr2=0x7f2ab019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5711700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ab0100ca0 0x7f2ab019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5711700 1 -- 172.21.15.31:0/1657628647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0095f0 con 0x7f2ab0100310 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab5711700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0198710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00b620 tx=0x7f2aac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac01d070 con 0x7f2ab0100310 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aac005030 con 0x7f2ab0100310 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab010aa80 con 0x7f2ab0100310 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac004bf0 con 0x7f2ab0100310 2023-12-14T18:26:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.509+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab010a8f0 con 0x7f2ab0100310 2023-12-14T18:26:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.510+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab01a0620 con 0x7f2ab0100310 2023-12-14T18:26:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.511+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2aac004d50 con 0x7f2ab0100310 2023-12-14T18:26:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.512+0000 7f2aa2ffd700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2a9c0a11d0 0x7f2a9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.512+0000 7f2ab5f12700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2a9c0a11d0 0x7f2a9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.512+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2aac0cdac0 con 0x7f2ab0100310 2023-12-14T18:26:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.513+0000 7f2ab5f12700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2a9c0a11d0 0x7f2a9c0a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0100170 tx=0x7f2aa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.517+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2aac0970b0 con 0x7f2ab0100310 2023-12-14T18:26:07.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/59891797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:07.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/898950979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:07.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.725+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ab01a0cc0 con 0x7f2ab0100310 2023-12-14T18:26:07.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.726+0000 7f2aa2ffd700 1 -- 172.21.15.31:0/1657628647 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2aac09a6d0 con 0x7f2ab0100310 2023-12-14T18:26:07.727 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2a9c0a11d0 msgr2=0x7f2a9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2a9c0a11d0 0x7f2a9c0a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0100170 tx=0x7f2aa4009260 comp rx=0 tx=0).stop 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 msgr2=0x7f2ab0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0198710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00b620 tx=0x7f2aac004730 comp rx=0 tx=0).stop 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 shutdown_connections 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ab0100ca0 0x7f2ab019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2a9c0a11d0 0x7f2a9c0a3680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ab00ff120 0x7f2ab01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 --2- 172.21.15.31:0/1657628647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ab0100310 0x7f2ab0198710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 >> 172.21.15.31:0/1657628647 conn(0x7f2ab00fa740 msgr2=0x7f2ab0067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:07.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 shutdown_connections 2023-12-14T18:26:07.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.728+0000 7f2ab8176700 1 -- 172.21.15.31:0/1657628647 wait complete. 2023-12-14T18:26:07.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:26:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/59891797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/898950979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:07.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/333120308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 msgr2=0x7fc6ec105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/333120308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec105950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009990 tx=0x7fc6dc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/333120308 shutdown_connections 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/333120308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec105e90 0x7fc6ec108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/333120308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec100c50 0x7fc6ec103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/333120308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec105950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/333120308 >> 172.21.15.31:0/333120308 conn(0x7fc6ec0fa670 msgr2=0x7fc6ec0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/333120308 shutdown_connections 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.883+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/333120308 wait complete. 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 Processor -- start 2023-12-14T18:26:07.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 -- start start 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec100c50 0x7fc6ec195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec105e90 0x7fc6ec19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec19d9e0 con 0x7fc6ec103570 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec19db20 con 0x7fc6ec105e90 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.884+0000 7fc6f1fa0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0fea40 con 0x7fc6ec100c50 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37664/0 (socket says 172.21.15.31:37664) 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 -- 172.21.15.31:0/3984872395 learned_addr learned my addr 172.21.15.31:0/3984872395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:07.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6f179f700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec105e90 0x7fc6ec19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6f0f9e700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec100c50 0x7fc6ec195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 -- 172.21.15.31:0/3984872395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec100c50 msgr2=0x7fc6ec195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec100c50 0x7fc6ec195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 -- 172.21.15.31:0/3984872395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec105e90 msgr2=0x7fc6ec19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec105e90 0x7fc6ec19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 -- 172.21.15.31:0/3984872395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0095f0 con 0x7fc6ec103570 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6f179f700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec105e90 0x7fc6ec19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e3fff700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec1963a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0059d0 tx=0x7fc6dc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc01d070 con 0x7fc6ec103570 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6dc00fc40 con 0x7fc6ec103570 2023-12-14T18:26:07.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc017650 con 0x7fc6ec103570 2023-12-14T18:26:07.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.885+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec10bac0 con 0x7fc6ec103570 2023-12-14T18:26:07.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.886+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec10b930 con 0x7fc6ec103570 2023-12-14T18:26:07.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.887+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec0fc1e0 con 0x7fc6ec103570 2023-12-14T18:26:07.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.888+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc6dc022470 con 0x7fc6ec103570 2023-12-14T18:26:07.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.888+0000 7fc6e1ffb700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc6d40a11d0 0x7fc6d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:07.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.889+0000 7fc6f0f9e700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc6d40a11d0 0x7fc6d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:07.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.889+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc6dc0ce7f0 con 0x7fc6ec103570 2023-12-14T18:26:07.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.889+0000 7fc6f0f9e700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc6d40a11d0 0x7fc6d40a3680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec068890 tx=0x7fc6e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:07.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:07.892+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc6dc097de0 con 0x7fc6ec103570 2023-12-14T18:26:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/59891797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/898950979' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.106+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6ec061700 con 0x7fc6ec103570 2023-12-14T18:26:08.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.106+0000 7fc6e1ffb700 1 -- 172.21.15.31:0/3984872395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6dc09b400 con 0x7fc6ec103570 2023-12-14T18:26:08.107 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc6d40a11d0 msgr2=0x7fc6d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc6d40a11d0 0x7fc6d40a3680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec068890 tx=0x7fc6e8009260 comp rx=0 tx=0).stop 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 msgr2=0x7fc6ec1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec1963a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0059d0 tx=0x7fc6dc005ef0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 shutdown_connections 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6ec105e90 0x7fc6ec19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc6d40a11d0 0x7fc6d40a3680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6ec100c50 0x7fc6ec195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 --2- 172.21.15.31:0/3984872395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec103570 0x7fc6ec1963a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 >> 172.21.15.31:0/3984872395 conn(0x7fc6ec0fa670 msgr2=0x7fc6ec1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 shutdown_connections 2023-12-14T18:26:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.109+0000 7fc6f1fa0700 1 -- 172.21.15.31:0/3984872395 wait complete. 2023-12-14T18:26:08.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 -- 172.21.15.31:0/102938492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80100d00 msgr2=0x7f9f801030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 --2- 172.21.15.31:0/102938492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80100d00 0x7f9f801030e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009990 tx=0x7f9f68009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 -- 172.21.15.31:0/102938492 shutdown_connections 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 --2- 172.21.15.31:0/102938492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f80105f40 0x7f9f80108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 --2- 172.21.15.31:0/102938492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80100d00 0x7f9f801030e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 --2- 172.21.15.31:0/102938492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f80103620 0x7f9f80105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 -- 172.21.15.31:0/102938492 >> 172.21.15.31:0/102938492 conn(0x7f9f800fa740 msgr2=0x7f9f800fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 -- 172.21.15.31:0/102938492 shutdown_connections 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.263+0000 7f9f859ec700 1 -- 172.21.15.31:0/102938492 wait complete. 2023-12-14T18:26:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 Processor -- start 2023-12-14T18:26:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 -- start start 2023-12-14T18:26:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f80100d00 0x7f9f80195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f80103620 0x7f9f801964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 0x7f9f8019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019db00 con 0x7f9f80105f40 2023-12-14T18:26:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019dc40 con 0x7f9f80100d00 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f859ec700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8019ddd0 con 0x7f9f80103620 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 0x7f9f8019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f7f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 0x7f9f8019d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37696/0 (socket says 172.21.15.31:37696) 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f7f7fe700 1 -- 172.21.15.31:0/2882735870 learned_addr learned my addr 172.21.15.31:0/2882735870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.264+0000 7f9f7effd700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f80100d00 0x7f9f80195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7e7fc700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f80103620 0x7f9f801964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7f7fe700 1 -- 172.21.15.31:0/2882735870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f80103620 msgr2=0x7f9f801964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7f7fe700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f80103620 0x7f9f801964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7f7fe700 1 -- 172.21.15.31:0/2882735870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f80100d00 msgr2=0x7f9f80195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7f7fe700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f80100d00 0x7f9f80195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7f7fe700 1 -- 172.21.15.31:0/2882735870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680095f0 con 0x7f9f80105f40 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f7f7fe700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 0x7f9f8019d530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800ef30 tx=0x7f9f7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:08.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78018070 con 0x7f9f80105f40 2023-12-14T18:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f7800f040 con 0x7f9f80105f40 2023-12-14T18:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8010bb80 con 0x7f9f80105f40 2023-12-14T18:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800cdb0 con 0x7f9f80105f40 2023-12-14T18:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.265+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8010b9f0 con 0x7f9f80105f40 2023-12-14T18:26:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.266+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f800fc2b0 con 0x7f9f80105f40 2023-12-14T18:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.267+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9f78003a00 con 0x7f9f80105f40 2023-12-14T18:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.268+0000 7f9f849ea700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f6c0a1150 0x7f9f6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.268+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9f780cc6f0 con 0x7f9f80105f40 2023-12-14T18:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.268+0000 7f9f7effd700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f6c0a1150 0x7f9f6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.269+0000 7f9f7effd700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f6c0a1150 0x7f9f6c0a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800b680 tx=0x7f9f68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.272+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9f78095c80 con 0x7f9f80105f40 2023-12-14T18:26:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.482+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9f800619a0 con 0x7f9f80105f40 2023-12-14T18:26:08.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.483+0000 7f9f849ea700 1 -- 172.21.15.31:0/2882735870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9f780992a0 con 0x7f9f80105f40 2023-12-14T18:26:08.484 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:26:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f6c0a1150 msgr2=0x7f9f6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f6c0a1150 0x7f9f6c0a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800b680 tx=0x7f9f68005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 msgr2=0x7f9f8019d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 0x7f9f8019d530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800ef30 tx=0x7f9f7800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 shutdown_connections 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f80105f40 0x7f9f8019d530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f80100d00 0x7f9f80195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f6c0a1150 0x7f9f6c0a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 --2- 172.21.15.31:0/2882735870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f80103620 0x7f9f801964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 >> 172.21.15.31:0/2882735870 conn(0x7f9f800fa740 msgr2=0x7f9f80101960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 shutdown_connections 2023-12-14T18:26:08.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.485+0000 7f9f859ec700 1 -- 172.21.15.31:0/2882735870 wait complete. 2023-12-14T18:26:08.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-14T18:26:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 -- 172.21.15.31:0/1975572457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 msgr2=0x7f5ef0108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/1975572457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef0108340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b0a0 tx=0x7f5ee800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 -- 172.21.15.31:0/1975572457 shutdown_connections 2023-12-14T18:26:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/1975572457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef0108340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/1975572457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 0x7f5ef01030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/1975572457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ef0103620 0x7f5ef0105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 -- 172.21.15.31:0/1975572457 >> 172.21.15.31:0/1975572457 conn(0x7f5ef00fa720 msgr2=0x7f5ef00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 -- 172.21.15.31:0/1975572457 shutdown_connections 2023-12-14T18:26:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.639+0000 7f5ef54d7700 1 -- 172.21.15.31:0/1975572457 wait complete. 2023-12-14T18:26:08.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.640+0000 7f5ef54d7700 1 Processor -- start 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.640+0000 7f5ef54d7700 1 -- start start 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5ef54d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 0x7f5ef0196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5ef54d7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ef0103620 0x7f5ef0196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5ef54d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5ef54d7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef019dbc0 con 0x7f5ef0105f40 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5ef54d7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef019dd00 con 0x7f5ef0103620 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5ef54d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef019de90 con 0x7f5ef0100d00 2023-12-14T18:26:08.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37722/0 (socket says 172.21.15.31:37722) 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 -- 172.21.15.31:0/3488162976 learned_addr learned my addr 172.21.15.31:0/3488162976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eee7fc700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ef0103620 0x7f5ef0196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eeeffd700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 0x7f5ef0196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 -- 172.21.15.31:0/3488162976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 msgr2=0x7f5ef0196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 0x7f5ef0196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 -- 172.21.15.31:0/3488162976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ef0103620 msgr2=0x7f5ef0196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ef0103620 0x7f5ef0196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 -- 172.21.15.31:0/3488162976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8009d00 con 0x7f5ef0105f40 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eeeffd700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 0x7f5ef0196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.641+0000 7f5eef7fe700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef019d5f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b780 tx=0x7f5ee800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.642+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800e040 con 0x7f5ef0105f40 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.642+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ee8004c50 con 0x7f5ef0105f40 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.642+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef010bbe0 con 0x7f5ef0105f40 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.642+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80053b0 con 0x7f5ef0105f40 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.642+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef010ba50 con 0x7f5ef0105f40 2023-12-14T18:26:08.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.643+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef00fc290 con 0x7f5ef0105f40 2023-12-14T18:26:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.644+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ee801d9c0 con 0x7f5ef0105f40 2023-12-14T18:26:08.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.645+0000 7f5ee7fff700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5edc0a11d0 0x7f5edc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.645+0000 7f5eeeffd700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5edc0a11d0 0x7f5edc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.645+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5ee80ce340 con 0x7f5ef0105f40 2023-12-14T18:26:08.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.646+0000 7f5eeeffd700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5edc0a11d0 0x7f5edc0a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009cb0 tx=0x7f5ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.649+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ee8097930 con 0x7f5ef0105f40 2023-12-14T18:26:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:08 smithi204 ceph-mon[43866]: pgmap v235: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-14T18:26:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:08 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1657628647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:08 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3984872395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:08 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2882735870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.876+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5ef002cfe0 con 0x7f5ef0105f40 2023-12-14T18:26:08.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.876+0000 7f5ee7fff700 1 -- 172.21.15.31:0/3488162976 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+589 (secure 0 0 0) 0x7f5ee809af50 con 0x7f5ef0105f40 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1680573410 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2330625991 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2330625991 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/384230437 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2094677585 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/132110935 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2094677585 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2892448986 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:08.878 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/463659266 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:08.879 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2117792145 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:08.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5edc0a11d0 msgr2=0x7f5edc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5edc0a11d0 0x7f5edc0a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009cb0 tx=0x7f5ee0009260 comp rx=0 tx=0).stop 2023-12-14T18:26:08.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 msgr2=0x7f5ef019d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef019d5f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b780 tx=0x7f5ee800bab0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 shutdown_connections 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef0105f40 0x7f5ef019d5f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f5edc0a11d0 0x7f5edc0a3680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ef0100d00 0x7f5ef0196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 --2- 172.21.15.31:0/3488162976 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ef0103620 0x7f5ef0196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 >> 172.21.15.31:0/3488162976 conn(0x7f5ef00fa720 msgr2=0x7f5ef0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 shutdown_connections 2023-12-14T18:26:08.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:08.879+0000 7f5ef54d7700 1 -- 172.21.15.31:0/3488162976 wait complete. 2023-12-14T18:26:08.882 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2023-12-14T18:26:08.892 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:08 smithi031 ceph-mon[42085]: pgmap v235: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-14T18:26:08.893 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:08 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1657628647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.893 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:08 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3984872395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.893 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:08 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2882735870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.894 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:26:08.894 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:26:08.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:08 smithi176 ceph-mon[43506]: pgmap v235: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-14T18:26:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:08 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1657628647' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:08 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3984872395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:08 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2882735870' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 -- 172.21.15.31:0/86064261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101db0 msgr2=0x7f2538108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 --2- 172.21.15.31:0/86064261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101db0 0x7f2538108670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f253000b0a0 tx=0x7f253000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 -- 172.21.15.31:0/86064261 shutdown_connections 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 --2- 172.21.15.31:0/86064261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101db0 0x7f2538108670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 --2- 172.21.15.31:0/86064261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538100230 0x7f2538100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 --2- 172.21.15.31:0/86064261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101420 0x7f2538101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 -- 172.21.15.31:0/86064261 >> 172.21.15.31:0/86064261 conn(0x7f25380fb810 msgr2=0x7f25380fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 -- 172.21.15.31:0/86064261 shutdown_connections 2023-12-14T18:26:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.096+0000 7f253ef4d700 1 -- 172.21.15.31:0/86064261 wait complete. 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 Processor -- start 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 -- start start 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538100230 0x7f25381960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 0x7f2538196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101db0 0x7f253819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253819dc70 con 0x7f2538101420 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253819ddb0 con 0x7f2538101db0 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.097+0000 7f253ef4d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253819df40 con 0x7f2538100230 2023-12-14T18:26:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 0x7f2538196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 0x7f2538196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37730/0 (socket says 172.21.15.31:37730) 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 -- 172.21.15.31:0/276489597 learned_addr learned my addr 172.21.15.31:0/276489597 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f253d4ea700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101db0 0x7f253819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f253cce9700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538100230 0x7f25381960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 -- 172.21.15.31:0/276489597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538100230 msgr2=0x7f25381960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538100230 0x7f25381960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 -- 172.21.15.31:0/276489597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101db0 msgr2=0x7f253819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101db0 0x7f253819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 -- 172.21.15.31:0/276489597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009d00 con 0x7f2538101420 2023-12-14T18:26:09.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2537fff700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 0x7f2538196630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f252000d950 tx=0x7f252000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000f890 con 0x7f2538101420 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2520010460 con 0x7f2538101420 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253810bc00 con 0x7f2538101420 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.098+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000e610 con 0x7f2538101420 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.099+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253810ba70 con 0x7f2538101420 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.099+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253819e680 con 0x7f2538101420 2023-12-14T18:26:09.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.101+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25200108d0 con 0x7f2538101420 2023-12-14T18:26:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.101+0000 7f2535ffb700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25240a11d0 0x7f25240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.101+0000 7f253cce9700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25240a11d0 0x7f25240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.101+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f25200cc8a0 con 0x7f2538101420 2023-12-14T18:26:09.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.102+0000 7f253cce9700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25240a11d0 0x7f25240a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2538101280 tx=0x7f2528009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:09.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.106+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2520095da0 con 0x7f2538101420 2023-12-14T18:26:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.314+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2538107090 con 0x7f2538101420 2023-12-14T18:26:09.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.316+0000 7f2535ffb700 1 -- 172.21.15.31:0/276489597 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f2520016090 con 0x7f2538101420 2023-12-14T18:26:09.317 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:09.317 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25240a11d0 msgr2=0x7f25240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25240a11d0 0x7f25240a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2538101280 tx=0x7f2528009260 comp rx=0 tx=0).stop 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 msgr2=0x7f2538196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 0x7f2538196630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f252000d950 tx=0x7f252000dc60 comp rx=0 tx=0).stop 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 shutdown_connections 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2538101db0 0x7f253819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f25240a11d0 0x7f25240a3680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2538100230 0x7f25381960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 --2- 172.21.15.31:0/276489597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2538101420 0x7f2538196630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 >> 172.21.15.31:0/276489597 conn(0x7f25380fb810 msgr2=0x7f25380fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:09.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 shutdown_connections 2023-12-14T18:26:09.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.318+0000 7f253ef4d700 1 -- 172.21.15.31:0/276489597 wait complete. 2023-12-14T18:26:09.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:26:09.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 -- 172.21.15.31:0/4043327183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b800fe4d0 msgr2=0x7f0b800fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 --2- 172.21.15.31:0/4043327183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b800fe4d0 0x7f0b800fe8e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009990 tx=0x7f0b7c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 -- 172.21.15.31:0/4043327183 shutdown_connections 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 --2- 172.21.15.31:0/4043327183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b80101e30 0x7f0b801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 --2- 172.21.15.31:0/4043327183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b800fe4d0 0x7f0b800fe8e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 --2- 172.21.15.31:0/4043327183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b801014a0 0x7f0b801018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 -- 172.21.15.31:0/4043327183 >> 172.21.15.31:0/4043327183 conn(0x7f0b800f9ed0 msgr2=0x7f0b800fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 -- 172.21.15.31:0/4043327183 shutdown_connections 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 -- 172.21.15.31:0/4043327183 wait complete. 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.474+0000 7f0b87394700 1 Processor -- start 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 -- start start 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b800fe4d0 0x7f0b80196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 0x7f0b80196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b80101e30 0x7f0b8019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019dbb0 con 0x7f0b801014a0 2023-12-14T18:26:09.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019dcf0 con 0x7f0b80101e30 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b87394700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8019de80 con 0x7f0b800fe4d0 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 0x7f0b80196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 0x7f0b80196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37754/0 (socket says 172.21.15.31:37754) 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 -- 172.21.15.31:0/3526461654 learned_addr learned my addr 172.21.15.31:0/3526461654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b85130700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b800fe4d0 0x7f0b80196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b85931700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b80101e30 0x7f0b8019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 -- 172.21.15.31:0/3526461654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b800fe4d0 msgr2=0x7f0b80196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b800fe4d0 0x7f0b80196030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 -- 172.21.15.31:0/3526461654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b80101e30 msgr2=0x7f0b8019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b80101e30 0x7f0b8019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.475+0000 7f0b8492f700 1 -- 172.21.15.31:0/3526461654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0095f0 con 0x7f0b801014a0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.476+0000 7f0b8492f700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 0x7f0b80196570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d950 tx=0x7f0b7000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.476+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000f890 con 0x7f0b801014a0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.476+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b70010460 con 0x7f0b801014a0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.476+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8010bbd0 con 0x7f0b801014a0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.476+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000e610 con 0x7f0b801014a0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.476+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8010ba40 con 0x7f0b801014a0 2023-12-14T18:26:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.477+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b80107260 con 0x7f0b801014a0 2023-12-14T18:26:09.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.478+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0b700108d0 con 0x7f0b801014a0 2023-12-14T18:26:09.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.479+0000 7f0b767fc700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b6c0a11d0 0x7f0b6c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.479+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0b700cc870 con 0x7f0b801014a0 2023-12-14T18:26:09.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.479+0000 7f0b85130700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b6c0a11d0 0x7f0b6c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.479+0000 7f0b85130700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b6c0a11d0 0x7f0b6c0a3680 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b5c0 tx=0x7f0b7c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:09.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.482+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0b70095dd0 con 0x7f0b801014a0 2023-12-14T18:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.695+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b80061700 con 0x7f0b801014a0 2023-12-14T18:26:09.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.695+0000 7f0b767fc700 1 -- 172.21.15.31:0/3526461654 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b700200a0 con 0x7f0b801014a0 2023-12-14T18:26:09.697 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:26:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b6c0a11d0 msgr2=0x7f0b6c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b6c0a11d0 0x7f0b6c0a3680 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00b5c0 tx=0x7f0b7c005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 msgr2=0x7f0b80196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 0x7f0b80196570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d950 tx=0x7f0b7000dc60 comp rx=0 tx=0).stop 2023-12-14T18:26:09.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 shutdown_connections 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b80101e30 0x7f0b8019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b6c0a11d0 0x7f0b6c0a3680 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b800fe4d0 0x7f0b80196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 --2- 172.21.15.31:0/3526461654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b801014a0 0x7f0b80196570 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 >> 172.21.15.31:0/3526461654 conn(0x7f0b800f9ed0 msgr2=0x7f0b800fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 shutdown_connections 2023-12-14T18:26:09.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.698+0000 7f0b87394700 1 -- 172.21.15.31:0/3526461654 wait complete. 2023-12-14T18:26:09.711 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:09 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3488162976' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:26:09.711 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:09 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/276489597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:09.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:26:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:09 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3488162976' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:26:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:09 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/276489597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:09.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 -- 172.21.15.31:0/182671859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb310074b30 msgr2=0x7fb310072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 --2- 172.21.15.31:0/182671859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb310074b30 0x7fb310072f80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009990 tx=0x7fb30c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 -- 172.21.15.31:0/182671859 shutdown_connections 2023-12-14T18:26:09.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 --2- 172.21.15.31:0/182671859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb31010c800 0x7fb31010ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 --2- 172.21.15.31:0/182671859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb310074b30 0x7fb310072f80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 --2- 172.21.15.31:0/182671859 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb310073550 0x7fb3100739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 -- 172.21.15.31:0/182671859 >> 172.21.15.31:0/182671859 conn(0x7fb3100fb850 msgr2=0x7fb3100fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 -- 172.21.15.31:0/182671859 shutdown_connections 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 -- 172.21.15.31:0/182671859 wait complete. 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 Processor -- start 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.852+0000 7fb317ad1700 1 -- start start 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb310073550 0x7fb31019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb310074b30 0x7fb31019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 0x7fb3101a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101a4190 con 0x7fb31010c800 2023-12-14T18:26:09.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101a42d0 con 0x7fb310074b30 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3101a4460 con 0x7fb310073550 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 0x7fb3101a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 0x7fb3101a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37784/0 (socket says 172.21.15.31:37784) 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 -- 172.21.15.31:0/2785686092 learned_addr learned my addr 172.21.15.31:0/2785686092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31506c700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb310074b30 0x7fb31019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31586d700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb310073550 0x7fb31019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 -- 172.21.15.31:0/2785686092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb310073550 msgr2=0x7fb31019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb310073550 0x7fb31019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 -- 172.21.15.31:0/2785686092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb310074b30 msgr2=0x7fb31019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:09.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb310074b30 0x7fb31019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 -- 172.21.15.31:0/2785686092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c0095f0 con 0x7fb31010c800 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31586d700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb310073550 0x7fb31019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb31606e700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 0x7fb3101a3bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb308007ee0 tx=0x7fb30800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308017070 con 0x7fb31010c800 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb30800f040 con 0x7fb31010c800 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3101121f0 con 0x7fb31010c800 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.853+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308005740 con 0x7fb31010c800 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.854+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb310112060 con 0x7fb31010c800 2023-12-14T18:26:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.854+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb310102350 con 0x7fb31010c800 2023-12-14T18:26:09.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.856+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb30800dc60 con 0x7fb31010c800 2023-12-14T18:26:09.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.856+0000 7fb306ffd700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2fc0a11d0 0x7fb2fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:09.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.857+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb3080cc940 con 0x7fb31010c800 2023-12-14T18:26:09.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.857+0000 7fb31586d700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2fc0a11d0 0x7fb2fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:09.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.857+0000 7fb31586d700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2fc0a11d0 0x7fb2fc0a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000c00 tx=0x7fb30c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:09.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:09.860+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb308095e40 con 0x7fb31010c800 2023-12-14T18:26:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:09 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3488162976' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:26:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:09 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/276489597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.069+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb310061930 con 0x7fb31010c800 2023-12-14T18:26:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.069+0000 7fb306ffd700 1 -- 172.21.15.31:0/2785686092 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb308099460 con 0x7fb31010c800 2023-12-14T18:26:10.070 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:26:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2fc0a11d0 msgr2=0x7fb2fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2fc0a11d0 0x7fb2fc0a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb30c000c00 tx=0x7fb30c005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 msgr2=0x7fb3101a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 0x7fb3101a3bc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb308007ee0 tx=0x7fb30800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 shutdown_connections 2023-12-14T18:26:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb31010c800 0x7fb3101a3bc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2fc0a11d0 0x7fb2fc0a3680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb310073550 0x7fb31019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 --2- 172.21.15.31:0/2785686092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb310074b30 0x7fb31019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.071+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 >> 172.21.15.31:0/2785686092 conn(0x7fb3100fb850 msgr2=0x7fb3100fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.072+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 shutdown_connections 2023-12-14T18:26:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.072+0000 7fb317ad1700 1 -- 172.21.15.31:0/2785686092 wait complete. 2023-12-14T18:26:10.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:26:10.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 -- 172.21.15.31:0/808714475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 msgr2=0x7f0c0c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/808714475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c103110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009990 tx=0x7f0bfc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 -- 172.21.15.31:0/808714475 shutdown_connections 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/808714475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 0x7f0c0c108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/808714475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c103110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/808714475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c0c103650 0x7f0c0c105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 -- 172.21.15.31:0/808714475 >> 172.21.15.31:0/808714475 conn(0x7f0c0c0fa770 msgr2=0x7f0c0c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 -- 172.21.15.31:0/808714475 shutdown_connections 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.226+0000 7f0c11f2e700 1 -- 172.21.15.31:0/808714475 wait complete. 2023-12-14T18:26:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 Processor -- start 2023-12-14T18:26:10.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 -- start start 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c0c103650 0x7f0c0c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 0x7f0c0c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c19db10 con 0x7f0c0c100d30 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c19dc50 con 0x7f0c0c103650 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c11f2e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c19dde0 con 0x7f0c0c105f70 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c0b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37810/0 (socket says 172.21.15.31:37810) 2023-12-14T18:26:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.227+0000 7f0c0b7fe700 1 -- 172.21.15.31:0/336993641 learned_addr learned my addr 172.21.15.31:0/336993641 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0bfff700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 0x7f0c0c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0affd700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c0c103650 0x7f0c0c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0b7fe700 1 -- 172.21.15.31:0/336993641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 msgr2=0x7f0c0c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0b7fe700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 0x7f0c0c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0b7fe700 1 -- 172.21.15.31:0/336993641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c0c103650 msgr2=0x7f0c0c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0b7fe700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c0c103650 0x7f0c0c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0b7fe700 1 -- 172.21.15.31:0/336993641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0095f0 con 0x7f0c0c100d30 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0bfff700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 0x7f0c0c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c0b7fe700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c195f90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc000c00 tx=0x7f0bfc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc01d070 con 0x7f0c0c100d30 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bfc00fc40 con 0x7f0c0c100d30 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c10bbb0 con 0x7f0c0c100d30 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc017650 con 0x7f0c0c100d30 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.228+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c10ba20 con 0x7f0c0c100d30 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.229+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c0fc2e0 con 0x7f0c0c100d30 2023-12-14T18:26:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.231+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0bfc021410 con 0x7f0c0c100d30 2023-12-14T18:26:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.231+0000 7f0c08ff9700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0bf40a1150 0x7f0bf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.231+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0bfc021e40 con 0x7f0c0c100d30 2023-12-14T18:26:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.231+0000 7f0c0affd700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0bf40a1150 0x7f0bf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.232+0000 7f0c0affd700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0bf40a1150 0x7f0bf40a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009bf0 tx=0x7f0c00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.235+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0bfc097d70 con 0x7f0c0c100d30 2023-12-14T18:26:10.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.443+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c0c061980 con 0x7f0c0c100d30 2023-12-14T18:26:10.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.443+0000 7f0c08ff9700 1 -- 172.21.15.31:0/336993641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0bfc09b390 con 0x7f0c0c100d30 2023-12-14T18:26:10.445 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:26:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0bf40a1150 msgr2=0x7f0bf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0bf40a1150 0x7f0bf40a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009bf0 tx=0x7f0c00009260 comp rx=0 tx=0).stop 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 msgr2=0x7f0c0c195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c195f90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc000c00 tx=0x7f0bfc005ef0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 shutdown_connections 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0bf40a1150 0x7f0bf40a3600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c0c105f70 0x7f0c0c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c0c100d30 0x7f0c0c195f90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 --2- 172.21.15.31:0/336993641 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0c0c103650 0x7f0c0c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 >> 172.21.15.31:0/336993641 conn(0x7f0c0c0fa770 msgr2=0x7f0c0c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 shutdown_connections 2023-12-14T18:26:10.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.446+0000 7f0c11f2e700 1 -- 172.21.15.31:0/336993641 wait complete. 2023-12-14T18:26:10.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 -- 172.21.15.31:0/3957917885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 msgr2=0x7f44b4100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 --2- 172.21.15.31:0/3957917885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b4100760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009990 tx=0x7f44a8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 -- 172.21.15.31:0/3957917885 shutdown_connections 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 --2- 172.21.15.31:0/3957917885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 0x7f44b4107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 --2- 172.21.15.31:0/3957917885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b40ff120 0x7f44b40ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 --2- 172.21.15.31:0/3957917885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b4100760 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 -- 172.21.15.31:0/3957917885 >> 172.21.15.31:0/3957917885 conn(0x7f44b40fa740 msgr2=0x7f44b40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 -- 172.21.15.31:0/3957917885 shutdown_connections 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.600+0000 7f44b887e700 1 -- 172.21.15.31:0/3957917885 wait complete. 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 Processor -- start 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 -- start start 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b40ff120 0x7f44b4195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 0x7f44b419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419db20 con 0x7f44b4100310 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419dc60 con 0x7f44b4100ca0 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.601+0000 7f44b887e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b419ddf0 con 0x7f44b40ff120 2023-12-14T18:26:10.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b41964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37840/0 (socket says 172.21.15.31:37840) 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 -- 172.21.15.31:0/4233527528 learned_addr learned my addr 172.21.15.31:0/4233527528 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b259c700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b40ff120 0x7f44b4195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b2d9d700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 0x7f44b419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 -- 172.21.15.31:0/4233527528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b40ff120 msgr2=0x7f44b4195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b40ff120 0x7f44b4195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 -- 172.21.15.31:0/4233527528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 msgr2=0x7f44b419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 0x7f44b419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 -- 172.21.15.31:0/4233527528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80095f0 con 0x7f44b4100310 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b2d9d700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 0x7f44b419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:10.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b1d9b700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b41964e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b620 tx=0x7f44a8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a801d070 con 0x7f44b4100310 2023-12-14T18:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44a80047e0 con 0x7f44b4100310 2023-12-14T18:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b410aa80 con 0x7f44b4100310 2023-12-14T18:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a8005560 con 0x7f44b4100310 2023-12-14T18:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.602+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b410a8f0 con 0x7f44b4100310 2023-12-14T18:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.603+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b419e390 con 0x7f44b4100310 2023-12-14T18:26:10.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.604+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f44a8004950 con 0x7f44b4100310 2023-12-14T18:26:10.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.605+0000 7f44a37fe700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f449c0a1150 0x7f449c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.605+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f44a80cdc80 con 0x7f44b4100310 2023-12-14T18:26:10.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.605+0000 7f44b259c700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f449c0a1150 0x7f449c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.606+0000 7f44b259c700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f449c0a1150 0x7f449c0a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44b4100170 tx=0x7f44a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.609+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f44a8097270 con 0x7f44b4100310 2023-12-14T18:26:10.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:10 smithi031 ceph-mon[42085]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2023-12-14T18:26:10.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3526461654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2785686092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/336993641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.819+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f44b419e940 con 0x7f44b4100310 2023-12-14T18:26:10.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.819+0000 7f44a37fe700 1 -- 172.21.15.31:0/4233527528 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f44a809a890 con 0x7f44b4100310 2023-12-14T18:26:10.821 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f449c0a1150 msgr2=0x7f449c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f449c0a1150 0x7f449c0a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44b4100170 tx=0x7f44a4009260 comp rx=0 tx=0).stop 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 msgr2=0x7f44b41964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b41964e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44a800b620 tx=0x7f44a8005ef0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 shutdown_connections 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f44b4100ca0 0x7f44b419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f449c0a1150 0x7f449c0a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44b40ff120 0x7f44b4195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 --2- 172.21.15.31:0/4233527528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44b4100310 0x7f44b41964e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 >> 172.21.15.31:0/4233527528 conn(0x7f44b40fa740 msgr2=0x7f44b4067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 shutdown_connections 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.822+0000 7f44b887e700 1 -- 172.21.15.31:0/4233527528 wait complete. 2023-12-14T18:26:10.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-14T18:26:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:10 smithi204 ceph-mon[43866]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2023-12-14T18:26:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3526461654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2785686092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/336993641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 -- 172.21.15.31:0/2419655597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100ca0 msgr2=0x7fe488107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/2419655597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100ca0 0x7fe488107560 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe48000b0a0 tx=0x7fe48000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 -- 172.21.15.31:0/2419655597 shutdown_connections 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/2419655597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100ca0 0x7fe488107560 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/2419655597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4880ff120 0x7fe4880ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/2419655597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe488100310 0x7fe488100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 -- 172.21.15.31:0/2419655597 >> 172.21.15.31:0/2419655597 conn(0x7fe4880fa740 msgr2=0x7fe4880fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 -- 172.21.15.31:0/2419655597 shutdown_connections 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 -- 172.21.15.31:0/2419655597 wait complete. 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.976+0000 7fe48dfd6700 1 Processor -- start 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 -- start start 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4880ff120 0x7fe488103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 0x7fe4881016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe488100ca0 0x7fe488101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881023d0 con 0x7fe488100310 2023-12-14T18:26:10.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488102540 con 0x7fe4880ff120 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe48dfd6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881a0160 con 0x7fe488100ca0 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 0x7fe4881016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 0x7fe4881016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37862/0 (socket says 172.21.15.31:37862) 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 -- 172.21.15.31:0/3614136001 learned_addr learned my addr 172.21.15.31:0/3614136001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe4877fe700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4880ff120 0x7fe488103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe487fff700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe488100ca0 0x7fe488101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 -- 172.21.15.31:0/3614136001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe488100ca0 msgr2=0x7fe488101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe488100ca0 0x7fe488101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 -- 172.21.15.31:0/3614136001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4880ff120 msgr2=0x7fe488103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4880ff120 0x7fe488103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 -- 172.21.15.31:0/3614136001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009d00 con 0x7fe488100310 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.977+0000 7fe486ffd700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 0x7fe4881016f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e9a0 tx=0x7fe47c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.978+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c009790 con 0x7fe488100310 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.978+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe47c00ce70 con 0x7fe488100310 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.978+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c018420 con 0x7fe488100310 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.978+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48810aae0 con 0x7fe488100310 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.978+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48810a950 con 0x7fe488100310 2023-12-14T18:26:10.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.979+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4880679f0 con 0x7fe488100310 2023-12-14T18:26:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.980+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe47c0106f0 con 0x7fe488100310 2023-12-14T18:26:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.981+0000 7fe484ff9700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe4700a1150 0x7fe4700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.981+0000 7fe4877fe700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe4700a1150 0x7fe4700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.981+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe47c014070 con 0x7fe488100310 2023-12-14T18:26:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.981+0000 7fe4877fe700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe4700a1150 0x7fe4700a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe488100170 tx=0x7fe478009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:10.984+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe47c096690 con 0x7fe488100310 2023-12-14T18:26:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:10 smithi176 ceph-mon[43506]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2023-12-14T18:26:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3526461654' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2785686092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/336993641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.208+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe4881a0c90 con 0x7fe488100310 2023-12-14T18:26:11.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.208+0000 7fe484ff9700 1 -- 172.21.15.31:0/3614136001 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+589 (secure 0 0 0) 0x7fe47c026260 con 0x7fe488100310 2023-12-14T18:26:11.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1680573410 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2330625991 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2330625991 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/384230437 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2094677585 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/132110935 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2094677585 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2892448986 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/463659266 2023-12-15T18:20:47.403710+0000 2023-12-14T18:26:11.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2117792145 2023-12-15T18:20:25.264468+0000 2023-12-14T18:26:11.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe4700a1150 msgr2=0x7fe4700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe4700a1150 0x7fe4700a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe488100170 tx=0x7fe478009260 comp rx=0 tx=0).stop 2023-12-14T18:26:11.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 msgr2=0x7fe4881016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 0x7fe4881016f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00e9a0 tx=0x7fe47c00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 shutdown_connections 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe4700a1150 0x7fe4700a3600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe488100ca0 0x7fe488101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe4880ff120 0x7fe488103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 --2- 172.21.15.31:0/3614136001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe488100310 0x7fe4881016f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.210+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 >> 172.21.15.31:0/3614136001 conn(0x7fe4880fa740 msgr2=0x7fe488067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.211+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 shutdown_connections 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:11.211+0000 7fe48dfd6700 1 -- 172.21.15.31:0/3614136001 wait complete. 2023-12-14T18:26:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:listed 10 entries 2023-12-14T18:26:11.225 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2023-12-14T18:26:11.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:11.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:26:11.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi031.stderr:ceph-fuse[79007]: fuse finished with error 0 and tester_r 0 2023-12-14T18:26:11.264 INFO:teuthology.orchestra.run:waiting for 300 2023-12-14T18:26:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:11 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4233527528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:11 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3614136001' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:26:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:11 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4233527528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:11 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3614136001' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:26:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:11 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4233527528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:11 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3614136001' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:26:12.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:12 smithi204 ceph-mon[43866]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 262 B/s wr, 0 op/s 2023-12-14T18:26:12.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:12 smithi031 ceph-mon[42085]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 262 B/s wr, 0 op/s 2023-12-14T18:26:12.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:12 smithi176 ceph-mon[43506]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 262 B/s wr, 0 op/s 2023-12-14T18:26:14.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:14 smithi204 ceph-mon[43866]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:26:14.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:14 smithi031 ceph-mon[42085]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:26:14.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:14 smithi176 ceph-mon[43506]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:26:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:16 smithi204 ceph-mon[43866]: pgmap v239: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:16 smithi031 ceph-mon[42085]: pgmap v239: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:16 smithi176 ceph-mon[43506]: pgmap v239: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:17.366 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:26:17.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:17.367 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-14T18:26:17.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:17.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 -- 172.21.15.31:0/3852071862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80fe4e0 msgr2=0x7f66b80fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 --2- 172.21.15.31:0/3852071862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80fe4e0 0x7f66b80fe8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009990 tx=0x7f66a8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 -- 172.21.15.31:0/3852071862 shutdown_connections 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 --2- 172.21.15.31:0/3852071862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b8101e40 0x7f66b8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 --2- 172.21.15.31:0/3852071862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80fe4e0 0x7f66b80fe8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 --2- 172.21.15.31:0/3852071862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66b81014b0 0x7f66b8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 -- 172.21.15.31:0/3852071862 >> 172.21.15.31:0/3852071862 conn(0x7f66b80f9e80 msgr2=0x7f66b80fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 -- 172.21.15.31:0/3852071862 shutdown_connections 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 -- 172.21.15.31:0/3852071862 wait complete. 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 Processor -- start 2023-12-14T18:26:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.596+0000 7f66bdea0700 1 -- start start 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66bdea0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b80fe4e0 0x7f66b8195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66bdea0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 0x7f66b8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66bdea0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66b8101e40 0x7f66b819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66bdea0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819db40 con 0x7f66b81014b0 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66bdea0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819dc80 con 0x7f66b80fe4e0 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66bdea0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b819de10 con 0x7f66b8101e40 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 0x7f66b8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 0x7f66b8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53142/0 (socket says 172.21.15.31:53142) 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 -- 172.21.15.31:0/248451655 learned_addr learned my addr 172.21.15.31:0/248451655 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66b7fff700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66b8101e40 0x7f66b819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66b77fe700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b80fe4e0 0x7f66b8195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 -- 172.21.15.31:0/248451655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66b8101e40 msgr2=0x7f66b819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66b8101e40 0x7f66b819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 -- 172.21.15.31:0/248451655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b80fe4e0 msgr2=0x7f66b8195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b80fe4e0 0x7f66b8195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 -- 172.21.15.31:0/248451655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a80095f0 con 0x7f66b81014b0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.597+0000 7f66aeffd700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 0x7f66b8196500 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66a000d950 tx=0x7f66a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.598+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a000f890 con 0x7f66b81014b0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.598+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66a0010460 con 0x7f66b81014b0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.598+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b810bb90 con 0x7f66b81014b0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.598+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a000e610 con 0x7f66b81014b0 2023-12-14T18:26:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.598+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b810ba00 con 0x7f66b81014b0 2023-12-14T18:26:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.599+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b8107240 con 0x7f66b81014b0 2023-12-14T18:26:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.600+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f66a00108d0 con 0x7f66b81014b0 2023-12-14T18:26:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.600+0000 7f66b57fa700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f66a40a11d0 0x7f66a40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.601+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f66a00cc870 con 0x7f66b81014b0 2023-12-14T18:26:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.601+0000 7f66b77fe700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f66a40a11d0 0x7f66a40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.601+0000 7f66b77fe700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f66a40a11d0 0x7f66a40a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b2e0 tx=0x7f66a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:17.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.604+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f66a0095dd0 con 0x7f66b81014b0 2023-12-14T18:26:17.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.816+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66b802cfe0 con 0x7f66b81014b0 2023-12-14T18:26:17.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.818+0000 7f66b57fa700 1 -- 172.21.15.31:0/248451655 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f66a0020090 con 0x7f66b81014b0 2023-12-14T18:26:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:17.820 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:17.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f66a40a11d0 msgr2=0x7f66a40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f66a40a11d0 0x7f66a40a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66a800b2e0 tx=0x7f66a8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 msgr2=0x7f66b8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 0x7f66b8196500 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f66a000d950 tx=0x7f66a000dc60 comp rx=0 tx=0).stop 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 shutdown_connections 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f66a40a11d0 0x7f66a40a3680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66b8101e40 0x7f66b819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f66b80fe4e0 0x7f66b8195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 --2- 172.21.15.31:0/248451655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b81014b0 0x7f66b8196500 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 >> 172.21.15.31:0/248451655 conn(0x7f66b80f9e80 msgr2=0x7f66b80fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 shutdown_connections 2023-12-14T18:26:17.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.820+0000 7f66bdea0700 1 -- 172.21.15.31:0/248451655 wait complete. 2023-12-14T18:26:17.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:17.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 -- 172.21.15.31:0/2823376036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fe270 msgr2=0x7f12840fe6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 --2- 172.21.15.31:0/2823376036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fe270 0x7f12840fe6c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1278009990 tx=0x7f1278009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 -- 172.21.15.31:0/2823376036 shutdown_connections 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 --2- 172.21.15.31:0/2823376036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12840fec00 0x7f12841087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 --2- 172.21.15.31:0/2823376036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284101800 0x7f1284101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 --2- 172.21.15.31:0/2823376036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fe270 0x7f12840fe6c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 -- 172.21.15.31:0/2823376036 >> 172.21.15.31:0/2823376036 conn(0x7f12840f9e80 msgr2=0x7f12840fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 -- 172.21.15.31:0/2823376036 shutdown_connections 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.973+0000 7f128b461700 1 -- 172.21.15.31:0/2823376036 wait complete. 2023-12-14T18:26:17.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 Processor -- start 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 -- start start 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12840fe270 0x7f1284196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 0x7f1284196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1284101800 0x7f128419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128419db80 con 0x7f12840fec00 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128419dcc0 con 0x7f12840fe270 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f128b461700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128419de50 con 0x7f1284101800 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f12889fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 0x7f1284196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f12889fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 0x7f1284196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53152/0 (socket says 172.21.15.31:53152) 2023-12-14T18:26:17.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f12891fd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12840fe270 0x7f1284196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.974+0000 7f12889fc700 1 -- 172.21.15.31:0/313684170 learned_addr learned my addr 172.21.15.31:0/313684170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12899fe700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1284101800 0x7f128419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12889fc700 1 -- 172.21.15.31:0/313684170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1284101800 msgr2=0x7f128419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12889fc700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1284101800 0x7f128419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12889fc700 1 -- 172.21.15.31:0/313684170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12840fe270 msgr2=0x7f1284196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12889fc700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12840fe270 0x7f1284196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12889fc700 1 -- 172.21.15.31:0/313684170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780095f0 con 0x7f12840fec00 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12899fe700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1284101800 0x7f128419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12889fc700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 0x7f1284196540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1278004730 tx=0x7f1278004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127801d070 con 0x7f12840fec00 2023-12-14T18:26:17.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128410bbd0 con 0x7f12840fec00 2023-12-14T18:26:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1278004c90 con 0x7f12840fec00 2023-12-14T18:26:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278005570 con 0x7f12840fec00 2023-12-14T18:26:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.975+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128410ba40 con 0x7f12840fec00 2023-12-14T18:26:17.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.976+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1284067160 con 0x7f12840fec00 2023-12-14T18:26:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.978+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f127800fc70 con 0x7f12840fec00 2023-12-14T18:26:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.978+0000 7f12767fc700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12700a1150 0x7f12700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.978+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f12780ce300 con 0x7f12840fec00 2023-12-14T18:26:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.978+0000 7f12891fd700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12700a1150 0x7f12700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.979+0000 7f12891fd700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12700a1150 0x7f12700a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1284067af0 tx=0x7f128000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:17.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:17.982+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f12780978f0 con 0x7f12840fec00 2023-12-14T18:26:18.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.194+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12841072f0 con 0x7f12840fec00 2023-12-14T18:26:18.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.196+0000 7f12767fc700 1 -- 172.21.15.31:0/313684170 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f127809af10 con 0x7f12840fec00 2023-12-14T18:26:18.197 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:18.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12700a1150 msgr2=0x7f12700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:18.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12700a1150 0x7f12700a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1284067af0 tx=0x7f128000b410 comp rx=0 tx=0).stop 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 msgr2=0x7f1284196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 0x7f1284196540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1278004730 tx=0x7f1278004760 comp rx=0 tx=0).stop 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 shutdown_connections 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12700a1150 0x7f12700a3600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1284101800 0x7f128419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12840fe270 0x7f1284196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 --2- 172.21.15.31:0/313684170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12840fec00 0x7f1284196540 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 >> 172.21.15.31:0/313684170 conn(0x7f12840f9e80 msgr2=0x7f12840fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 shutdown_connections 2023-12-14T18:26:18.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.198+0000 7f128b461700 1 -- 172.21.15.31:0/313684170 wait complete. 2023-12-14T18:26:18.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-14T18:26:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 -- 172.21.15.31:0/2402479472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0074ad0 msgr2=0x7f2ef0072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 --2- 172.21.15.31:0/2402479472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0074ad0 0x7f2ef0072f20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009990 tx=0x7f2eec009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 -- 172.21.15.31:0/2402479472 shutdown_connections 2023-12-14T18:26:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 --2- 172.21.15.31:0/2402479472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef01080c0 0x7f2ef010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 --2- 172.21.15.31:0/2402479472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef0074ad0 0x7f2ef0072f20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 --2- 172.21.15.31:0/2402479472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ef00734f0 0x7f2ef0073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 -- 172.21.15.31:0/2402479472 >> 172.21.15.31:0/2402479472 conn(0x7f2ef00fb850 msgr2=0x7f2ef00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 -- 172.21.15.31:0/2402479472 shutdown_connections 2023-12-14T18:26:18.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.353+0000 7f2ef8690700 1 -- 172.21.15.31:0/2402479472 wait complete. 2023-12-14T18:26:18.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 Processor -- start 2023-12-14T18:26:18.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 -- start start 2023-12-14T18:26:18.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ef00734f0 0x7f2ef0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:18.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef0074ad0 0x7f2ef01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:18.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 0x7f2ef019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef019fd10 con 0x7f2ef01080c0 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef019fe50 con 0x7f2ef00734f0 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef8690700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef019ffe0 con 0x7f2ef0074ad0 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 0x7f2ef019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 0x7f2ef019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53166/0 (socket says 172.21.15.31:53166) 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 -- 172.21.15.31:0/3237366604 learned_addr learned my addr 172.21.15.31:0/3237366604 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef642c700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ef00734f0 0x7f2ef0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef5c2b700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef0074ad0 0x7f2ef01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 -- 172.21.15.31:0/3237366604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef0074ad0 msgr2=0x7f2ef01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef0074ad0 0x7f2ef01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 -- 172.21.15.31:0/3237366604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ef00734f0 msgr2=0x7f2ef0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ef00734f0 0x7f2ef0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:18.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 -- 172.21.15.31:0/3237366604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec0095f0 con 0x7f2ef01080c0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.354+0000 7f2ef6c2d700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 0x7f2ef019f740 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8007ee0 tx=0x7f2ee800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.355+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8017070 con 0x7f2ef01080c0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.355+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef010dd90 con 0x7f2ef01080c0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.355+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee800f040 con 0x7f2ef01080c0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.355+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8005550 con 0x7f2ef01080c0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.355+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef010dc00 con 0x7f2ef01080c0 2023-12-14T18:26:18.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.356+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0106800 con 0x7f2ef01080c0 2023-12-14T18:26:18.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.357+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2ee800db20 con 0x7f2ef01080c0 2023-12-14T18:26:18.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.358+0000 7f2ee77fe700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2edc0a11d0 0x7f2edc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:18.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.358+0000 7f2ef642c700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2edc0a11d0 0x7f2edc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:18.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.358+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2ee80cc780 con 0x7f2ef01080c0 2023-12-14T18:26:18.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.358+0000 7f2ef642c700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2edc0a11d0 0x7f2edc0a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b5c0 tx=0x7f2eec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:18.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.361+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2ee8095ce0 con 0x7f2ef01080c0 2023-12-14T18:26:18.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:18.621+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2ef002cfe0 con 0x7f2ef01080c0 2023-12-14T18:26:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:18 smithi204 ceph-mon[43866]: pgmap v240: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:18 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/248451655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:18 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/313684170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:18.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:18 smithi031 ceph-mon[42085]: pgmap v240: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:18 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/248451655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:18.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:18 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/313684170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:18 smithi176 ceph-mon[43506]: pgmap v240: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:18 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/248451655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:18 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/313684170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:19.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.541+0000 7f2ee77fe700 1 -- 172.21.15.31:0/3237366604 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2ee8099300 con 0x7f2ef01080c0 2023-12-14T18:26:19.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2edc0a11d0 msgr2=0x7f2edc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:19.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2edc0a11d0 0x7f2edc0a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00b5c0 tx=0x7f2eec005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 msgr2=0x7f2ef019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:19.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 0x7f2ef019f740 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8007ee0 tx=0x7f2ee800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 shutdown_connections 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef01080c0 0x7f2ef019f740 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ef00734f0 0x7f2ef0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2edc0a11d0 0x7f2edc0a3680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 --2- 172.21.15.31:0/3237366604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ef0074ad0 0x7f2ef01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 >> 172.21.15.31:0/3237366604 conn(0x7f2ef00fb850 msgr2=0x7f2ef00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 shutdown_connections 2023-12-14T18:26:19.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.544+0000 7f2ef8690700 1 -- 172.21.15.31:0/3237366604 wait complete. 2023-12-14T18:26:19.560 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-12-14T18:26:19.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 -- 172.21.15.31:0/561018758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 msgr2=0x7ff800100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 --2- 172.21.15.31:0/561018758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff800100760 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009990 tx=0x7ff7e8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 -- 172.21.15.31:0/561018758 shutdown_connections 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 --2- 172.21.15.31:0/561018758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff800100ca0 0x7ff800107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 --2- 172.21.15.31:0/561018758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8000ff120 0x7ff8000ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 --2- 172.21.15.31:0/561018758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff800100760 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 -- 172.21.15.31:0/561018758 >> 172.21.15.31:0/561018758 conn(0x7ff8000fa740 msgr2=0x7ff8000fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 -- 172.21.15.31:0/561018758 shutdown_connections 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 -- 172.21.15.31:0/561018758 wait complete. 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.704+0000 7ff805f66700 1 Processor -- start 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 -- start start 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8000ff120 0x7ff800195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff8001964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff800100ca0 0x7ff80019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80019db30 con 0x7ff800100310 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80019dc70 con 0x7ff800100ca0 2023-12-14T18:26:19.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff805f66700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80019de00 con 0x7ff8000ff120 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff8001964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff8001964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53180/0 (socket says 172.21.15.31:53180) 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 -- 172.21.15.31:0/4047656064 learned_addr learned my addr 172.21.15.31:0/4047656064 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7fffff700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff800100ca0 0x7ff80019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7ff7fe700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8000ff120 0x7ff800195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 -- 172.21.15.31:0/4047656064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8000ff120 msgr2=0x7ff800195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8000ff120 0x7ff800195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 -- 172.21.15.31:0/4047656064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff800100ca0 msgr2=0x7ff80019d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff800100ca0 0x7ff80019d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.705+0000 7ff7feffd700 1 -- 172.21.15.31:0/4047656064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80095f0 con 0x7ff800100310 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.706+0000 7ff7feffd700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff8001964f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8005910 tx=0x7ff7e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.706+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e801d070 con 0x7ff800100310 2023-12-14T18:26:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.706+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80010bc50 con 0x7ff800100310 2023-12-14T18:26:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.706+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7e8003f60 con 0x7ff800100310 2023-12-14T18:26:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.706+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8022ca0 con 0x7ff800100310 2023-12-14T18:26:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.706+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80010bac0 con 0x7ff800100310 2023-12-14T18:26:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.707+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8001066e0 con 0x7ff800100310 2023-12-14T18:26:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.708+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff7e80040d0 con 0x7ff800100310 2023-12-14T18:26:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.709+0000 7ff7fcff9700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff7ec0a1150 0x7ff7ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.709+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7e80ce0c0 con 0x7ff800100310 2023-12-14T18:26:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.709+0000 7ff7ff7fe700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff7ec0a1150 0x7ff7ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.709+0000 7ff7ff7fe700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff7ec0a1150 0x7ff7ec0a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005fd0 tx=0x7ff7f0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.713+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff7e8097650 con 0x7ff800100310 2023-12-14T18:26:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:19 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3237366604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-14T18:26:19.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:19 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3237366604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-14T18:26:19.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:19.972+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7ff800061c60 con 0x7ff800100310 2023-12-14T18:26:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:19 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3237366604' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-14T18:26:20.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.551+0000 7ff7fcff9700 1 -- 172.21.15.31:0/4047656064 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff7e809ac70 con 0x7ff800100310 2023-12-14T18:26:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff7ec0a1150 msgr2=0x7ff7ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff7ec0a1150 0x7ff7ec0a3600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005fd0 tx=0x7ff7f0005f60 comp rx=0 tx=0).stop 2023-12-14T18:26:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 msgr2=0x7ff8001964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff8001964f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8005910 tx=0x7ff7e8005740 comp rx=0 tx=0).stop 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 shutdown_connections 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff800100ca0 0x7ff80019d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff7ec0a1150 0x7ff7ec0a3600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8000ff120 0x7ff800195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 --2- 172.21.15.31:0/4047656064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff800100310 0x7ff8001964f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 >> 172.21.15.31:0/4047656064 conn(0x7ff8000fa740 msgr2=0x7ff800103820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.554+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 shutdown_connections 2023-12-14T18:26:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.555+0000 7ff805f66700 1 -- 172.21.15.31:0/4047656064 wait complete. 2023-12-14T18:26:20.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-14T18:26:20.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 -- 172.21.15.31:0/681781425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 msgr2=0x7f400c107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 --2- 172.21.15.31:0/681781425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c107580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f400400b0a0 tx=0x7f400400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 -- 172.21.15.31:0/681781425 shutdown_connections 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 --2- 172.21.15.31:0/681781425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c107580 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 --2- 172.21.15.31:0/681781425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f400c0ff140 0x7f400c0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 --2- 172.21.15.31:0/681781425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f400c100330 0x7f400c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 -- 172.21.15.31:0/681781425 >> 172.21.15.31:0/681781425 conn(0x7f400c0fa780 msgr2=0x7f400c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 -- 172.21.15.31:0/681781425 shutdown_connections 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.718+0000 7f401146b700 1 -- 172.21.15.31:0/681781425 wait complete. 2023-12-14T18:26:20.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 Processor -- start 2023-12-14T18:26:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 -- start start 2023-12-14T18:26:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f400c0ff140 0x7f400c194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f400c100330 0x7f400c1954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c19cb20 con 0x7f400c100cc0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c19cc60 con 0x7f400c0ff140 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f401146b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400c19cdf0 con 0x7f400c100330 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f400b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c19c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53194/0 (socket says 172.21.15.31:53194) 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.719+0000 7f400b7fe700 1 -- 172.21.15.31:0/1557439692 learned_addr learned my addr 172.21.15.31:0/1557439692 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400a7fc700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f400c100330 0x7f400c1954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400affd700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f400c0ff140 0x7f400c194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400b7fe700 1 -- 172.21.15.31:0/1557439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f400c100330 msgr2=0x7f400c1954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400b7fe700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f400c100330 0x7f400c1954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400b7fe700 1 -- 172.21.15.31:0/1557439692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f400c0ff140 msgr2=0x7f400c194fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400b7fe700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f400c0ff140 0x7f400c194fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400b7fe700 1 -- 172.21.15.31:0/1557439692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004009d00 con 0x7f400c100cc0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f400b7fe700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c19c550 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f400400b830 tx=0x7f4004004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400400e040 con 0x7f400c100cc0 2023-12-14T18:26:20.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400401ab90 con 0x7f400c100cc0 2023-12-14T18:26:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400c10aab0 con 0x7f400c100cc0 2023-12-14T18:26:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40040056f0 con 0x7f400c100cc0 2023-12-14T18:26:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.720+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400c10a920 con 0x7f400c100cc0 2023-12-14T18:26:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.721+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400c067e40 con 0x7f400c100cc0 2023-12-14T18:26:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.723+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f400401ad00 con 0x7f400c100cc0 2023-12-14T18:26:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.723+0000 7f3ff3fff700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ff40a1150 0x7f3ff40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.723+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f40040cff90 con 0x7f400c100cc0 2023-12-14T18:26:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.723+0000 7f400affd700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ff40a1150 0x7f3ff40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.724+0000 7f400affd700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ff40a1150 0x7f3ff40a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f400c100190 tx=0x7f400000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.728+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4004102730 con 0x7f400c100cc0 2023-12-14T18:26:20.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:20 smithi204 ceph-mon[43866]: pgmap v241: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:20.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:20 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3237366604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-14T18:26:20.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:20 smithi204 ceph-mon[43866]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:20.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:20 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4047656064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-14T18:26:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:20 smithi031 ceph-mon[42085]: pgmap v241: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:20 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3237366604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-14T18:26:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:20 smithi031 ceph-mon[42085]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:20 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4047656064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-14T18:26:20.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:20.987+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f400c069c90 con 0x7f400c100cc0 2023-12-14T18:26:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:20 smithi176 ceph-mon[43506]: pgmap v241: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-14T18:26:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:20 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3237366604' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-14T18:26:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:20 smithi176 ceph-mon[43506]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:20 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4047656064' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-14T18:26:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.566+0000 7f3ff3fff700 1 -- 172.21.15.31:0/1557439692 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f4004099580 con 0x7f400c100cc0 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ff40a1150 msgr2=0x7f3ff40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ff40a1150 0x7f3ff40a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f400c100190 tx=0x7f400000b410 comp rx=0 tx=0).stop 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 msgr2=0x7f400c19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c19c550 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f400400b830 tx=0x7f4004004c80 comp rx=0 tx=0).stop 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 shutdown_connections 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f400c100cc0 0x7f400c19c550 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f400c0ff140 0x7f400c194fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ff40a1150 0x7f3ff40a3600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 --2- 172.21.15.31:0/1557439692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f400c100330 0x7f400c1954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.569+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 >> 172.21.15.31:0/1557439692 conn(0x7f400c0fa780 msgr2=0x7f400c0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.570+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 shutdown_connections 2023-12-14T18:26:21.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.570+0000 7f401146b700 1 -- 172.21.15.31:0/1557439692 wait complete. 2023-12-14T18:26:21.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-14T18:26:21.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 -- 172.21.15.31:0/1919271055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc1014a0 msgr2=0x7f9ffc1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 --2- 172.21.15.31:0/1919271055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc1014a0 0x7f9ffc1018f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009990 tx=0x7f9fe4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 -- 172.21.15.31:0/1919271055 shutdown_connections 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 --2- 172.21.15.31:0/1919271055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ffc101e30 0x7f9ffc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 --2- 172.21.15.31:0/1919271055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ffc0fe4d0 0x7f9ffc0fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 --2- 172.21.15.31:0/1919271055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc1014a0 0x7f9ffc1018f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 -- 172.21.15.31:0/1919271055 >> 172.21.15.31:0/1919271055 conn(0x7f9ffc0f9ed0 msgr2=0x7f9ffc0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 -- 172.21.15.31:0/1919271055 shutdown_connections 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 -- 172.21.15.31:0/1919271055 wait complete. 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.733+0000 7fa002133700 1 Processor -- start 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 -- start start 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ffc0fe4d0 0x7f9ffc195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ffc1014a0 0x7f9ffc196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 0x7f9ffc19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc19db60 con 0x7f9ffc101e30 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc19dca0 con 0x7f9ffc0fe4d0 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7fa002133700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc19de30 con 0x7f9ffc1014a0 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 0x7f9ffc19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 0x7f9ffc19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53214/0 (socket says 172.21.15.31:53214) 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 -- 172.21.15.31:0/771471494 learned_addr learned my addr 172.21.15.31:0/771471494 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffb7fe700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ffc0fe4d0 0x7f9ffc195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffaffd700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ffc1014a0 0x7f9ffc196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 -- 172.21.15.31:0/771471494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ffc1014a0 msgr2=0x7f9ffc196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ffc1014a0 0x7f9ffc196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 -- 172.21.15.31:0/771471494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ffc0fe4d0 msgr2=0x7f9ffc195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ffc0fe4d0 0x7f9ffc195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.734+0000 7f9ffbfff700 1 -- 172.21.15.31:0/771471494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40095f0 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.735+0000 7f9ffbfff700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 0x7f9ffc19d590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4007ee0 tx=0x7f9ff400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.735+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4017070 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.735+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ff400f040 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.735+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc10bbc0 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.735+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4005740 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.735+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc10ba30 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.736+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc107230 con 0x7f9ffc101e30 2023-12-14T18:26:21.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.737+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9ff400dc60 con 0x7f9ffc101e30 2023-12-14T18:26:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.738+0000 7f9ff8ff9700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9fe80a11d0 0x7f9fe80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.738+0000 7f9ffb7fe700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9fe80a11d0 0x7f9fe80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.738+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9ff40cc2b0 con 0x7f9ffc101e30 2023-12-14T18:26:21.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.739+0000 7f9ffb7fe700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9fe80a11d0 0x7f9fe80a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc067bd0 tx=0x7f9fec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:21.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:21.743+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9ff4095810 con 0x7f9ffc101e30 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4047656064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-12-14T18:26:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:21 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1557439692' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-14T18:26:21.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-14T18:26:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4047656064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-14T18:26:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-14T18:26:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-12-14T18:26:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-12-14T18:26:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:21 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1557439692' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4047656064' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: daemon mds.b is now active in filesystem cephfs as rank 1 2023-12-14T18:26:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:21 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1557439692' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-14T18:26:22.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.004+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f9ffc061930 con 0x7f9ffc101e30 2023-12-14T18:26:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.577+0000 7f9ff8ff9700 1 -- 172.21.15.31:0/771471494 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9ff4098e30 con 0x7f9ffc101e30 2023-12-14T18:26:22.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9fe80a11d0 msgr2=0x7f9fe80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9fe80a11d0 0x7f9fe80a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc067bd0 tx=0x7f9fec009260 comp rx=0 tx=0).stop 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 msgr2=0x7f9ffc19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 0x7f9ffc19d590 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4007ee0 tx=0x7f9ff400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 shutdown_connections 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ffc101e30 0x7f9ffc19d590 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ffc0fe4d0 0x7f9ffc195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9fe80a11d0 0x7f9fe80a3680 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 --2- 172.21.15.31:0/771471494 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ffc1014a0 0x7f9ffc196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 >> 172.21.15.31:0/771471494 conn(0x7f9ffc0f9ed0 msgr2=0x7f9ffc0fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 shutdown_connections 2023-12-14T18:26:22.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.582+0000 7fa002133700 1 -- 172.21.15.31:0/771471494 wait complete. 2023-12-14T18:26:22.599 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-12-14T18:26:22.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.743+0000 7f7bd025b700 1 -- 172.21.15.31:0/338285566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8100240 msgr2=0x7f7bc8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.743+0000 7f7bd025b700 1 --2- 172.21.15.31:0/338285566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8100240 0x7f7bc8100650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009990 tx=0x7f7bc4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 -- 172.21.15.31:0/338285566 shutdown_connections 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 --2- 172.21.15.31:0/338285566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8101dc0 0x7f7bc8108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 --2- 172.21.15.31:0/338285566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8100240 0x7f7bc8100650 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 --2- 172.21.15.31:0/338285566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 0x7f7bc8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 -- 172.21.15.31:0/338285566 >> 172.21.15.31:0/338285566 conn(0x7f7bc80fb840 msgr2=0x7f7bc80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 -- 172.21.15.31:0/338285566 shutdown_connections 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 -- 172.21.15.31:0/338285566 wait complete. 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 Processor -- start 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.744+0000 7f7bd025b700 1 -- start start 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bd025b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8100240 0x7f7bc8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bd025b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 0x7f7bc81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bd025b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 0x7f7bc819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bd025b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc819db00 con 0x7f7bc8101dc0 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bd025b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc819dc40 con 0x7f7bc8101430 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bd025b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc819ddd0 con 0x7f7bc8100240 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 0x7f7bc819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 0x7f7bc819d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53242/0 (socket says 172.21.15.31:53242) 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 -- 172.21.15.31:0/1197710152 learned_addr learned my addr 172.21.15.31:0/1197710152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bcdff7700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8100240 0x7f7bc8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bcd7f6700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 0x7f7bc81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 -- 172.21.15.31:0/1197710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8100240 msgr2=0x7f7bc8195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8100240 0x7f7bc8195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 -- 172.21.15.31:0/1197710152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 msgr2=0x7f7bc81964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 0x7f7bc81964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 -- 172.21.15.31:0/1197710152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc40095f0 con 0x7f7bc8101dc0 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bcd7f6700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 0x7f7bc81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bce7f8700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 0x7f7bc819d530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0007d70 tx=0x7f7bc000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000ad40 con 0x7f7bc8101dc0 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.745+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bc00128e0 con 0x7f7bc8101dc0 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.746+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc810bbf0 con 0x7f7bc8101dc0 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.746+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00116b0 con 0x7f7bc8101dc0 2023-12-14T18:26:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.746+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc810ba60 con 0x7f7bc8101dc0 2023-12-14T18:26:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.747+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc81077b0 con 0x7f7bc8101dc0 2023-12-14T18:26:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.748+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7bc0012420 con 0x7f7bc8101dc0 2023-12-14T18:26:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.748+0000 7f7bbeffd700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7bb40a11d0 0x7f7bb40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.748+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7bc00ccd10 con 0x7f7bc8101dc0 2023-12-14T18:26:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.748+0000 7f7bcdff7700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7bb40a11d0 0x7f7bb40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.749+0000 7f7bcdff7700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7bb40a11d0 0x7f7bb40a3680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400b2e0 tx=0x7f7bc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:22.753+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7bc0096270 con 0x7f7bc8101dc0 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] up:active 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1557439692' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: Cluster is now healthy 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-14T18:26:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:22 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/771471494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-14T18:26:22.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:26:22.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] up:active 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1557439692' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: Cluster is now healthy 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-14T18:26:22.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:22 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/771471494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] up:active 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1557439692' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: Cluster is now healthy 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-14T18:26:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:22 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/771471494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-14T18:26:23.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.000+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bc8061c40 con 0x7f7bb40a11d0 2023-12-14T18:26:23.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.049+0000 7f7bbeffd700 1 -- 172.21.15.31:0/1197710152 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7bc8061c40 con 0x7f7bb40a11d0 2023-12-14T18:26:23.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7bb40a11d0 msgr2=0x7f7bb40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7bb40a11d0 0x7f7bb40a3680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400b2e0 tx=0x7f7bc4005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 msgr2=0x7f7bc819d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 0x7f7bc819d530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0007d70 tx=0x7f7bc000a340 comp rx=0 tx=0).stop 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 shutdown_connections 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc8101dc0 0x7f7bc819d530 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7bb40a11d0 0x7f7bb40a3680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc8100240 0x7f7bc8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 --2- 172.21.15.31:0/1197710152 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7bc8101430 0x7f7bc81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 >> 172.21.15.31:0/1197710152 conn(0x7f7bc80fb840 msgr2=0x7f7bc81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 shutdown_connections 2023-12-14T18:26:23.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.051+0000 7f7bd025b700 1 -- 172.21.15.31:0/1197710152 wait complete. 2023-12-14T18:26:23.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-14T18:26:23.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 -- 172.21.15.31:0/2971550725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68100ca0 msgr2=0x7f8b68103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/2971550725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68100ca0 0x7f8b68103080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009990 tx=0x7f8b58009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 -- 172.21.15.31:0/2971550725 shutdown_connections 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/2971550725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68105ee0 0x7f8b68108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/2971550725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68100ca0 0x7f8b68103080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/2971550725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b681035c0 0x7f8b681059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 -- 172.21.15.31:0/2971550725 >> 172.21.15.31:0/2971550725 conn(0x7f8b680fa740 msgr2=0x7f8b680fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 -- 172.21.15.31:0/2971550725 shutdown_connections 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.208+0000 7f8b6d16c700 1 -- 172.21.15.31:0/2971550725 wait complete. 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 Processor -- start 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 -- start start 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68100ca0 0x7f8b68195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b681035c0 0x7f8b68196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 0x7f8b6819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819db50 con 0x7f8b68105ee0 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819dc90 con 0x7f8b68100ca0 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.209+0000 7f8b6d16c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819de20 con 0x7f8b681035c0 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 0x7f8b6819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 0x7f8b6819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53272/0 (socket says 172.21.15.31:53272) 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 -- 172.21.15.31:0/4161108211 learned_addr learned my addr 172.21.15.31:0/4161108211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b66d9d700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68100ca0 0x7f8b68195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b5ffff700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b681035c0 0x7f8b68196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 -- 172.21.15.31:0/4161108211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b681035c0 msgr2=0x7f8b68196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b681035c0 0x7f8b68196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 -- 172.21.15.31:0/4161108211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68100ca0 msgr2=0x7f8b68195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68100ca0 0x7f8b68195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 -- 172.21.15.31:0/4161108211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580095f0 con 0x7f8b68105ee0 2023-12-14T18:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6759e700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 0x7f8b6819d580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b60007ee0 tx=0x7f8b6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000ddd0 con 0x7f8b68105ee0 2023-12-14T18:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b6000f040 con 0x7f8b68105ee0 2023-12-14T18:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6810bb80 con 0x7f8b68105ee0 2023-12-14T18:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60015400 con 0x7f8b68105ee0 2023-12-14T18:26:23.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.210+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6810b9f0 con 0x7f8b68105ee0 2023-12-14T18:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.211+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b680fc2b0 con 0x7f8b68105ee0 2023-12-14T18:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.213+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8b60004000 con 0x7f8b68105ee0 2023-12-14T18:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.213+0000 7f8b64d99700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b540a1150 0x7f8b540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.213+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8b600ccce0 con 0x7f8b68105ee0 2023-12-14T18:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.213+0000 7f8b66d9d700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b540a1150 0x7f8b540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.214+0000 7f8b66d9d700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b540a1150 0x7f8b540a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006010 tx=0x7f8b5800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:23.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.217+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b60096240 con 0x7f8b68105ee0 2023-12-14T18:26:23.463 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:26:23.006+0000 7f2bb4553700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:26:23.006+0000 7f2bb4553700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:26:23.006+0000 7f2bb4553700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:26:23.006+0000 7f2bb4553700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:23.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:26:23.006+0000 7f2bb4553700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:23.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.468+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b680619f0 con 0x7f8b540a1150 2023-12-14T18:26:23.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.472+0000 7f8b64d99700 1 -- 172.21.15.31:0/4161108211 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f8b680619f0 con 0x7f8b540a1150 2023-12-14T18:26:23.474 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/sv_0/418c23a2-4f59-42cc-87c2-1e4b8f33a089 2023-12-14T18:26:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b540a1150 msgr2=0x7f8b540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b540a1150 0x7f8b540a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006010 tx=0x7f8b5800b540 comp rx=0 tx=0).stop 2023-12-14T18:26:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 msgr2=0x7f8b6819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 0x7f8b6819d580 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8b60007ee0 tx=0x7f8b6000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 shutdown_connections 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b68105ee0 0x7f8b6819d580 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b68100ca0 0x7f8b68195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b540a1150 0x7f8b540a3600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 --2- 172.21.15.31:0/4161108211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b681035c0 0x7f8b68196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 >> 172.21.15.31:0/4161108211 conn(0x7f8b680fa740 msgr2=0x7f8b68101940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 shutdown_connections 2023-12-14T18:26:23.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.475+0000 7f8b6d16c700 1 -- 172.21.15.31:0/4161108211 wait complete. 2023-12-14T18:26:23.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-14T18:26:23.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 -- 172.21.15.31:0/1063778231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 msgr2=0x7f1ebc101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 --2- 172.21.15.31:0/1063778231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc101870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009990 tx=0x7f1eac009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 -- 172.21.15.31:0/1063778231 shutdown_connections 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 --2- 172.21.15.31:0/1063778231 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ebc101db0 0x7f1ebc108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 --2- 172.21.15.31:0/1063778231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ebc100230 0x7f1ebc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 --2- 172.21.15.31:0/1063778231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc101870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 -- 172.21.15.31:0/1063778231 >> 172.21.15.31:0/1063778231 conn(0x7f1ebc0fb850 msgr2=0x7f1ebc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 -- 172.21.15.31:0/1063778231 shutdown_connections 2023-12-14T18:26:23.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.631+0000 7f1ec182e700 1 -- 172.21.15.31:0/1063778231 wait complete. 2023-12-14T18:26:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.632+0000 7f1ec182e700 1 Processor -- start 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.632+0000 7f1ec182e700 1 -- start start 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ebc100230 0x7f1ebc195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ebc101db0 0x7f1ebc19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19db40 con 0x7f1ebc101420 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19dc80 con 0x7f1ebc100230 2023-12-14T18:26:23.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19de10 con 0x7f1ebc101db0 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53290/0 (socket says 172.21.15.31:53290) 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 -- 172.21.15.31:0/907879345 learned_addr learned my addr 172.21.15.31:0/907879345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ebaffd700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ebc100230 0x7f1ebc195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ebb7fe700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ebc101db0 0x7f1ebc19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 -- 172.21.15.31:0/907879345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ebc101db0 msgr2=0x7f1ebc19d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ebc101db0 0x7f1ebc19d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 -- 172.21.15.31:0/907879345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ebc100230 msgr2=0x7f1ebc195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ebc100230 0x7f1ebc195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 -- 172.21.15.31:0/907879345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eac0095f0 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1eba7fc700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc196500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005910 tx=0x7f1eac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac01d070 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc10bbf0 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1eac003f60 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.633+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac022da0 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.634+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc10ba60 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.634+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc19e3b0 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.636+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1eac003aa0 con 0x7f1ebc101420 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.636+0000 7f1ec082c700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ea40a1150 0x7f1ea40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:23.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.637+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1eac0ce740 con 0x7f1ebc101420 2023-12-14T18:26:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.637+0000 7f1ebaffd700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ea40a1150 0x7f1ea40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.637+0000 7f1ebaffd700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ea40a1150 0x7f1ea40a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc101280 tx=0x7f1eb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:23.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.640+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1eac097cd0 con 0x7f1ebc101420 2023-12-14T18:26:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:23 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] up:active 2023-12-14T18:26:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:23 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/771471494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-14T18:26:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:23 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-12-14T18:26:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:23 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:26:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:23 smithi204 ceph-mon[43866]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:23.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.885+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ebc060f30 con 0x7f1ea40a1150 2023-12-14T18:26:23.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.910+0000 7f1ec082c700 1 -- 172.21.15.31:0/907879345 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1ebc060f30 con 0x7f1ea40a1150 2023-12-14T18:26:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ea40a1150 msgr2=0x7f1ea40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ea40a1150 0x7f1ea40a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc101280 tx=0x7f1eb000b410 comp rx=0 tx=0).stop 2023-12-14T18:26:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 msgr2=0x7f1ebc196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:23.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc196500 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005910 tx=0x7f1eac004730 comp rx=0 tx=0).stop 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 shutdown_connections 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1ea40a1150 0x7f1ea40a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ebc101db0 0x7f1ebc19d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ebc100230 0x7f1ebc195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 --2- 172.21.15.31:0/907879345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ebc101420 0x7f1ebc196500 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 >> 172.21.15.31:0/907879345 conn(0x7f1ebc0fb850 msgr2=0x7f1ebc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 shutdown_connections 2023-12-14T18:26:23.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:23.913+0000 7f1ec182e700 1 -- 172.21.15.31:0/907879345 wait complete. 2023-12-14T18:26:23.927 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] up:active 2023-12-14T18:26:23.927 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/771471494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-14T18:26:23.927 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-12-14T18:26:23.927 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:26:23.927 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:23 smithi031 ceph-mon[42085]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:23.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-14T18:26:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:23 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] up:active 2023-12-14T18:26:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:23 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/771471494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-14T18:26:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:23 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 9 up:standby 2023-12-14T18:26:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:23 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:26:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:23 smithi176 ceph-mon[43506]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 -- 172.21.15.31:0/2619102278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 msgr2=0x7f1524108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 --2- 172.21.15.31:0/2619102278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f1524108370 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b0a0 tx=0x7f151c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 -- 172.21.15.31:0/2619102278 shutdown_connections 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 --2- 172.21.15.31:0/2619102278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f1524108370 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 --2- 172.21.15.31:0/2619102278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1524100cd0 0x7f15241030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 --2- 172.21.15.31:0/2619102278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15241035f0 0x7f15241059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 -- 172.21.15.31:0/2619102278 >> 172.21.15.31:0/2619102278 conn(0x7f15240fa750 msgr2=0x7f15240fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 -- 172.21.15.31:0/2619102278 shutdown_connections 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.069+0000 7f152a81e700 1 -- 172.21.15.31:0/2619102278 wait complete. 2023-12-14T18:26:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 Processor -- start 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 -- start start 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1524100cd0 0x7f1524100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15241035f0 0x7f15240fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f15240ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240ff960 con 0x7f1524105f10 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15240ffad0 con 0x7f15241035f0 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f152a81e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15241a00b0 con 0x7f1524100cd0 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f1528dbb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f15240ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f1528dbb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f15240ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53312/0 (socket says 172.21.15.31:53312) 2023-12-14T18:26:24.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f1528dbb700 1 -- 172.21.15.31:0/779793240 learned_addr learned my addr 172.21.15.31:0/779793240 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f1523fff700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1524100cd0 0x7f1524100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.070+0000 7f15237fe700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15241035f0 0x7f15240fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f1528dbb700 1 -- 172.21.15.31:0/779793240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1524100cd0 msgr2=0x7f1524100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f1528dbb700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1524100cd0 0x7f1524100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f1528dbb700 1 -- 172.21.15.31:0/779793240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15241035f0 msgr2=0x7f15240fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f1528dbb700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15241035f0 0x7f15240fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f1528dbb700 1 -- 172.21.15.31:0/779793240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009520 con 0x7f1524105f10 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f1528dbb700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f15240ff1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b780 tx=0x7f151c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c00e040 con 0x7f1524105f10 2023-12-14T18:26:24.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151c004c50 con 0x7f1524105f10 2023-12-14T18:26:24.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c009d00 con 0x7f1524105f10 2023-12-14T18:26:24.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c005280 con 0x7f1524105f10 2023-12-14T18:26:24.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.071+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152410bbb0 con 0x7f1524105f10 2023-12-14T18:26:24.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.072+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152410bd40 con 0x7f1524105f10 2023-12-14T18:26:24.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.073+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f151c01d470 con 0x7f1524105f10 2023-12-14T18:26:24.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.074+0000 7f15217fa700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f150c0a1150 0x7f150c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.074+0000 7f1523fff700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f150c0a1150 0x7f150c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.074+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f151c0ce160 con 0x7f1524105f10 2023-12-14T18:26:24.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.075+0000 7f1523fff700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f150c0a1150 0x7f150c0a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1514009df0 tx=0x7f1514009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:24.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.079+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f151c097750 con 0x7f1524105f10 2023-12-14T18:26:24.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.306+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f151c097b20 con 0x7f1524105f10 2023-12-14T18:26:24.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.329+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1524060f30 con 0x7f150c0a1150 2023-12-14T18:26:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.333+0000 7f15217fa700 1 -- 172.21.15.31:0/779793240 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f1524060f30 con 0x7f150c0a1150 2023-12-14T18:26:24.335 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/sv_1/7844f291-1382-4478-8c9e-c751cc2b39de 2023-12-14T18:26:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f150c0a1150 msgr2=0x7f150c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f150c0a1150 0x7f150c0a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1514009df0 tx=0x7f1514009260 comp rx=0 tx=0).stop 2023-12-14T18:26:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 msgr2=0x7f15240ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f15240ff1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b780 tx=0x7f151c00bab0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 shutdown_connections 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1524105f10 0x7f15240ff1c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f150c0a1150 0x7f150c0a3600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1524100cd0 0x7f1524100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 --2- 172.21.15.31:0/779793240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15241035f0 0x7f15240fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 >> 172.21.15.31:0/779793240 conn(0x7f15240fa750 msgr2=0x7f1524101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 shutdown_connections 2023-12-14T18:26:24.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.336+0000 7f152a81e700 1 -- 172.21.15.31:0/779793240 wait complete. 2023-12-14T18:26:24.339 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:26:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: 2023-12-14T18:26:24.032+0000 7fdf8b999700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-14T18:26:24.339 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:26:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: 2023-12-14T18:26:24.032+0000 7fdf8b999700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-12-14T18:26:24.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:26:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 -- 172.21.15.31:0/2120290903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100ce0 msgr2=0x7f0cac1075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 --2- 172.21.15.31:0/2120290903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100ce0 0x7f0cac1075a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400b0a0 tx=0x7f0ca400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 -- 172.21.15.31:0/2120290903 shutdown_connections 2023-12-14T18:26:24.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 --2- 172.21.15.31:0/2120290903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100ce0 0x7f0cac1075a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 --2- 172.21.15.31:0/2120290903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 0x7f0cac0ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 --2- 172.21.15.31:0/2120290903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cac100350 0x7f0cac1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 -- 172.21.15.31:0/2120290903 >> 172.21.15.31:0/2120290903 conn(0x7f0cac0fa780 msgr2=0x7f0cac0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 -- 172.21.15.31:0/2120290903 shutdown_connections 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.488+0000 7f0cb2811700 1 -- 172.21.15.31:0/2120290903 wait complete. 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 Processor -- start 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 -- start start 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 0x7f0cac196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 0x7f0cac1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cac100ce0 0x7f0cac19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac19dc00 con 0x7f0cac100350 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac19dd40 con 0x7f0cac0ff160 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb2811700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac19ded0 con 0x7f0cac100ce0 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 0x7f0cac1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 0x7f0cac1965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35308/0 (socket says 172.21.15.31:35308) 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cab7fe700 1 -- 172.21.15.31:0/853200192 learned_addr learned my addr 172.21.15.31:0/853200192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cb0dae700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cac100ce0 0x7f0cac19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.489+0000 7f0cabfff700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 0x7f0cac196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cab7fe700 1 -- 172.21.15.31:0/853200192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cac100ce0 msgr2=0x7f0cac19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cab7fe700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cac100ce0 0x7f0cac19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cab7fe700 1 -- 172.21.15.31:0/853200192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 msgr2=0x7f0cac196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cab7fe700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 0x7f0cac196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cab7fe700 1 -- 172.21.15.31:0/853200192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009d00 con 0x7f0cac100350 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cabfff700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 0x7f0cac196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cab7fe700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 0x7f0cac1965c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000d950 tx=0x7f0ca000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0009940 con 0x7f0cac100350 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ca0010460 con 0x7f0cac100350 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cac10aad0 con 0x7f0cac100350 2023-12-14T18:26:24.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000f620 con 0x7f0cac100350 2023-12-14T18:26:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.490+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac10a940 con 0x7f0cac100350 2023-12-14T18:26:24.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.491+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cac067160 con 0x7f0cac100350 2023-12-14T18:26:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.492+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0ca00108d0 con 0x7f0cac100350 2023-12-14T18:26:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.493+0000 7f0ca97fa700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0c940a11e0 0x7f0c940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.493+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0ca00ccb00 con 0x7f0cac100350 2023-12-14T18:26:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.493+0000 7f0cabfff700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0c940a11e0 0x7f0c940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.493+0000 7f0cabfff700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0c940a11e0 0x7f0c940a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0cac1001b0 tx=0x7f0c9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:24.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.498+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0ca0095f00 con 0x7f0cac100350 2023-12-14T18:26:24.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:24 smithi031 ceph-mon[42085]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:24 smithi031 ceph-mon[42085]: pgmap v243: 129 pgs: 129 active+clean; 583 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.3 KiB/s wr, 3 op/s 2023-12-14T18:26:24.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:24 smithi031 ceph-mon[42085]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:24 smithi031 ceph-mon[42085]: mgrmap e17: x(active, since 5m), standbys: z, y 2023-12-14T18:26:24.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.753+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cac19eb70 con 0x7f0cac100350 2023-12-14T18:26:24.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.754+0000 7f0ca97fa700 1 -- 172.21.15.31:0/853200192 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f0ca0016090 con 0x7f0cac100350 2023-12-14T18:26:24.755 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:24.755 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:26:24.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.756+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0c940a11e0 msgr2=0x7f0c940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.756+0000 7f0cb2811700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0c940a11e0 0x7f0c940a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0cac1001b0 tx=0x7f0c9c009260 comp rx=0 tx=0).stop 2023-12-14T18:26:24.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.756+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 msgr2=0x7f0cac1965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.756+0000 7f0cb2811700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 0x7f0cac1965c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000d950 tx=0x7f0ca000dc60 comp rx=0 tx=0).stop 2023-12-14T18:26:24.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 shutdown_connections 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0c940a11e0 0x7f0c940a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0cac100ce0 0x7f0cac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cac0ff160 0x7f0cac196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 --2- 172.21.15.31:0/853200192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cac100350 0x7f0cac1965c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 >> 172.21.15.31:0/853200192 conn(0x7f0cac0fa780 msgr2=0x7f0cac068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 shutdown_connections 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.757+0000 7f0cb2811700 1 -- 172.21.15.31:0/853200192 wait complete. 2023-12-14T18:26:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:26:24.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:24.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:24 smithi204 ceph-mon[43866]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:24 smithi204 ceph-mon[43866]: pgmap v243: 129 pgs: 129 active+clean; 583 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.3 KiB/s wr, 3 op/s 2023-12-14T18:26:24.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:24 smithi204 ceph-mon[43866]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:24 smithi204 ceph-mon[43866]: mgrmap e17: x(active, since 5m), standbys: z, y 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 -- 172.21.15.31:0/656367502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc101420 msgr2=0x7f49fc101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 --2- 172.21.15.31:0/656367502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc101420 0x7f49fc101870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009990 tx=0x7f49f0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 -- 172.21.15.31:0/656367502 shutdown_connections 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 --2- 172.21.15.31:0/656367502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49fc101db0 0x7f49fc108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 --2- 172.21.15.31:0/656367502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49fc100230 0x7f49fc100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 --2- 172.21.15.31:0/656367502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc101420 0x7f49fc101870 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 -- 172.21.15.31:0/656367502 >> 172.21.15.31:0/656367502 conn(0x7f49fc0fb810 msgr2=0x7f49fc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 -- 172.21.15.31:0/656367502 shutdown_connections 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.908+0000 7f4a0337f700 1 -- 172.21.15.31:0/656367502 wait complete. 2023-12-14T18:26:24.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 Processor -- start 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 -- start start 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 0x7f49fc195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49fc101420 0x7f49fc1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49fc101db0 0x7f49fc19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc19db10 con 0x7f49fc100230 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc19dc50 con 0x7f49fc101db0 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0337f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc19dde0 con 0x7f49fc101420 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0111b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 0x7f49fc195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0111b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 0x7f49fc195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35328/0 (socket says 172.21.15.31:35328) 2023-12-14T18:26:24.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0111b700 1 -- 172.21.15.31:0/533831389 learned_addr learned my addr 172.21.15.31:0/533831389 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.909+0000 7f4a0191c700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49fc101db0 0x7f49fc19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0091a700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49fc101420 0x7f49fc1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0111b700 1 -- 172.21.15.31:0/533831389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49fc101420 msgr2=0x7f49fc1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0111b700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49fc101420 0x7f49fc1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0111b700 1 -- 172.21.15.31:0/533831389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49fc101db0 msgr2=0x7f49fc19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0111b700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49fc101db0 0x7f49fc19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:24.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0111b700 1 -- 172.21.15.31:0/533831389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00095f0 con 0x7f49fc100230 2023-12-14T18:26:24.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0111b700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 0x7f49fc195f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f49f800d980 tx=0x7f49f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:24.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800b950 con 0x7f49fc100230 2023-12-14T18:26:24.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc10bb90 con 0x7f49fc100230 2023-12-14T18:26:24.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49f8007620 con 0x7f49fc100230 2023-12-14T18:26:24.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800f3c0 con 0x7f49fc100230 2023-12-14T18:26:24.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.910+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc10ba00 con 0x7f49fc100230 2023-12-14T18:26:24.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.911+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc067160 con 0x7f49fc100230 2023-12-14T18:26:24.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.912+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f49f8010640 con 0x7f49fc100230 2023-12-14T18:26:24.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.913+0000 7f49ee7fc700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49e80a11e0 0x7f49e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:24.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.913+0000 7f4a0091a700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49e80a11e0 0x7f49e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:24.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.913+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f49f80ccb70 con 0x7f49fc100230 2023-12-14T18:26:24.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.914+0000 7f4a0091a700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49e80a11e0 0x7f49e80a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006010 tx=0x7f49f000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:24.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:24.918+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f49f8095f70 con 0x7f49fc100230 2023-12-14T18:26:24.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:24 smithi176 ceph-mon[43506]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:24 smithi176 ceph-mon[43506]: pgmap v243: 129 pgs: 129 active+clean; 583 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.3 KiB/s wr, 3 op/s 2023-12-14T18:26:24.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:24 smithi176 ceph-mon[43506]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:24.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:24 smithi176 ceph-mon[43506]: mgrmap e17: x(active, since 5m), standbys: z, y 2023-12-14T18:26:25.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.130+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49fc107160 con 0x7f49fc100230 2023-12-14T18:26:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.132+0000 7f49ee7fc700 1 -- 172.21.15.31:0/533831389 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7f49f8099590 con 0x7f49fc100230 2023-12-14T18:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49e80a11e0 msgr2=0x7f49e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49e80a11e0 0x7f49e80a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006010 tx=0x7f49f000b540 comp rx=0 tx=0).stop 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 msgr2=0x7f49fc195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 0x7f49fc195f90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f49f800d980 tx=0x7f49f800dc90 comp rx=0 tx=0).stop 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 shutdown_connections 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49fc101db0 0x7f49fc19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49fc100230 0x7f49fc195f90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49e80a11e0 0x7f49e80a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 --2- 172.21.15.31:0/533831389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49fc101420 0x7f49fc1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 >> 172.21.15.31:0/533831389 conn(0x7f49fc0fb810 msgr2=0x7f49fc0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 shutdown_connections 2023-12-14T18:26:25.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.134+0000 7f4a0337f700 1 -- 172.21.15.31:0/533831389 wait complete. 2023-12-14T18:26:25.150 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-14T18:26:25.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:26:25.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 -- 172.21.15.31:0/1150220929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c105f10 msgr2=0x7fad1c108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 --2- 172.21.15.31:0/1150220929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c105f10 0x7fad1c108370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fad1400b0a0 tx=0x7fad1400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 -- 172.21.15.31:0/1150220929 shutdown_connections 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 --2- 172.21.15.31:0/1150220929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c105f10 0x7fad1c108370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 --2- 172.21.15.31:0/1150220929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c100cd0 0x7fad1c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 --2- 172.21.15.31:0/1150220929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad1c1035f0 0x7fad1c1059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 -- 172.21.15.31:0/1150220929 >> 172.21.15.31:0/1150220929 conn(0x7fad1c0fa730 msgr2=0x7fad1c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 -- 172.21.15.31:0/1150220929 shutdown_connections 2023-12-14T18:26:25.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.286+0000 7fad208d3700 1 -- 172.21.15.31:0/1150220929 wait complete. 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 Processor -- start 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 -- start start 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 0x7fad1c1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1035f0 0x7fad1c196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad1c105f10 0x7fad1c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c19dc40 con 0x7fad1c100cd0 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c19dd80 con 0x7fad1c105f10 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.287+0000 7fad208d3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c19df10 con 0x7fad1c1035f0 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 0x7fad1c1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 0x7fad1c1960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35338/0 (socket says 172.21.15.31:35338) 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 -- 172.21.15.31:0/59309589 learned_addr learned my addr 172.21.15.31:0/59309589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad19d9b700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1035f0 0x7fad1c196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1ad9d700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad1c105f10 0x7fad1c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 -- 172.21.15.31:0/59309589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1035f0 msgr2=0x7fad1c196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1035f0 0x7fad1c196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 -- 172.21.15.31:0/59309589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad1c105f10 msgr2=0x7fad1c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad1c105f10 0x7fad1c19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 -- 172.21.15.31:0/59309589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14009d00 con 0x7fad1c100cd0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad1a59c700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 0x7fad1c1960c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00e9a0 tx=0x7fad0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c009790 con 0x7fad1c100cd0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad0c00cbd0 con 0x7fad1c100cd0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c018680 con 0x7fad1c100cd0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c10bc60 con 0x7fad1c100cd0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.288+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c10bad0 con 0x7fad1c100cd0 2023-12-14T18:26:25.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.289+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c0fc2a0 con 0x7fad1c100cd0 2023-12-14T18:26:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.291+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fad0c00cd40 con 0x7fad1c100cd0 2023-12-14T18:26:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.291+0000 7fad0b7fe700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fad040a1260 0x7fad040a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.292+0000 7fad19d9b700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fad040a1260 0x7fad040a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.292+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fad0c014070 con 0x7fad1c100cd0 2023-12-14T18:26:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.292+0000 7fad19d9b700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fad040a1260 0x7fad040a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad10009c60 tx=0x7fad10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:25.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.295+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fad0c096520 con 0x7fad1c100cd0 2023-12-14T18:26:25.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.500+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad1c061c90 con 0x7fad040a1260 2023-12-14T18:26:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.513+0000 7fad0b7fe700 1 -- 172.21.15.31:0/59309589 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418066 (secure 0 0 0) 0x7fad1c061c90 con 0x7fad040a1260 2023-12-14T18:26:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fad040a1260 msgr2=0x7fad040a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fad040a1260 0x7fad040a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fad10009c60 tx=0x7fad10009260 comp rx=0 tx=0).stop 2023-12-14T18:26:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 msgr2=0x7fad1c1960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 0x7fad1c1960c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00e9a0 tx=0x7fad0c00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 shutdown_connections 2023-12-14T18:26:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad1c105f10 0x7fad1c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad1c100cd0 0x7fad1c1960c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fad040a1260 0x7fad040a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 --2- 172.21.15.31:0/59309589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad1c1035f0 0x7fad1c196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.517+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 >> 172.21.15.31:0/59309589 conn(0x7fad1c0fa730 msgr2=0x7fad1c101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.518+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 shutdown_connections 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.518+0000 7fad208d3700 1 -- 172.21.15.31:0/59309589 wait complete. 2023-12-14T18:26:25.531 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:26:25.533 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2023-12-14T18:26:25.476784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632552,"num_objects":56,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":155,"num_read_kb":153,"num_write":305,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":248,"ondisk_log_size":248,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79316,"kb_used_data":11188,"kb_used_omap":0,"kb_used_meta":68032,"kb_avail":1124862508,"statfs":{"total":1151940427776,"available":1151859208192,"internally_reserved":0,"allocated":11456512,"data_stored":7369373,"data_compressed":247512,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69664768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37734,"num_objects":32,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":34,"num_write":77,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003683"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261626+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:00.261626+0000","last_peered":"2023-12-14T18:26:00.261626+0000","last_clean":"2023-12-14T18:26:00.261626+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:00.261626+0000","last_undegraded":"2023-12-14T18:26:00.261626+0000","last_fullsized":"2023-12-14T18:26:00.261626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:50.100097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260549+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:00.260549+0000","last_peered":"2023-12-14T18:26:00.260549+0000","last_clean":"2023-12-14T18:26:00.260549+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:00.260549+0000","last_undegraded":"2023-12-14T18:26:00.260549+0000","last_fullsized":"2023-12-14T18:26:00.260549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:55:13.469285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260329+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:00.260329+0000","last_peered":"2023-12-14T18:26:00.260329+0000","last_clean":"2023-12-14T18:26:00.260329+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:00.260329+0000","last_undegraded":"2023-12-14T18:26:00.260329+0000","last_fullsized":"2023-12-14T18:26:00.260329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:28.361542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096659+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:00.096659+0000","last_peered":"2023-12-14T18:26:00.096659+0000","last_clean":"2023-12-14T18:26:00.096659+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:00.096659+0000","last_undegraded":"2023-12-14T18:26:00.096659+0000","last_fullsized":"2023-12-14T18:26:00.096659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:26:49.516693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261413+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:00.261413+0000","last_peered":"2023-12-14T18:26:00.261413+0000","last_clean":"2023-12-14T18:26:00.261413+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:00.261413+0000","last_undegraded":"2023-12-14T18:26:00.261413+0000","last_fullsized":"2023-12-14T18:26:00.261413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:30:31.503322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784119+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:25:59.784119+0000","last_peered":"2023-12-14T18:25:59.784119+0000","last_clean":"2023-12-14T18:25:59.784119+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:25:59.784119+0000","last_undegraded":"2023-12-14T18:25:59.784119+0000","last_fullsized":"2023-12-14T18:25:59.784119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:19:46.100215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260889+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:00.260889+0000","last_peered":"2023-12-14T18:26:00.260889+0000","last_clean":"2023-12-14T18:26:00.260889+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:00.260889+0000","last_undegraded":"2023-12-14T18:26:00.260889+0000","last_fullsized":"2023-12-14T18:26:00.260889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:59:55.977518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260998+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:00.260998+0000","last_peered":"2023-12-14T18:26:00.260998+0000","last_clean":"2023-12-14T18:26:00.260998+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:00.260998+0000","last_undegraded":"2023-12-14T18:26:00.260998+0000","last_fullsized":"2023-12-14T18:26:00.260998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:39.415624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260479+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:00.260479+0000","last_peered":"2023-12-14T18:26:00.260479+0000","last_clean":"2023-12-14T18:26:00.260479+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:00.260479+0000","last_undegraded":"2023-12-14T18:26:00.260479+0000","last_fullsized":"2023-12-14T18:26:00.260479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:11.718555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784600+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:25:59.784600+0000","last_peered":"2023-12-14T18:25:59.784600+0000","last_clean":"2023-12-14T18:25:59.784600+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:25:59.784600+0000","last_undegraded":"2023-12-14T18:25:59.784600+0000","last_fullsized":"2023-12-14T18:25:59.784600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:49:08.435988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261384+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:00.261384+0000","last_peered":"2023-12-14T18:26:00.261384+0000","last_clean":"2023-12-14T18:26:00.261384+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:00.261384+0000","last_undegraded":"2023-12-14T18:26:00.261384+0000","last_fullsized":"2023-12-14T18:26:00.261384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:04:17.200866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096607+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:00.096607+0000","last_peered":"2023-12-14T18:26:00.096607+0000","last_clean":"2023-12-14T18:26:00.096607+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:00.096607+0000","last_undegraded":"2023-12-14T18:26:00.096607+0000","last_fullsized":"2023-12-14T18:26:00.096607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:00.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689738+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:00.689738+0000","last_peered":"2023-12-14T18:26:00.689738+0000","last_clean":"2023-12-14T18:26:00.689738+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:00.689738+0000","last_undegraded":"2023-12-14T18:26:00.689738+0000","last_fullsized":"2023-12-14T18:26:00.689738+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:44.738081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259857+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:00.259857+0000","last_peered":"2023-12-14T18:26:00.259857+0000","last_clean":"2023-12-14T18:26:00.259857+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:00.259857+0000","last_undegraded":"2023-12-14T18:26:00.259857+0000","last_fullsized":"2023-12-14T18:26:00.259857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:07:36.273043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259796+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:00.259796+0000","last_peered":"2023-12-14T18:26:00.259796+0000","last_clean":"2023-12-14T18:26:00.259796+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:00.259796+0000","last_undegraded":"2023-12-14T18:26:00.259796+0000","last_fullsized":"2023-12-14T18:26:00.259796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:58:32.333906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259822+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:00.259822+0000","last_peered":"2023-12-14T18:26:00.259822+0000","last_clean":"2023-12-14T18:26:00.259822+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:00.259822+0000","last_undegraded":"2023-12-14T18:26:00.259822+0000","last_fullsized":"2023-12-14T18:26:00.259822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:23:15.944985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261073+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:00.261073+0000","last_peered":"2023-12-14T18:26:00.261073+0000","last_clean":"2023-12-14T18:26:00.261073+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:00.261073+0000","last_undegraded":"2023-12-14T18:26:00.261073+0000","last_fullsized":"2023-12-14T18:26:00.261073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:22:40.190452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783795+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:25:59.783795+0000","last_peered":"2023-12-14T18:25:59.783795+0000","last_clean":"2023-12-14T18:25:59.783795+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:25:59.783795+0000","last_undegraded":"2023-12-14T18:25:59.783795+0000","last_fullsized":"2023-12-14T18:25:59.783795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:44.187839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784400+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:25:59.784400+0000","last_peered":"2023-12-14T18:25:59.784400+0000","last_clean":"2023-12-14T18:25:59.784400+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:25:59.784400+0000","last_undegraded":"2023-12-14T18:25:59.784400+0000","last_fullsized":"2023-12-14T18:25:59.784400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:26:58.991735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783970+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:25:59.783970+0000","last_peered":"2023-12-14T18:25:59.783970+0000","last_clean":"2023-12-14T18:25:59.783970+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:25:59.783970+0000","last_undegraded":"2023-12-14T18:25:59.783970+0000","last_fullsized":"2023-12-14T18:25:59.783970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:42:15.259545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783907+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:25:59.783907+0000","last_peered":"2023-12-14T18:25:59.783907+0000","last_clean":"2023-12-14T18:25:59.783907+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:25:59.783907+0000","last_undegraded":"2023-12-14T18:25:59.783907+0000","last_fullsized":"2023-12-14T18:25:59.783907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:28.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261992+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:00.261992+0000","last_peered":"2023-12-14T18:26:00.261992+0000","last_clean":"2023-12-14T18:26:00.261992+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:00.261992+0000","last_undegraded":"2023-12-14T18:26:00.261992+0000","last_fullsized":"2023-12-14T18:26:00.261992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:28:53.531523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096009+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:00.096009+0000","last_peered":"2023-12-14T18:26:00.096009+0000","last_clean":"2023-12-14T18:26:00.096009+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:00.096009+0000","last_undegraded":"2023-12-14T18:26:00.096009+0000","last_fullsized":"2023-12-14T18:26:00.096009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:58:11.032456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.262117+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:00.262117+0000","last_peered":"2023-12-14T18:26:00.262117+0000","last_clean":"2023-12-14T18:26:00.262117+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:00.262117+0000","last_undegraded":"2023-12-14T18:26:00.262117+0000","last_fullsized":"2023-12-14T18:26:00.262117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:49:55.507582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260790+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:00.260790+0000","last_peered":"2023-12-14T18:26:00.260790+0000","last_clean":"2023-12-14T18:26:00.260790+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:00.260790+0000","last_undegraded":"2023-12-14T18:26:00.260790+0000","last_fullsized":"2023-12-14T18:26:00.260790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:06:10.842072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096117+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:00.096117+0000","last_peered":"2023-12-14T18:26:00.096117+0000","last_clean":"2023-12-14T18:26:00.096117+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:00.096117+0000","last_undegraded":"2023-12-14T18:26:00.096117+0000","last_fullsized":"2023-12-14T18:26:00.096117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:18.832885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.580396+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:21.580396+0000","last_peered":"2023-12-14T18:26:21.580396+0000","last_clean":"2023-12-14T18:26:21.580396+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:21.580396+0000","last_undegraded":"2023-12-14T18:26:21.580396+0000","last_fullsized":"2023-12-14T18:26:21.580396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:27:29.369178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260164+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:00.260164+0000","last_peered":"2023-12-14T18:26:00.260164+0000","last_clean":"2023-12-14T18:26:00.260164+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:00.260164+0000","last_undegraded":"2023-12-14T18:26:00.260164+0000","last_fullsized":"2023-12-14T18:26:00.260164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:20:33.695577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096040+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:00.096040+0000","last_peered":"2023-12-14T18:26:00.096040+0000","last_clean":"2023-12-14T18:26:00.096040+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:00.096040+0000","last_undegraded":"2023-12-14T18:26:00.096040+0000","last_fullsized":"2023-12-14T18:26:00.096040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:13.719713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.262302+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:00.262302+0000","last_peered":"2023-12-14T18:26:00.262302+0000","last_clean":"2023-12-14T18:26:00.262302+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:00.262302+0000","last_undegraded":"2023-12-14T18:26:00.262302+0000","last_fullsized":"2023-12-14T18:26:00.262302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:26.993698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261459+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:00.261459+0000","last_peered":"2023-12-14T18:26:00.261459+0000","last_clean":"2023-12-14T18:26:00.261459+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:00.261459+0000","last_undegraded":"2023-12-14T18:26:00.261459+0000","last_fullsized":"2023-12-14T18:26:00.261459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:16:42.962687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784427+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:25:59.784427+0000","last_peered":"2023-12-14T18:25:59.784427+0000","last_clean":"2023-12-14T18:25:59.784427+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:25:59.784427+0000","last_undegraded":"2023-12-14T18:25:59.784427+0000","last_fullsized":"2023-12-14T18:25:59.784427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:59:28.675903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567704+0000","last_change":"2023-12-14T18:25:54.853066+0000","last_active":"2023-12-14T18:26:20.567704+0000","last_peered":"2023-12-14T18:26:20.567704+0000","last_clean":"2023-12-14T18:26:20.567704+0000","last_became_active":"2023-12-14T18:25:54.852663+0000","last_became_peered":"2023-12-14T18:25:54.852663+0000","last_unstale":"2023-12-14T18:26:20.567704+0000","last_undegraded":"2023-12-14T18:26:20.567704+0000","last_fullsized":"2023-12-14T18:26:20.567704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:41.765930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261259+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:00.261259+0000","last_peered":"2023-12-14T18:26:00.261259+0000","last_clean":"2023-12-14T18:26:00.261259+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:00.261259+0000","last_undegraded":"2023-12-14T18:26:00.261259+0000","last_fullsized":"2023-12-14T18:26:00.261259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:01:59.504071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784181+0000","last_change":"2023-12-14T18:25:59.344884+0000","last_active":"2023-12-14T18:25:59.784181+0000","last_peered":"2023-12-14T18:25:59.784181+0000","last_clean":"2023-12-14T18:25:59.784181+0000","last_became_active":"2023-12-14T18:25:59.344494+0000","last_became_peered":"2023-12-14T18:25:59.344494+0000","last_unstale":"2023-12-14T18:25:59.784181+0000","last_undegraded":"2023-12-14T18:25:59.784181+0000","last_fullsized":"2023-12-14T18:25:59.784181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:03:55.332790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.581473+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:21.581473+0000","last_peered":"2023-12-14T18:26:21.581473+0000","last_clean":"2023-12-14T18:26:21.581473+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:21.581473+0000","last_undegraded":"2023-12-14T18:26:21.581473+0000","last_fullsized":"2023-12-14T18:26:21.581473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:39.969717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261169+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:00.261169+0000","last_peered":"2023-12-14T18:26:00.261169+0000","last_clean":"2023-12-14T18:26:00.261169+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:00.261169+0000","last_undegraded":"2023-12-14T18:26:00.261169+0000","last_fullsized":"2023-12-14T18:26:00.261169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:21:01.585382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260973+0000","last_change":"2023-12-14T18:25:59.337495+0000","last_active":"2023-12-14T18:26:00.260973+0000","last_peered":"2023-12-14T18:26:00.260973+0000","last_clean":"2023-12-14T18:26:00.260973+0000","last_became_active":"2023-12-14T18:25:59.337072+0000","last_became_peered":"2023-12-14T18:25:59.337072+0000","last_unstale":"2023-12-14T18:26:00.260973+0000","last_undegraded":"2023-12-14T18:26:00.260973+0000","last_fullsized":"2023-12-14T18:26:00.260973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:57:35.575059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689415+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:00.689415+0000","last_peered":"2023-12-14T18:26:00.689415+0000","last_clean":"2023-12-14T18:26:00.689415+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:00.689415+0000","last_undegraded":"2023-12-14T18:26:00.689415+0000","last_fullsized":"2023-12-14T18:26:00.689415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:11:16.910059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260864+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:00.260864+0000","last_peered":"2023-12-14T18:26:00.260864+0000","last_clean":"2023-12-14T18:26:00.260864+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:00.260864+0000","last_undegraded":"2023-12-14T18:26:00.260864+0000","last_fullsized":"2023-12-14T18:26:00.260864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:36.097770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260576+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:00.260576+0000","last_peered":"2023-12-14T18:26:00.260576+0000","last_clean":"2023-12-14T18:26:00.260576+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:00.260576+0000","last_undegraded":"2023-12-14T18:26:00.260576+0000","last_fullsized":"2023-12-14T18:26:00.260576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:21:14.664141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260191+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:00.260191+0000","last_peered":"2023-12-14T18:26:00.260191+0000","last_clean":"2023-12-14T18:26:00.260191+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:00.260191+0000","last_undegraded":"2023-12-14T18:26:00.260191+0000","last_fullsized":"2023-12-14T18:26:00.260191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:39.435686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783256+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:25:59.783256+0000","last_peered":"2023-12-14T18:25:59.783256+0000","last_clean":"2023-12-14T18:25:59.783256+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:25:59.783256+0000","last_undegraded":"2023-12-14T18:25:59.783256+0000","last_fullsized":"2023-12-14T18:25:59.783256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:48:20.799919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261237+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:00.261237+0000","last_peered":"2023-12-14T18:26:00.261237+0000","last_clean":"2023-12-14T18:26:00.261237+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:00.261237+0000","last_undegraded":"2023-12-14T18:26:00.261237+0000","last_fullsized":"2023-12-14T18:26:00.261237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:39:37.673400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:22.585112+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:26:22.585112+0000","last_peered":"2023-12-14T18:26:22.585112+0000","last_clean":"2023-12-14T18:26:22.585112+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:26:22.585112+0000","last_undegraded":"2023-12-14T18:26:22.585112+0000","last_fullsized":"2023-12-14T18:26:22.585112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:22:35.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260416+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:00.260416+0000","last_peered":"2023-12-14T18:26:00.260416+0000","last_clean":"2023-12-14T18:26:00.260416+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:00.260416+0000","last_undegraded":"2023-12-14T18:26:00.260416+0000","last_fullsized":"2023-12-14T18:26:00.260416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:43:31.511979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783182+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:25:59.783182+0000","last_peered":"2023-12-14T18:25:59.783182+0000","last_clean":"2023-12-14T18:25:59.783182+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:25:59.783182+0000","last_undegraded":"2023-12-14T18:25:59.783182+0000","last_fullsized":"2023-12-14T18:25:59.783182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:51:41.408987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567338+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:20.567338+0000","last_peered":"2023-12-14T18:26:20.567338+0000","last_clean":"2023-12-14T18:26:20.567338+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:20.567338+0000","last_undegraded":"2023-12-14T18:26:20.567338+0000","last_fullsized":"2023-12-14T18:26:20.567338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:32:04.781498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260658+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:00.260658+0000","last_peered":"2023-12-14T18:26:00.260658+0000","last_clean":"2023-12-14T18:26:00.260658+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:00.260658+0000","last_undegraded":"2023-12-14T18:26:00.260658+0000","last_fullsized":"2023-12-14T18:26:00.260658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:08:17.341655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096343+0000","last_change":"2023-12-14T18:25:59.342905+0000","last_active":"2023-12-14T18:26:00.096343+0000","last_peered":"2023-12-14T18:26:00.096343+0000","last_clean":"2023-12-14T18:26:00.096343+0000","last_became_active":"2023-12-14T18:25:59.342417+0000","last_became_peered":"2023-12-14T18:25:59.342417+0000","last_unstale":"2023-12-14T18:26:00.096343+0000","last_undegraded":"2023-12-14T18:26:00.096343+0000","last_fullsized":"2023-12-14T18:26:00.096343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:05.418599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261423+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:00.261423+0000","last_peered":"2023-12-14T18:26:00.261423+0000","last_clean":"2023-12-14T18:26:00.261423+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:00.261423+0000","last_undegraded":"2023-12-14T18:26:00.261423+0000","last_fullsized":"2023-12-14T18:26:00.261423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:17.005591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260368+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:00.260368+0000","last_peered":"2023-12-14T18:26:00.260368+0000","last_clean":"2023-12-14T18:26:00.260368+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:00.260368+0000","last_undegraded":"2023-12-14T18:26:00.260368+0000","last_fullsized":"2023-12-14T18:26:00.260368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:39:02.954403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783341+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:25:59.783341+0000","last_peered":"2023-12-14T18:25:59.783341+0000","last_clean":"2023-12-14T18:25:59.783341+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:25:59.783341+0000","last_undegraded":"2023-12-14T18:25:59.783341+0000","last_fullsized":"2023-12-14T18:25:59.783341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:37:49.702818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'23","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:23.903406+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:26:23.903406+0000","last_peered":"2023-12-14T18:26:23.903406+0000","last_clean":"2023-12-14T18:26:23.903406+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:26:23.903406+0000","last_undegraded":"2023-12-14T18:26:23.903406+0000","last_fullsized":"2023-12-14T18:26:23.903406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:56:23.209261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":24,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261747+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:00.261747+0000","last_peered":"2023-12-14T18:26:00.261747+0000","last_clean":"2023-12-14T18:26:00.261747+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:00.261747+0000","last_undegraded":"2023-12-14T18:26:00.261747+0000","last_fullsized":"2023-12-14T18:26:00.261747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:25:58.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261773+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:00.261773+0000","last_peered":"2023-12-14T18:26:00.261773+0000","last_clean":"2023-12-14T18:26:00.261773+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:00.261773+0000","last_undegraded":"2023-12-14T18:26:00.261773+0000","last_fullsized":"2023-12-14T18:26:00.261773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:14:26.658005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.579829+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:21.579829+0000","last_peered":"2023-12-14T18:26:21.579829+0000","last_clean":"2023-12-14T18:26:21.579829+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:21.579829+0000","last_undegraded":"2023-12-14T18:26:21.579829+0000","last_fullsized":"2023-12-14T18:26:21.579829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:28.651606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689579+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:00.689579+0000","last_peered":"2023-12-14T18:26:00.689579+0000","last_clean":"2023-12-14T18:26:00.689579+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:00.689579+0000","last_undegraded":"2023-12-14T18:26:00.689579+0000","last_fullsized":"2023-12-14T18:26:00.689579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:54.349300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260580+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:00.260580+0000","last_peered":"2023-12-14T18:26:00.260580+0000","last_clean":"2023-12-14T18:26:00.260580+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:00.260580+0000","last_undegraded":"2023-12-14T18:26:00.260580+0000","last_fullsized":"2023-12-14T18:26:00.260580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:21:10.802288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567563+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:20.567563+0000","last_peered":"2023-12-14T18:26:20.567563+0000","last_clean":"2023-12-14T18:26:20.567563+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:20.567563+0000","last_undegraded":"2023-12-14T18:26:20.567563+0000","last_fullsized":"2023-12-14T18:26:20.567563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:24:25.354538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261195+0000","last_change":"2023-12-14T18:25:55.316217+0000","last_active":"2023-12-14T18:26:00.261195+0000","last_peered":"2023-12-14T18:26:00.261195+0000","last_clean":"2023-12-14T18:26:00.261195+0000","last_became_active":"2023-12-14T18:25:55.315842+0000","last_became_peered":"2023-12-14T18:25:55.315842+0000","last_unstale":"2023-12-14T18:26:00.261195+0000","last_undegraded":"2023-12-14T18:26:00.261195+0000","last_fullsized":"2023-12-14T18:26:00.261195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:25:51.857936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783806+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:25:59.783806+0000","last_peered":"2023-12-14T18:25:59.783806+0000","last_clean":"2023-12-14T18:25:59.783806+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:25:59.783806+0000","last_undegraded":"2023-12-14T18:25:59.783806+0000","last_fullsized":"2023-12-14T18:25:59.783806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:50:45.642801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:22.588667+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:26:22.588667+0000","last_peered":"2023-12-14T18:26:22.588667+0000","last_clean":"2023-12-14T18:26:22.588667+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:26:22.588667+0000","last_undegraded":"2023-12-14T18:26:22.588667+0000","last_fullsized":"2023-12-14T18:26:22.588667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:57:18.362489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260931+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:00.260931+0000","last_peered":"2023-12-14T18:26:00.260931+0000","last_clean":"2023-12-14T18:26:00.260931+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:00.260931+0000","last_undegraded":"2023-12-14T18:26:00.260931+0000","last_fullsized":"2023-12-14T18:26:00.260931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:28:53.590151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.690000+0000","last_change":"2023-12-14T18:25:59.348833+0000","last_active":"2023-12-14T18:26:00.690000+0000","last_peered":"2023-12-14T18:26:00.690000+0000","last_clean":"2023-12-14T18:26:00.690000+0000","last_became_active":"2023-12-14T18:25:59.348367+0000","last_became_peered":"2023-12-14T18:25:59.348367+0000","last_unstale":"2023-12-14T18:26:00.690000+0000","last_undegraded":"2023-12-14T18:26:00.690000+0000","last_fullsized":"2023-12-14T18:26:00.690000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:33:20.898030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.582089+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:21.582089+0000","last_peered":"2023-12-14T18:26:21.582089+0000","last_clean":"2023-12-14T18:26:21.582089+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:21.582089+0000","last_undegraded":"2023-12-14T18:26:21.582089+0000","last_fullsized":"2023-12-14T18:26:21.582089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:53:54.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260781+0000","last_change":"2023-12-14T18:25:55.305447+0000","last_active":"2023-12-14T18:26:00.260781+0000","last_peered":"2023-12-14T18:26:00.260781+0000","last_clean":"2023-12-14T18:26:00.260781+0000","last_became_active":"2023-12-14T18:25:55.305054+0000","last_became_peered":"2023-12-14T18:25:55.305054+0000","last_unstale":"2023-12-14T18:26:00.260781+0000","last_undegraded":"2023-12-14T18:26:00.260781+0000","last_fullsized":"2023-12-14T18:26:00.260781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:52:51.044461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260692+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:00.260692+0000","last_peered":"2023-12-14T18:26:00.260692+0000","last_clean":"2023-12-14T18:26:00.260692+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:00.260692+0000","last_undegraded":"2023-12-14T18:26:00.260692+0000","last_fullsized":"2023-12-14T18:26:00.260692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:24.105560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.262002+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:00.262002+0000","last_peered":"2023-12-14T18:26:00.262002+0000","last_clean":"2023-12-14T18:26:00.262002+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:00.262002+0000","last_undegraded":"2023-12-14T18:26:00.262002+0000","last_fullsized":"2023-12-14T18:26:00.262002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:57:29.873612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260746+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:00.260746+0000","last_peered":"2023-12-14T18:26:00.260746+0000","last_clean":"2023-12-14T18:26:00.260746+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:00.260746+0000","last_undegraded":"2023-12-14T18:26:00.260746+0000","last_fullsized":"2023-12-14T18:26:00.260746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:23:48.805350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261174+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:00.261174+0000","last_peered":"2023-12-14T18:26:00.261174+0000","last_clean":"2023-12-14T18:26:00.261174+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:00.261174+0000","last_undegraded":"2023-12-14T18:26:00.261174+0000","last_fullsized":"2023-12-14T18:26:00.261174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:30:54.421948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783421+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:25:59.783421+0000","last_peered":"2023-12-14T18:25:59.783421+0000","last_clean":"2023-12-14T18:25:59.783421+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:25:59.783421+0000","last_undegraded":"2023-12-14T18:25:59.783421+0000","last_fullsized":"2023-12-14T18:25:59.783421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:18:04.320749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784394+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:25:59.784394+0000","last_peered":"2023-12-14T18:25:59.784394+0000","last_clean":"2023-12-14T18:25:59.784394+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:25:59.784394+0000","last_undegraded":"2023-12-14T18:25:59.784394+0000","last_fullsized":"2023-12-14T18:25:59.784394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:03:42.297304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689797+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:00.689797+0000","last_peered":"2023-12-14T18:26:00.689797+0000","last_clean":"2023-12-14T18:26:00.689797+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:00.689797+0000","last_undegraded":"2023-12-14T18:26:00.689797+0000","last_fullsized":"2023-12-14T18:26:00.689797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:19:10.100748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260990+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:00.260990+0000","last_peered":"2023-12-14T18:26:00.260990+0000","last_clean":"2023-12-14T18:26:00.260990+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:00.260990+0000","last_undegraded":"2023-12-14T18:26:00.260990+0000","last_fullsized":"2023-12-14T18:26:00.260990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:23:18.627153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784335+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:25:59.784335+0000","last_peered":"2023-12-14T18:25:59.784335+0000","last_clean":"2023-12-14T18:25:59.784335+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:25:59.784335+0000","last_undegraded":"2023-12-14T18:25:59.784335+0000","last_fullsized":"2023-12-14T18:25:59.784335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.499504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689863+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:00.689863+0000","last_peered":"2023-12-14T18:26:00.689863+0000","last_clean":"2023-12-14T18:26:00.689863+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:00.689863+0000","last_undegraded":"2023-12-14T18:26:00.689863+0000","last_fullsized":"2023-12-14T18:26:00.689863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:26.633680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260298+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:00.260298+0000","last_peered":"2023-12-14T18:26:00.260298+0000","last_clean":"2023-12-14T18:26:00.260298+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:00.260298+0000","last_undegraded":"2023-12-14T18:26:00.260298+0000","last_fullsized":"2023-12-14T18:26:00.260298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:23:55.056652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689316+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:26:00.689316+0000","last_peered":"2023-12-14T18:26:00.689316+0000","last_clean":"2023-12-14T18:26:00.689316+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:26:00.689316+0000","last_undegraded":"2023-12-14T18:26:00.689316+0000","last_fullsized":"2023-12-14T18:26:00.689316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:10:53.820789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261316+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:00.261316+0000","last_peered":"2023-12-14T18:26:00.261316+0000","last_clean":"2023-12-14T18:26:00.261316+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:00.261316+0000","last_undegraded":"2023-12-14T18:26:00.261316+0000","last_fullsized":"2023-12-14T18:26:00.261316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:35:25.807560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261645+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:00.261645+0000","last_peered":"2023-12-14T18:26:00.261645+0000","last_clean":"2023-12-14T18:26:00.261645+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:00.261645+0000","last_undegraded":"2023-12-14T18:26:00.261645+0000","last_fullsized":"2023-12-14T18:26:00.261645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:35:11.047723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:22.583401+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:22.583401+0000","last_peered":"2023-12-14T18:26:22.583401+0000","last_clean":"2023-12-14T18:26:22.583401+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:22.583401+0000","last_undegraded":"2023-12-14T18:26:22.583401+0000","last_fullsized":"2023-12-14T18:26:22.583401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:42:38.074930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260822+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:00.260822+0000","last_peered":"2023-12-14T18:26:00.260822+0000","last_clean":"2023-12-14T18:26:00.260822+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:00.260822+0000","last_undegraded":"2023-12-14T18:26:00.260822+0000","last_fullsized":"2023-12-14T18:26:00.260822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:27:04.810823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260773+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:00.260773+0000","last_peered":"2023-12-14T18:26:00.260773+0000","last_clean":"2023-12-14T18:26:00.260773+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:00.260773+0000","last_undegraded":"2023-12-14T18:26:00.260773+0000","last_fullsized":"2023-12-14T18:26:00.260773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:17:01.451483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689259+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:00.689259+0000","last_peered":"2023-12-14T18:26:00.689259+0000","last_clean":"2023-12-14T18:26:00.689259+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:00.689259+0000","last_undegraded":"2023-12-14T18:26:00.689259+0000","last_fullsized":"2023-12-14T18:26:00.689259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:32:23.251389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.579722+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:21.579722+0000","last_peered":"2023-12-14T18:26:21.579722+0000","last_clean":"2023-12-14T18:26:21.579722+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:21.579722+0000","last_undegraded":"2023-12-14T18:26:21.579722+0000","last_fullsized":"2023-12-14T18:26:21.579722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:28:35.723032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261019+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:00.261019+0000","last_peered":"2023-12-14T18:26:00.261019+0000","last_clean":"2023-12-14T18:26:00.261019+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:00.261019+0000","last_undegraded":"2023-12-14T18:26:00.261019+0000","last_fullsized":"2023-12-14T18:26:00.261019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:40:04.222426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261943+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:00.261943+0000","last_peered":"2023-12-14T18:26:00.261943+0000","last_clean":"2023-12-14T18:26:00.261943+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:00.261943+0000","last_undegraded":"2023-12-14T18:26:00.261943+0000","last_fullsized":"2023-12-14T18:26:00.261943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:55:28.262805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689898+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:00.689898+0000","last_peered":"2023-12-14T18:26:00.689898+0000","last_clean":"2023-12-14T18:26:00.689898+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:00.689898+0000","last_undegraded":"2023-12-14T18:26:00.689898+0000","last_fullsized":"2023-12-14T18:26:00.689898+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:35:00.581814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260787+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:00.260787+0000","last_peered":"2023-12-14T18:26:00.260787+0000","last_clean":"2023-12-14T18:26:00.260787+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:00.260787+0000","last_undegraded":"2023-12-14T18:26:00.260787+0000","last_fullsized":"2023-12-14T18:26:00.260787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:28:31.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783879+0000","last_change":"2023-12-14T18:25:54.852944+0000","last_active":"2023-12-14T18:25:59.783879+0000","last_peered":"2023-12-14T18:25:59.783879+0000","last_clean":"2023-12-14T18:25:59.783879+0000","last_became_active":"2023-12-14T18:25:54.852565+0000","last_became_peered":"2023-12-14T18:25:54.852565+0000","last_unstale":"2023-12-14T18:25:59.783879+0000","last_undegraded":"2023-12-14T18:25:59.783879+0000","last_fullsized":"2023-12-14T18:25:59.783879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:49.128592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261642+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:00.261642+0000","last_peered":"2023-12-14T18:26:00.261642+0000","last_clean":"2023-12-14T18:26:00.261642+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:00.261642+0000","last_undegraded":"2023-12-14T18:26:00.261642+0000","last_fullsized":"2023-12-14T18:26:00.261642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:39:39.802371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783842+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:25:59.783842+0000","last_peered":"2023-12-14T18:25:59.783842+0000","last_clean":"2023-12-14T18:25:59.783842+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:25:59.783842+0000","last_undegraded":"2023-12-14T18:25:59.783842+0000","last_fullsized":"2023-12-14T18:25:59.783842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:42.465645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.565619+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:20.565619+0000","last_peered":"2023-12-14T18:26:20.565619+0000","last_clean":"2023-12-14T18:26:20.565619+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:20.565619+0000","last_undegraded":"2023-12-14T18:26:20.565619+0000","last_fullsized":"2023-12-14T18:26:20.565619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:45:18.559609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260676+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:00.260676+0000","last_peered":"2023-12-14T18:26:00.260676+0000","last_clean":"2023-12-14T18:26:00.260676+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:00.260676+0000","last_undegraded":"2023-12-14T18:26:00.260676+0000","last_fullsized":"2023-12-14T18:26:00.260676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:12:07.158191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096316+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:00.096316+0000","last_peered":"2023-12-14T18:26:00.096316+0000","last_clean":"2023-12-14T18:26:00.096316+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:00.096316+0000","last_undegraded":"2023-12-14T18:26:00.096316+0000","last_fullsized":"2023-12-14T18:26:00.096316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:43.864201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.566288+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:20.566288+0000","last_peered":"2023-12-14T18:26:20.566288+0000","last_clean":"2023-12-14T18:26:20.566288+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:20.566288+0000","last_undegraded":"2023-12-14T18:26:20.566288+0000","last_fullsized":"2023-12-14T18:26:20.566288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:49:16.660803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260683+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:00.260683+0000","last_peered":"2023-12-14T18:26:00.260683+0000","last_clean":"2023-12-14T18:26:00.260683+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:00.260683+0000","last_undegraded":"2023-12-14T18:26:00.260683+0000","last_fullsized":"2023-12-14T18:26:00.260683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:27:42.833359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783900+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:25:59.783900+0000","last_peered":"2023-12-14T18:25:59.783900+0000","last_clean":"2023-12-14T18:25:59.783900+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:25:59.783900+0000","last_undegraded":"2023-12-14T18:25:59.783900+0000","last_fullsized":"2023-12-14T18:25:59.783900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:59.055992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567068+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:20.567068+0000","last_peered":"2023-12-14T18:26:20.567068+0000","last_clean":"2023-12-14T18:26:20.567068+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:20.567068+0000","last_undegraded":"2023-12-14T18:26:20.567068+0000","last_fullsized":"2023-12-14T18:26:20.567068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:54:18.694073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260875+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:00.260875+0000","last_peered":"2023-12-14T18:26:00.260875+0000","last_clean":"2023-12-14T18:26:00.260875+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:00.260875+0000","last_undegraded":"2023-12-14T18:26:00.260875+0000","last_fullsized":"2023-12-14T18:26:00.260875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:53:27.261881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096368+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:00.096368+0000","last_peered":"2023-12-14T18:26:00.096368+0000","last_clean":"2023-12-14T18:26:00.096368+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:00.096368+0000","last_undegraded":"2023-12-14T18:26:00.096368+0000","last_fullsized":"2023-12-14T18:26:00.096368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:22.345101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783853+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:25:59.783853+0000","last_peered":"2023-12-14T18:25:59.783853+0000","last_clean":"2023-12-14T18:25:59.783853+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:25:59.783853+0000","last_undegraded":"2023-12-14T18:25:59.783853+0000","last_fullsized":"2023-12-14T18:25:59.783853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:47.086736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261151+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:00.261151+0000","last_peered":"2023-12-14T18:26:00.261151+0000","last_clean":"2023-12-14T18:26:00.261151+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:00.261151+0000","last_undegraded":"2023-12-14T18:26:00.261151+0000","last_fullsized":"2023-12-14T18:26:00.261151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:54:54.197194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096435+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:00.096435+0000","last_peered":"2023-12-14T18:26:00.096435+0000","last_clean":"2023-12-14T18:26:00.096435+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:00.096435+0000","last_undegraded":"2023-12-14T18:26:00.096435+0000","last_fullsized":"2023-12-14T18:26:00.096435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:29:39.716647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.565419+0000","last_change":"2023-12-14T18:25:54.850903+0000","last_active":"2023-12-14T18:26:20.565419+0000","last_peered":"2023-12-14T18:26:20.565419+0000","last_clean":"2023-12-14T18:26:20.565419+0000","last_became_active":"2023-12-14T18:25:54.850502+0000","last_became_peered":"2023-12-14T18:25:54.850502+0000","last_unstale":"2023-12-14T18:26:20.565419+0000","last_undegraded":"2023-12-14T18:26:20.565419+0000","last_fullsized":"2023-12-14T18:26:20.565419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:58:08.788921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261131+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:00.261131+0000","last_peered":"2023-12-14T18:26:00.261131+0000","last_clean":"2023-12-14T18:26:00.261131+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:00.261131+0000","last_undegraded":"2023-12-14T18:26:00.261131+0000","last_fullsized":"2023-12-14T18:26:00.261131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:42:23.981776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260129+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:00.260129+0000","last_peered":"2023-12-14T18:26:00.260129+0000","last_clean":"2023-12-14T18:26:00.260129+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:00.260129+0000","last_undegraded":"2023-12-14T18:26:00.260129+0000","last_fullsized":"2023-12-14T18:26:00.260129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:16:10.873684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.582563+0000","last_change":"2023-12-14T18:25:54.860133+0000","last_active":"2023-12-14T18:26:21.582563+0000","last_peered":"2023-12-14T18:26:21.582563+0000","last_clean":"2023-12-14T18:26:21.582563+0000","last_became_active":"2023-12-14T18:25:54.859782+0000","last_became_peered":"2023-12-14T18:25:54.859782+0000","last_unstale":"2023-12-14T18:26:21.582563+0000","last_undegraded":"2023-12-14T18:26:21.582563+0000","last_fullsized":"2023-12-14T18:26:21.582563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:03:24.896495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261523+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:00.261523+0000","last_peered":"2023-12-14T18:26:00.261523+0000","last_clean":"2023-12-14T18:26:00.261523+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:00.261523+0000","last_undegraded":"2023-12-14T18:26:00.261523+0000","last_fullsized":"2023-12-14T18:26:00.261523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:57:46.923639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259985+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:00.259985+0000","last_peered":"2023-12-14T18:26:00.259985+0000","last_clean":"2023-12-14T18:26:00.259985+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:00.259985+0000","last_undegraded":"2023-12-14T18:26:00.259985+0000","last_fullsized":"2023-12-14T18:26:00.259985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:17.333368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261488+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:00.261488+0000","last_peered":"2023-12-14T18:26:00.261488+0000","last_clean":"2023-12-14T18:26:00.261488+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:00.261488+0000","last_undegraded":"2023-12-14T18:26:00.261488+0000","last_fullsized":"2023-12-14T18:26:00.261488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:32.435550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261370+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:00.261370+0000","last_peered":"2023-12-14T18:26:00.261370+0000","last_clean":"2023-12-14T18:26:00.261370+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:00.261370+0000","last_undegraded":"2023-12-14T18:26:00.261370+0000","last_fullsized":"2023-12-14T18:26:00.261370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:55:17.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260121+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:00.260121+0000","last_peered":"2023-12-14T18:26:00.260121+0000","last_clean":"2023-12-14T18:26:00.260121+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:00.260121+0000","last_undegraded":"2023-12-14T18:26:00.260121+0000","last_fullsized":"2023-12-14T18:26:00.260121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:06.010260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689833+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:00.689833+0000","last_peered":"2023-12-14T18:26:00.689833+0000","last_clean":"2023-12-14T18:26:00.689833+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:00.689833+0000","last_undegraded":"2023-12-14T18:26:00.689833+0000","last_fullsized":"2023-12-14T18:26:00.689833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:23:23.345143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261510+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:00.261510+0000","last_peered":"2023-12-14T18:26:00.261510+0000","last_clean":"2023-12-14T18:26:00.261510+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:00.261510+0000","last_undegraded":"2023-12-14T18:26:00.261510+0000","last_fullsized":"2023-12-14T18:26:00.261510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:00:54.564106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261020+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:00.261020+0000","last_peered":"2023-12-14T18:26:00.261020+0000","last_clean":"2023-12-14T18:26:00.261020+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:00.261020+0000","last_undegraded":"2023-12-14T18:26:00.261020+0000","last_fullsized":"2023-12-14T18:26:00.261020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:10:55.378712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784089+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:25:59.784089+0000","last_peered":"2023-12-14T18:25:59.784089+0000","last_clean":"2023-12-14T18:25:59.784089+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:25:59.784089+0000","last_undegraded":"2023-12-14T18:25:59.784089+0000","last_fullsized":"2023-12-14T18:25:59.784089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:37.784010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783831+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:25:59.783831+0000","last_peered":"2023-12-14T18:25:59.783831+0000","last_clean":"2023-12-14T18:25:59.783831+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:25:59.783831+0000","last_undegraded":"2023-12-14T18:25:59.783831+0000","last_fullsized":"2023-12-14T18:25:59.783831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:34:55.579739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260069+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:00.260069+0000","last_peered":"2023-12-14T18:26:00.260069+0000","last_clean":"2023-12-14T18:26:00.260069+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:00.260069+0000","last_undegraded":"2023-12-14T18:26:00.260069+0000","last_fullsized":"2023-12-14T18:26:00.260069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:27:04.926460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689932+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:00.689932+0000","last_peered":"2023-12-14T18:26:00.689932+0000","last_clean":"2023-12-14T18:26:00.689932+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:00.689932+0000","last_undegraded":"2023-12-14T18:26:00.689932+0000","last_fullsized":"2023-12-14T18:26:00.689932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:59:17.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261452+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:00.261452+0000","last_peered":"2023-12-14T18:26:00.261452+0000","last_clean":"2023-12-14T18:26:00.261452+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:00.261452+0000","last_undegraded":"2023-12-14T18:26:00.261452+0000","last_fullsized":"2023-12-14T18:26:00.261452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:33.344146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784499+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:25:59.784499+0000","last_peered":"2023-12-14T18:25:59.784499+0000","last_clean":"2023-12-14T18:25:59.784499+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:25:59.784499+0000","last_undegraded":"2023-12-14T18:25:59.784499+0000","last_fullsized":"2023-12-14T18:25:59.784499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:38.320340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261896+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:00.261896+0000","last_peered":"2023-12-14T18:26:00.261896+0000","last_clean":"2023-12-14T18:26:00.261896+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:00.261896+0000","last_undegraded":"2023-12-14T18:26:00.261896+0000","last_fullsized":"2023-12-14T18:26:00.261896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:49:32.561679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260230+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:00.260230+0000","last_peered":"2023-12-14T18:26:00.260230+0000","last_clean":"2023-12-14T18:26:00.260230+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:00.260230+0000","last_undegraded":"2023-12-14T18:26:00.260230+0000","last_fullsized":"2023-12-14T18:26:00.260230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:20.517919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260923+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:00.260923+0000","last_peered":"2023-12-14T18:26:00.260923+0000","last_clean":"2023-12-14T18:26:00.260923+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:00.260923+0000","last_undegraded":"2023-12-14T18:26:00.260923+0000","last_fullsized":"2023-12-14T18:26:00.260923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:47:43.144080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260248+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:00.260248+0000","last_peered":"2023-12-14T18:26:00.260248+0000","last_clean":"2023-12-14T18:26:00.260248+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:00.260248+0000","last_undegraded":"2023-12-14T18:26:00.260248+0000","last_fullsized":"2023-12-14T18:26:00.260248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:42:31.252913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689577+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:00.689577+0000","last_peered":"2023-12-14T18:26:00.689577+0000","last_clean":"2023-12-14T18:26:00.689577+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:00.689577+0000","last_undegraded":"2023-12-14T18:26:00.689577+0000","last_fullsized":"2023-12-14T18:26:00.689577+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:50:54.626392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260687+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:00.260687+0000","last_peered":"2023-12-14T18:26:00.260687+0000","last_clean":"2023-12-14T18:26:00.260687+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:00.260687+0000","last_undegraded":"2023-12-14T18:26:00.260687+0000","last_fullsized":"2023-12-14T18:26:00.260687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:49:08.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41938,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":19,"num_read_kb":34,"num_write":122,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":91539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6028,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739124,"statfs":{"total":95995035648,"available":95988862976,"internally_reserved":0,"allocated":921600,"data_stored":460250,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":10,"up_from":68,"seq":292057776147,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":1032192,"data_stored":1073041,"data_compressed":47404,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.503}]},{"osd":5,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.145}]},{"osd":9,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.4960000000000004,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.60499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":9,"up_from":63,"seq":270582939669,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6224,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738928,"statfs":{"total":95995035648,"available":95988662272,"internally_reserved":0,"allocated":925696,"data_stored":460832,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.312}]},{"osd":1,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.45600000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.72999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":8,"up_from":55,"seq":236223201306,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6368,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738784,"statfs":{"total":95995035648,"available":95988514816,"internally_reserved":0,"allocated":942080,"data_stored":460988,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.628},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.89900000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6468,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738684,"statfs":{"total":95995035648,"available":95988412416,"internally_reserved":0,"allocated":978944,"data_stored":1051710,"data_compressed":47404,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.41099999999999998,"15min":0.39700000000000002},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.40799999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.752,"5min":1.8,"15min":1.8},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.36199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63}]},{"osd":5,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.88300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.59599999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84899999999999998,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.55000000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Thu Dec 14 18:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.96499999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.872},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":6,"up_from":43,"seq":184683593761,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6348,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738804,"statfs":{"total":95995035648,"available":95988535296,"internally_reserved":0,"allocated":921600,"data_stored":460250,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.32100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.42899999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59899999999999998,"15min":0.58699999999999997},"min":{"1min":0.52800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.749}]},{"osd":3,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.627}]},{"osd":4,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70999999999999996},"min":{"1min":0.622,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.76600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85599999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.91000000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":498491,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.502,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50600000000000001,"15min":0.49199999999999999},"min":{"1min":0.39600000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.495}]},{"osd":2,"last update":"Thu Dec 14 18:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.48999999999999999,"15min":0.46600000000000003},"min":{"1min":0.41999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.61099999999999999,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.3150000000000004,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.61299999999999999,"15min":0.54700000000000004},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.312,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70899999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.52200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.44900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78600000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.48199999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.752}]},{"osd":8,"last update":"Thu Dec 14 18:25:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85299999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.3570000000000002,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":5.1159999999999997,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.81799999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7272,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737880,"statfs":{"total":95995035648,"available":95987589120,"internally_reserved":0,"allocated":950272,"data_stored":461728,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.52900000000000003,"15min":0.5},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.627}]},{"osd":3,"last update":"Thu Dec 14 18:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.308,"5min":0.251,"15min":0.251},"max":{"1min":0.70399999999999996,"5min":0.751,"15min":0.751},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.32700000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:26:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":1.091,"15min":1.091},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93100000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.44},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.69999999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.747},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7280,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737872,"statfs":{"total":95995035648,"available":95987580928,"internally_reserved":0,"allocated":958464,"data_stored":465345,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.36599999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90400000000000003,"5min":1.484,"15min":1.484},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.34100000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71699999999999997,"5min":0.75,"15min":0.75},"last":0.748}]},{"osd":5,"last update":"Thu Dec 14 18:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.376},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":1.514,"15min":1.514},"last":0.505}]},{"osd":6,"last update":"Thu Dec 14 18:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92500000000000004,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215147,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":460081,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38200000000000001},"min":{"1min":0.36499999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.40899999999999997},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.503},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.747},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.62}]},{"osd":5,"last update":"Thu Dec 14 18:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.629,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95199999999999996,"5min":1.093,"15min":1.093},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.623,"15min":0.62},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.441},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47999999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986217,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":937984,"data_stored":464942,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.29099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71799999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.504},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":1.121,"15min":1.121},"last":0.379}]},{"osd":5,"last update":"Thu Dec 14 18:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.42599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.435},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.879,"15min":0.879},"last":0.46100000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.73999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.52200000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822693,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6612,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738540,"statfs":{"total":95995035648,"available":95988264960,"internally_reserved":0,"allocated":995328,"data_stored":1051715,"data_compressed":47404,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66400000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.622},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67600000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58099999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71399999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:26:25.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:26:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 -- 172.21.15.31:0/849460670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 msgr2=0x7f6da4107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 --2- 172.21.15.31:0/849460670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da4107590 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b0a0 tx=0x7f6d9c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 -- 172.21.15.31:0/849460670 shutdown_connections 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 --2- 172.21.15.31:0/849460670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da4107590 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 --2- 172.21.15.31:0/849460670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da40ff150 0x7f6da40ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 --2- 172.21.15.31:0/849460670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da4100340 0x7f6da4100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 -- 172.21.15.31:0/849460670 >> 172.21.15.31:0/849460670 conn(0x7f6da40fa730 msgr2=0x7f6da40fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 -- 172.21.15.31:0/849460670 shutdown_connections 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.712+0000 7f6dac09d700 1 -- 172.21.15.31:0/849460670 wait complete. 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.713+0000 7f6dac09d700 1 Processor -- start 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.713+0000 7f6dac09d700 1 -- start start 2023-12-14T18:26:25.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6dac09d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da40ff150 0x7f6da4194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6dac09d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da4100340 0x7f6da41954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6dac09d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6dac09d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da419caf0 con 0x7f6da4100cd0 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6dac09d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da419cc30 con 0x7f6da4100340 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6dac09d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da419cdc0 con 0x7f6da40ff150 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35348/0 (socket says 172.21.15.31:35348) 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 -- 172.21.15.31:0/3632051077 learned_addr learned my addr 172.21.15.31:0/3632051077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:25.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6da9638700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da4100340 0x7f6da41954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6da9e39700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da40ff150 0x7f6da4194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 -- 172.21.15.31:0/3632051077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da40ff150 msgr2=0x7f6da4194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da40ff150 0x7f6da4194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 -- 172.21.15.31:0/3632051077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da4100340 msgr2=0x7f6da41954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da4100340 0x7f6da41954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 -- 172.21.15.31:0/3632051077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c009d00 con 0x7f6da4100cd0 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6da9638700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da4100340 0x7f6da41954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6daa63a700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da419c520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b780 tx=0x7f6d9c00bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:25.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.714+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c00e040 con 0x7f6da4100cd0 2023-12-14T18:26:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.715+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da410aac0 con 0x7f6da4100cd0 2023-12-14T18:26:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.715+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d9c004e60 con 0x7f6da4100cd0 2023-12-14T18:26:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.715+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c01b720 con 0x7f6da4100cd0 2023-12-14T18:26:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.715+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da410a930 con 0x7f6da4100cd0 2023-12-14T18:26:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.716+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da4067160 con 0x7f6da4100cd0 2023-12-14T18:26:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.717+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6d9c01b880 con 0x7f6da4100cd0 2023-12-14T18:26:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.717+0000 7f6d9affd700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6d900a11e0 0x7f6d900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.717+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6d9c0ce450 con 0x7f6da4100cd0 2023-12-14T18:26:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.717+0000 7f6da9e39700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6d900a11e0 0x7f6d900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:25.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.718+0000 7f6da9e39700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6d900a11e0 0x7f6d900a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6da41001a0 tx=0x7f6da0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.722+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6d9c0978e0 con 0x7f6da4100cd0 2023-12-14T18:26:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:25 smithi204 ceph-mon[43866]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:25 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/853200192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:25 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/533831389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:25.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.931+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6da419d7e0 con 0x7f6d900a11e0 2023-12-14T18:26:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.943+0000 7f6d9affd700 1 -- 172.21.15.31:0/3632051077 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418066 (secure 0 0 0) 0x7f6da419d7e0 con 0x7f6d900a11e0 2023-12-14T18:26:25.951 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:25.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6d900a11e0 msgr2=0x7f6d900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6d900a11e0 0x7f6d900a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6da41001a0 tx=0x7f6da0009260 comp rx=0 tx=0).stop 2023-12-14T18:26:25.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 msgr2=0x7f6da419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da419c520 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00b780 tx=0x7f6d9c00bdb0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 shutdown_connections 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6da4100cd0 0x7f6da419c520 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6d900a11e0 0x7f6d900a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6da40ff150 0x7f6da4194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 --2- 172.21.15.31:0/3632051077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6da4100340 0x7f6da41954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 >> 172.21.15.31:0/3632051077 conn(0x7f6da40fa730 msgr2=0x7f6da40fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 shutdown_connections 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:25.948+0000 7f6dac09d700 1 -- 172.21.15.31:0/3632051077 wait complete. 2023-12-14T18:26:25.958 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:26:25.961 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:25 smithi031 ceph-mon[42085]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:25.962 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:25 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/853200192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:25.962 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:25 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/533831389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:25.963 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2023-12-14T18:26:25.476784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632552,"num_objects":56,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":155,"num_read_kb":153,"num_write":305,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":248,"ondisk_log_size":248,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79316,"kb_used_data":11188,"kb_used_omap":0,"kb_used_meta":68032,"kb_avail":1124862508,"statfs":{"total":1151940427776,"available":1151859208192,"internally_reserved":0,"allocated":11456512,"data_stored":7369373,"data_compressed":247512,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69664768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37734,"num_objects":32,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":34,"num_write":77,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003683"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261626+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:00.261626+0000","last_peered":"2023-12-14T18:26:00.261626+0000","last_clean":"2023-12-14T18:26:00.261626+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:00.261626+0000","last_undegraded":"2023-12-14T18:26:00.261626+0000","last_fullsized":"2023-12-14T18:26:00.261626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:50.100097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260549+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:00.260549+0000","last_peered":"2023-12-14T18:26:00.260549+0000","last_clean":"2023-12-14T18:26:00.260549+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:00.260549+0000","last_undegraded":"2023-12-14T18:26:00.260549+0000","last_fullsized":"2023-12-14T18:26:00.260549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:55:13.469285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260329+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:00.260329+0000","last_peered":"2023-12-14T18:26:00.260329+0000","last_clean":"2023-12-14T18:26:00.260329+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:00.260329+0000","last_undegraded":"2023-12-14T18:26:00.260329+0000","last_fullsized":"2023-12-14T18:26:00.260329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:28.361542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096659+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:00.096659+0000","last_peered":"2023-12-14T18:26:00.096659+0000","last_clean":"2023-12-14T18:26:00.096659+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:00.096659+0000","last_undegraded":"2023-12-14T18:26:00.096659+0000","last_fullsized":"2023-12-14T18:26:00.096659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:26:49.516693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261413+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:00.261413+0000","last_peered":"2023-12-14T18:26:00.261413+0000","last_clean":"2023-12-14T18:26:00.261413+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:00.261413+0000","last_undegraded":"2023-12-14T18:26:00.261413+0000","last_fullsized":"2023-12-14T18:26:00.261413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:30:31.503322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784119+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:25:59.784119+0000","last_peered":"2023-12-14T18:25:59.784119+0000","last_clean":"2023-12-14T18:25:59.784119+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:25:59.784119+0000","last_undegraded":"2023-12-14T18:25:59.784119+0000","last_fullsized":"2023-12-14T18:25:59.784119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:19:46.100215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260889+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:00.260889+0000","last_peered":"2023-12-14T18:26:00.260889+0000","last_clean":"2023-12-14T18:26:00.260889+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:00.260889+0000","last_undegraded":"2023-12-14T18:26:00.260889+0000","last_fullsized":"2023-12-14T18:26:00.260889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:59:55.977518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260998+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:00.260998+0000","last_peered":"2023-12-14T18:26:00.260998+0000","last_clean":"2023-12-14T18:26:00.260998+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:00.260998+0000","last_undegraded":"2023-12-14T18:26:00.260998+0000","last_fullsized":"2023-12-14T18:26:00.260998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:39.415624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260479+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:00.260479+0000","last_peered":"2023-12-14T18:26:00.260479+0000","last_clean":"2023-12-14T18:26:00.260479+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:00.260479+0000","last_undegraded":"2023-12-14T18:26:00.260479+0000","last_fullsized":"2023-12-14T18:26:00.260479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:11.718555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784600+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:25:59.784600+0000","last_peered":"2023-12-14T18:25:59.784600+0000","last_clean":"2023-12-14T18:25:59.784600+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:25:59.784600+0000","last_undegraded":"2023-12-14T18:25:59.784600+0000","last_fullsized":"2023-12-14T18:25:59.784600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:49:08.435988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261384+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:00.261384+0000","last_peered":"2023-12-14T18:26:00.261384+0000","last_clean":"2023-12-14T18:26:00.261384+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:00.261384+0000","last_undegraded":"2023-12-14T18:26:00.261384+0000","last_fullsized":"2023-12-14T18:26:00.261384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:04:17.200866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096607+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:00.096607+0000","last_peered":"2023-12-14T18:26:00.096607+0000","last_clean":"2023-12-14T18:26:00.096607+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:00.096607+0000","last_undegraded":"2023-12-14T18:26:00.096607+0000","last_fullsized":"2023-12-14T18:26:00.096607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:00.042878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689738+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:00.689738+0000","last_peered":"2023-12-14T18:26:00.689738+0000","last_clean":"2023-12-14T18:26:00.689738+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:00.689738+0000","last_undegraded":"2023-12-14T18:26:00.689738+0000","last_fullsized":"2023-12-14T18:26:00.689738+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:44.738081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259857+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:00.259857+0000","last_peered":"2023-12-14T18:26:00.259857+0000","last_clean":"2023-12-14T18:26:00.259857+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:00.259857+0000","last_undegraded":"2023-12-14T18:26:00.259857+0000","last_fullsized":"2023-12-14T18:26:00.259857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:07:36.273043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259796+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:00.259796+0000","last_peered":"2023-12-14T18:26:00.259796+0000","last_clean":"2023-12-14T18:26:00.259796+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:00.259796+0000","last_undegraded":"2023-12-14T18:26:00.259796+0000","last_fullsized":"2023-12-14T18:26:00.259796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:58:32.333906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259822+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:00.259822+0000","last_peered":"2023-12-14T18:26:00.259822+0000","last_clean":"2023-12-14T18:26:00.259822+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:00.259822+0000","last_undegraded":"2023-12-14T18:26:00.259822+0000","last_fullsized":"2023-12-14T18:26:00.259822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:23:15.944985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261073+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:00.261073+0000","last_peered":"2023-12-14T18:26:00.261073+0000","last_clean":"2023-12-14T18:26:00.261073+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:00.261073+0000","last_undegraded":"2023-12-14T18:26:00.261073+0000","last_fullsized":"2023-12-14T18:26:00.261073+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:22:40.190452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783795+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:25:59.783795+0000","last_peered":"2023-12-14T18:25:59.783795+0000","last_clean":"2023-12-14T18:25:59.783795+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:25:59.783795+0000","last_undegraded":"2023-12-14T18:25:59.783795+0000","last_fullsized":"2023-12-14T18:25:59.783795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:44.187839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784400+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:25:59.784400+0000","last_peered":"2023-12-14T18:25:59.784400+0000","last_clean":"2023-12-14T18:25:59.784400+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:25:59.784400+0000","last_undegraded":"2023-12-14T18:25:59.784400+0000","last_fullsized":"2023-12-14T18:25:59.784400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:26:58.991735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783970+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:25:59.783970+0000","last_peered":"2023-12-14T18:25:59.783970+0000","last_clean":"2023-12-14T18:25:59.783970+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:25:59.783970+0000","last_undegraded":"2023-12-14T18:25:59.783970+0000","last_fullsized":"2023-12-14T18:25:59.783970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:42:15.259545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'4","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783907+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:25:59.783907+0000","last_peered":"2023-12-14T18:25:59.783907+0000","last_clean":"2023-12-14T18:25:59.783907+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:25:59.783907+0000","last_undegraded":"2023-12-14T18:25:59.783907+0000","last_fullsized":"2023-12-14T18:25:59.783907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:28.490880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261992+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:00.261992+0000","last_peered":"2023-12-14T18:26:00.261992+0000","last_clean":"2023-12-14T18:26:00.261992+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:00.261992+0000","last_undegraded":"2023-12-14T18:26:00.261992+0000","last_fullsized":"2023-12-14T18:26:00.261992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:28:53.531523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096009+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:00.096009+0000","last_peered":"2023-12-14T18:26:00.096009+0000","last_clean":"2023-12-14T18:26:00.096009+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:00.096009+0000","last_undegraded":"2023-12-14T18:26:00.096009+0000","last_fullsized":"2023-12-14T18:26:00.096009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:58:11.032456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"80'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.262117+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:00.262117+0000","last_peered":"2023-12-14T18:26:00.262117+0000","last_clean":"2023-12-14T18:26:00.262117+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:00.262117+0000","last_undegraded":"2023-12-14T18:26:00.262117+0000","last_fullsized":"2023-12-14T18:26:00.262117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:49:55.507582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260790+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:00.260790+0000","last_peered":"2023-12-14T18:26:00.260790+0000","last_clean":"2023-12-14T18:26:00.260790+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:00.260790+0000","last_undegraded":"2023-12-14T18:26:00.260790+0000","last_fullsized":"2023-12-14T18:26:00.260790+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:06:10.842072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096117+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:00.096117+0000","last_peered":"2023-12-14T18:26:00.096117+0000","last_clean":"2023-12-14T18:26:00.096117+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:00.096117+0000","last_undegraded":"2023-12-14T18:26:00.096117+0000","last_fullsized":"2023-12-14T18:26:00.096117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:18.832885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.580396+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:21.580396+0000","last_peered":"2023-12-14T18:26:21.580396+0000","last_clean":"2023-12-14T18:26:21.580396+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:21.580396+0000","last_undegraded":"2023-12-14T18:26:21.580396+0000","last_fullsized":"2023-12-14T18:26:21.580396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:27:29.369178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260164+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:00.260164+0000","last_peered":"2023-12-14T18:26:00.260164+0000","last_clean":"2023-12-14T18:26:00.260164+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:00.260164+0000","last_undegraded":"2023-12-14T18:26:00.260164+0000","last_fullsized":"2023-12-14T18:26:00.260164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:20:33.695577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096040+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:00.096040+0000","last_peered":"2023-12-14T18:26:00.096040+0000","last_clean":"2023-12-14T18:26:00.096040+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:00.096040+0000","last_undegraded":"2023-12-14T18:26:00.096040+0000","last_fullsized":"2023-12-14T18:26:00.096040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:13.719713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.262302+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:00.262302+0000","last_peered":"2023-12-14T18:26:00.262302+0000","last_clean":"2023-12-14T18:26:00.262302+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:00.262302+0000","last_undegraded":"2023-12-14T18:26:00.262302+0000","last_fullsized":"2023-12-14T18:26:00.262302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:26.993698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261459+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:00.261459+0000","last_peered":"2023-12-14T18:26:00.261459+0000","last_clean":"2023-12-14T18:26:00.261459+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:00.261459+0000","last_undegraded":"2023-12-14T18:26:00.261459+0000","last_fullsized":"2023-12-14T18:26:00.261459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:16:42.962687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784427+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:25:59.784427+0000","last_peered":"2023-12-14T18:25:59.784427+0000","last_clean":"2023-12-14T18:25:59.784427+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:25:59.784427+0000","last_undegraded":"2023-12-14T18:25:59.784427+0000","last_fullsized":"2023-12-14T18:25:59.784427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:59:28.675903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567704+0000","last_change":"2023-12-14T18:25:54.853066+0000","last_active":"2023-12-14T18:26:20.567704+0000","last_peered":"2023-12-14T18:26:20.567704+0000","last_clean":"2023-12-14T18:26:20.567704+0000","last_became_active":"2023-12-14T18:25:54.852663+0000","last_became_peered":"2023-12-14T18:25:54.852663+0000","last_unstale":"2023-12-14T18:26:20.567704+0000","last_undegraded":"2023-12-14T18:26:20.567704+0000","last_fullsized":"2023-12-14T18:26:20.567704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:41.765930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261259+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:00.261259+0000","last_peered":"2023-12-14T18:26:00.261259+0000","last_clean":"2023-12-14T18:26:00.261259+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:00.261259+0000","last_undegraded":"2023-12-14T18:26:00.261259+0000","last_fullsized":"2023-12-14T18:26:00.261259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:01:59.504071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784181+0000","last_change":"2023-12-14T18:25:59.344884+0000","last_active":"2023-12-14T18:25:59.784181+0000","last_peered":"2023-12-14T18:25:59.784181+0000","last_clean":"2023-12-14T18:25:59.784181+0000","last_became_active":"2023-12-14T18:25:59.344494+0000","last_became_peered":"2023-12-14T18:25:59.344494+0000","last_unstale":"2023-12-14T18:25:59.784181+0000","last_undegraded":"2023-12-14T18:25:59.784181+0000","last_fullsized":"2023-12-14T18:25:59.784181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:03:55.332790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.581473+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:21.581473+0000","last_peered":"2023-12-14T18:26:21.581473+0000","last_clean":"2023-12-14T18:26:21.581473+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:21.581473+0000","last_undegraded":"2023-12-14T18:26:21.581473+0000","last_fullsized":"2023-12-14T18:26:21.581473+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:39.969717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261169+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:00.261169+0000","last_peered":"2023-12-14T18:26:00.261169+0000","last_clean":"2023-12-14T18:26:00.261169+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:00.261169+0000","last_undegraded":"2023-12-14T18:26:00.261169+0000","last_fullsized":"2023-12-14T18:26:00.261169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:21:01.585382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260973+0000","last_change":"2023-12-14T18:25:59.337495+0000","last_active":"2023-12-14T18:26:00.260973+0000","last_peered":"2023-12-14T18:26:00.260973+0000","last_clean":"2023-12-14T18:26:00.260973+0000","last_became_active":"2023-12-14T18:25:59.337072+0000","last_became_peered":"2023-12-14T18:25:59.337072+0000","last_unstale":"2023-12-14T18:26:00.260973+0000","last_undegraded":"2023-12-14T18:26:00.260973+0000","last_fullsized":"2023-12-14T18:26:00.260973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:57:35.575059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689415+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:00.689415+0000","last_peered":"2023-12-14T18:26:00.689415+0000","last_clean":"2023-12-14T18:26:00.689415+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:00.689415+0000","last_undegraded":"2023-12-14T18:26:00.689415+0000","last_fullsized":"2023-12-14T18:26:00.689415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:11:16.910059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260864+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:00.260864+0000","last_peered":"2023-12-14T18:26:00.260864+0000","last_clean":"2023-12-14T18:26:00.260864+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:00.260864+0000","last_undegraded":"2023-12-14T18:26:00.260864+0000","last_fullsized":"2023-12-14T18:26:00.260864+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:36.097770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260576+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:00.260576+0000","last_peered":"2023-12-14T18:26:00.260576+0000","last_clean":"2023-12-14T18:26:00.260576+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:00.260576+0000","last_undegraded":"2023-12-14T18:26:00.260576+0000","last_fullsized":"2023-12-14T18:26:00.260576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:21:14.664141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260191+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:00.260191+0000","last_peered":"2023-12-14T18:26:00.260191+0000","last_clean":"2023-12-14T18:26:00.260191+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:00.260191+0000","last_undegraded":"2023-12-14T18:26:00.260191+0000","last_fullsized":"2023-12-14T18:26:00.260191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:39.435686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783256+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:25:59.783256+0000","last_peered":"2023-12-14T18:25:59.783256+0000","last_clean":"2023-12-14T18:25:59.783256+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:25:59.783256+0000","last_undegraded":"2023-12-14T18:25:59.783256+0000","last_fullsized":"2023-12-14T18:25:59.783256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:48:20.799919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261237+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:00.261237+0000","last_peered":"2023-12-14T18:26:00.261237+0000","last_clean":"2023-12-14T18:26:00.261237+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:00.261237+0000","last_undegraded":"2023-12-14T18:26:00.261237+0000","last_fullsized":"2023-12-14T18:26:00.261237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:39:37.673400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:22.585112+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:26:22.585112+0000","last_peered":"2023-12-14T18:26:22.585112+0000","last_clean":"2023-12-14T18:26:22.585112+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:26:22.585112+0000","last_undegraded":"2023-12-14T18:26:22.585112+0000","last_fullsized":"2023-12-14T18:26:22.585112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:22:35.754355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260416+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:00.260416+0000","last_peered":"2023-12-14T18:26:00.260416+0000","last_clean":"2023-12-14T18:26:00.260416+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:00.260416+0000","last_undegraded":"2023-12-14T18:26:00.260416+0000","last_fullsized":"2023-12-14T18:26:00.260416+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:43:31.511979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783182+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:25:59.783182+0000","last_peered":"2023-12-14T18:25:59.783182+0000","last_clean":"2023-12-14T18:25:59.783182+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:25:59.783182+0000","last_undegraded":"2023-12-14T18:25:59.783182+0000","last_fullsized":"2023-12-14T18:25:59.783182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:51:41.408987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567338+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:20.567338+0000","last_peered":"2023-12-14T18:26:20.567338+0000","last_clean":"2023-12-14T18:26:20.567338+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:20.567338+0000","last_undegraded":"2023-12-14T18:26:20.567338+0000","last_fullsized":"2023-12-14T18:26:20.567338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:32:04.781498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260658+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:00.260658+0000","last_peered":"2023-12-14T18:26:00.260658+0000","last_clean":"2023-12-14T18:26:00.260658+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:00.260658+0000","last_undegraded":"2023-12-14T18:26:00.260658+0000","last_fullsized":"2023-12-14T18:26:00.260658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:08:17.341655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096343+0000","last_change":"2023-12-14T18:25:59.342905+0000","last_active":"2023-12-14T18:26:00.096343+0000","last_peered":"2023-12-14T18:26:00.096343+0000","last_clean":"2023-12-14T18:26:00.096343+0000","last_became_active":"2023-12-14T18:25:59.342417+0000","last_became_peered":"2023-12-14T18:25:59.342417+0000","last_unstale":"2023-12-14T18:26:00.096343+0000","last_undegraded":"2023-12-14T18:26:00.096343+0000","last_fullsized":"2023-12-14T18:26:00.096343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:05.418599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261423+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:00.261423+0000","last_peered":"2023-12-14T18:26:00.261423+0000","last_clean":"2023-12-14T18:26:00.261423+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:00.261423+0000","last_undegraded":"2023-12-14T18:26:00.261423+0000","last_fullsized":"2023-12-14T18:26:00.261423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:17.005591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260368+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:00.260368+0000","last_peered":"2023-12-14T18:26:00.260368+0000","last_clean":"2023-12-14T18:26:00.260368+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:00.260368+0000","last_undegraded":"2023-12-14T18:26:00.260368+0000","last_fullsized":"2023-12-14T18:26:00.260368+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:39:02.954403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783341+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:25:59.783341+0000","last_peered":"2023-12-14T18:25:59.783341+0000","last_clean":"2023-12-14T18:25:59.783341+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:25:59.783341+0000","last_undegraded":"2023-12-14T18:25:59.783341+0000","last_fullsized":"2023-12-14T18:25:59.783341+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:37:49.702818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'23","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:23.903406+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:26:23.903406+0000","last_peered":"2023-12-14T18:26:23.903406+0000","last_clean":"2023-12-14T18:26:23.903406+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:26:23.903406+0000","last_undegraded":"2023-12-14T18:26:23.903406+0000","last_fullsized":"2023-12-14T18:26:23.903406+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:56:23.209261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":24,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261747+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:00.261747+0000","last_peered":"2023-12-14T18:26:00.261747+0000","last_clean":"2023-12-14T18:26:00.261747+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:00.261747+0000","last_undegraded":"2023-12-14T18:26:00.261747+0000","last_fullsized":"2023-12-14T18:26:00.261747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:25:58.855362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261773+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:00.261773+0000","last_peered":"2023-12-14T18:26:00.261773+0000","last_clean":"2023-12-14T18:26:00.261773+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:00.261773+0000","last_undegraded":"2023-12-14T18:26:00.261773+0000","last_fullsized":"2023-12-14T18:26:00.261773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:14:26.658005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.579829+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:21.579829+0000","last_peered":"2023-12-14T18:26:21.579829+0000","last_clean":"2023-12-14T18:26:21.579829+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:21.579829+0000","last_undegraded":"2023-12-14T18:26:21.579829+0000","last_fullsized":"2023-12-14T18:26:21.579829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:28.651606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689579+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:00.689579+0000","last_peered":"2023-12-14T18:26:00.689579+0000","last_clean":"2023-12-14T18:26:00.689579+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:00.689579+0000","last_undegraded":"2023-12-14T18:26:00.689579+0000","last_fullsized":"2023-12-14T18:26:00.689579+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:54.349300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260580+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:00.260580+0000","last_peered":"2023-12-14T18:26:00.260580+0000","last_clean":"2023-12-14T18:26:00.260580+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:00.260580+0000","last_undegraded":"2023-12-14T18:26:00.260580+0000","last_fullsized":"2023-12-14T18:26:00.260580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:21:10.802288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567563+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:20.567563+0000","last_peered":"2023-12-14T18:26:20.567563+0000","last_clean":"2023-12-14T18:26:20.567563+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:20.567563+0000","last_undegraded":"2023-12-14T18:26:20.567563+0000","last_fullsized":"2023-12-14T18:26:20.567563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:24:25.354538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261195+0000","last_change":"2023-12-14T18:25:55.316217+0000","last_active":"2023-12-14T18:26:00.261195+0000","last_peered":"2023-12-14T18:26:00.261195+0000","last_clean":"2023-12-14T18:26:00.261195+0000","last_became_active":"2023-12-14T18:25:55.315842+0000","last_became_peered":"2023-12-14T18:25:55.315842+0000","last_unstale":"2023-12-14T18:26:00.261195+0000","last_undegraded":"2023-12-14T18:26:00.261195+0000","last_fullsized":"2023-12-14T18:26:00.261195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:25:51.857936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783806+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:25:59.783806+0000","last_peered":"2023-12-14T18:25:59.783806+0000","last_clean":"2023-12-14T18:25:59.783806+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:25:59.783806+0000","last_undegraded":"2023-12-14T18:25:59.783806+0000","last_fullsized":"2023-12-14T18:25:59.783806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:50:45.642801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:22.588667+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:26:22.588667+0000","last_peered":"2023-12-14T18:26:22.588667+0000","last_clean":"2023-12-14T18:26:22.588667+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:26:22.588667+0000","last_undegraded":"2023-12-14T18:26:22.588667+0000","last_fullsized":"2023-12-14T18:26:22.588667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:57:18.362489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260931+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:00.260931+0000","last_peered":"2023-12-14T18:26:00.260931+0000","last_clean":"2023-12-14T18:26:00.260931+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:00.260931+0000","last_undegraded":"2023-12-14T18:26:00.260931+0000","last_fullsized":"2023-12-14T18:26:00.260931+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:28:53.590151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.690000+0000","last_change":"2023-12-14T18:25:59.348833+0000","last_active":"2023-12-14T18:26:00.690000+0000","last_peered":"2023-12-14T18:26:00.690000+0000","last_clean":"2023-12-14T18:26:00.690000+0000","last_became_active":"2023-12-14T18:25:59.348367+0000","last_became_peered":"2023-12-14T18:25:59.348367+0000","last_unstale":"2023-12-14T18:26:00.690000+0000","last_undegraded":"2023-12-14T18:26:00.690000+0000","last_fullsized":"2023-12-14T18:26:00.690000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:33:20.898030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.582089+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:21.582089+0000","last_peered":"2023-12-14T18:26:21.582089+0000","last_clean":"2023-12-14T18:26:21.582089+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:21.582089+0000","last_undegraded":"2023-12-14T18:26:21.582089+0000","last_fullsized":"2023-12-14T18:26:21.582089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:53:54.171519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260781+0000","last_change":"2023-12-14T18:25:55.305447+0000","last_active":"2023-12-14T18:26:00.260781+0000","last_peered":"2023-12-14T18:26:00.260781+0000","last_clean":"2023-12-14T18:26:00.260781+0000","last_became_active":"2023-12-14T18:25:55.305054+0000","last_became_peered":"2023-12-14T18:25:55.305054+0000","last_unstale":"2023-12-14T18:26:00.260781+0000","last_undegraded":"2023-12-14T18:26:00.260781+0000","last_fullsized":"2023-12-14T18:26:00.260781+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:52:51.044461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260692+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:00.260692+0000","last_peered":"2023-12-14T18:26:00.260692+0000","last_clean":"2023-12-14T18:26:00.260692+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:00.260692+0000","last_undegraded":"2023-12-14T18:26:00.260692+0000","last_fullsized":"2023-12-14T18:26:00.260692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:24.105560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.262002+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:00.262002+0000","last_peered":"2023-12-14T18:26:00.262002+0000","last_clean":"2023-12-14T18:26:00.262002+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:00.262002+0000","last_undegraded":"2023-12-14T18:26:00.262002+0000","last_fullsized":"2023-12-14T18:26:00.262002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:57:29.873612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260746+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:00.260746+0000","last_peered":"2023-12-14T18:26:00.260746+0000","last_clean":"2023-12-14T18:26:00.260746+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:00.260746+0000","last_undegraded":"2023-12-14T18:26:00.260746+0000","last_fullsized":"2023-12-14T18:26:00.260746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:23:48.805350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261174+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:00.261174+0000","last_peered":"2023-12-14T18:26:00.261174+0000","last_clean":"2023-12-14T18:26:00.261174+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:00.261174+0000","last_undegraded":"2023-12-14T18:26:00.261174+0000","last_fullsized":"2023-12-14T18:26:00.261174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:30:54.421948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783421+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:25:59.783421+0000","last_peered":"2023-12-14T18:25:59.783421+0000","last_clean":"2023-12-14T18:25:59.783421+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:25:59.783421+0000","last_undegraded":"2023-12-14T18:25:59.783421+0000","last_fullsized":"2023-12-14T18:25:59.783421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:18:04.320749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784394+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:25:59.784394+0000","last_peered":"2023-12-14T18:25:59.784394+0000","last_clean":"2023-12-14T18:25:59.784394+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:25:59.784394+0000","last_undegraded":"2023-12-14T18:25:59.784394+0000","last_fullsized":"2023-12-14T18:25:59.784394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:03:42.297304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689797+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:00.689797+0000","last_peered":"2023-12-14T18:26:00.689797+0000","last_clean":"2023-12-14T18:26:00.689797+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:00.689797+0000","last_undegraded":"2023-12-14T18:26:00.689797+0000","last_fullsized":"2023-12-14T18:26:00.689797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:19:10.100748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260990+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:00.260990+0000","last_peered":"2023-12-14T18:26:00.260990+0000","last_clean":"2023-12-14T18:26:00.260990+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:00.260990+0000","last_undegraded":"2023-12-14T18:26:00.260990+0000","last_fullsized":"2023-12-14T18:26:00.260990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:23:18.627153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784335+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:25:59.784335+0000","last_peered":"2023-12-14T18:25:59.784335+0000","last_clean":"2023-12-14T18:25:59.784335+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:25:59.784335+0000","last_undegraded":"2023-12-14T18:25:59.784335+0000","last_fullsized":"2023-12-14T18:25:59.784335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.499504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689863+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:00.689863+0000","last_peered":"2023-12-14T18:26:00.689863+0000","last_clean":"2023-12-14T18:26:00.689863+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:00.689863+0000","last_undegraded":"2023-12-14T18:26:00.689863+0000","last_fullsized":"2023-12-14T18:26:00.689863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:26.633680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260298+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:00.260298+0000","last_peered":"2023-12-14T18:26:00.260298+0000","last_clean":"2023-12-14T18:26:00.260298+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:00.260298+0000","last_undegraded":"2023-12-14T18:26:00.260298+0000","last_fullsized":"2023-12-14T18:26:00.260298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:23:55.056652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689316+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:26:00.689316+0000","last_peered":"2023-12-14T18:26:00.689316+0000","last_clean":"2023-12-14T18:26:00.689316+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:26:00.689316+0000","last_undegraded":"2023-12-14T18:26:00.689316+0000","last_fullsized":"2023-12-14T18:26:00.689316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:10:53.820789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261316+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:00.261316+0000","last_peered":"2023-12-14T18:26:00.261316+0000","last_clean":"2023-12-14T18:26:00.261316+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:00.261316+0000","last_undegraded":"2023-12-14T18:26:00.261316+0000","last_fullsized":"2023-12-14T18:26:00.261316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:35:25.807560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261645+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:00.261645+0000","last_peered":"2023-12-14T18:26:00.261645+0000","last_clean":"2023-12-14T18:26:00.261645+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:00.261645+0000","last_undegraded":"2023-12-14T18:26:00.261645+0000","last_fullsized":"2023-12-14T18:26:00.261645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:35:11.047723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:22.583401+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:22.583401+0000","last_peered":"2023-12-14T18:26:22.583401+0000","last_clean":"2023-12-14T18:26:22.583401+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:22.583401+0000","last_undegraded":"2023-12-14T18:26:22.583401+0000","last_fullsized":"2023-12-14T18:26:22.583401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:42:38.074930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":340,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260822+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:00.260822+0000","last_peered":"2023-12-14T18:26:00.260822+0000","last_clean":"2023-12-14T18:26:00.260822+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:00.260822+0000","last_undegraded":"2023-12-14T18:26:00.260822+0000","last_fullsized":"2023-12-14T18:26:00.260822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:27:04.810823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260773+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:00.260773+0000","last_peered":"2023-12-14T18:26:00.260773+0000","last_clean":"2023-12-14T18:26:00.260773+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:00.260773+0000","last_undegraded":"2023-12-14T18:26:00.260773+0000","last_fullsized":"2023-12-14T18:26:00.260773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:17:01.451483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689259+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:00.689259+0000","last_peered":"2023-12-14T18:26:00.689259+0000","last_clean":"2023-12-14T18:26:00.689259+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:00.689259+0000","last_undegraded":"2023-12-14T18:26:00.689259+0000","last_fullsized":"2023-12-14T18:26:00.689259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:32:23.251389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.579722+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:21.579722+0000","last_peered":"2023-12-14T18:26:21.579722+0000","last_clean":"2023-12-14T18:26:21.579722+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:21.579722+0000","last_undegraded":"2023-12-14T18:26:21.579722+0000","last_fullsized":"2023-12-14T18:26:21.579722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:28:35.723032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261019+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:00.261019+0000","last_peered":"2023-12-14T18:26:00.261019+0000","last_clean":"2023-12-14T18:26:00.261019+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:00.261019+0000","last_undegraded":"2023-12-14T18:26:00.261019+0000","last_fullsized":"2023-12-14T18:26:00.261019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:40:04.222426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261943+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:00.261943+0000","last_peered":"2023-12-14T18:26:00.261943+0000","last_clean":"2023-12-14T18:26:00.261943+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:00.261943+0000","last_undegraded":"2023-12-14T18:26:00.261943+0000","last_fullsized":"2023-12-14T18:26:00.261943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:55:28.262805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689898+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:00.689898+0000","last_peered":"2023-12-14T18:26:00.689898+0000","last_clean":"2023-12-14T18:26:00.689898+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:00.689898+0000","last_undegraded":"2023-12-14T18:26:00.689898+0000","last_fullsized":"2023-12-14T18:26:00.689898+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:35:00.581814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260787+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:00.260787+0000","last_peered":"2023-12-14T18:26:00.260787+0000","last_clean":"2023-12-14T18:26:00.260787+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:00.260787+0000","last_undegraded":"2023-12-14T18:26:00.260787+0000","last_fullsized":"2023-12-14T18:26:00.260787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:28:31.908021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"83'1","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783879+0000","last_change":"2023-12-14T18:25:54.852944+0000","last_active":"2023-12-14T18:25:59.783879+0000","last_peered":"2023-12-14T18:25:59.783879+0000","last_clean":"2023-12-14T18:25:59.783879+0000","last_became_active":"2023-12-14T18:25:54.852565+0000","last_became_peered":"2023-12-14T18:25:54.852565+0000","last_unstale":"2023-12-14T18:25:59.783879+0000","last_undegraded":"2023-12-14T18:25:59.783879+0000","last_fullsized":"2023-12-14T18:25:59.783879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:49.128592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261642+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:00.261642+0000","last_peered":"2023-12-14T18:26:00.261642+0000","last_clean":"2023-12-14T18:26:00.261642+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:00.261642+0000","last_undegraded":"2023-12-14T18:26:00.261642+0000","last_fullsized":"2023-12-14T18:26:00.261642+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:39:39.802371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783842+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:25:59.783842+0000","last_peered":"2023-12-14T18:25:59.783842+0000","last_clean":"2023-12-14T18:25:59.783842+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:25:59.783842+0000","last_undegraded":"2023-12-14T18:25:59.783842+0000","last_fullsized":"2023-12-14T18:25:59.783842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:42.465645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.565619+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:20.565619+0000","last_peered":"2023-12-14T18:26:20.565619+0000","last_clean":"2023-12-14T18:26:20.565619+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:20.565619+0000","last_undegraded":"2023-12-14T18:26:20.565619+0000","last_fullsized":"2023-12-14T18:26:20.565619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:45:18.559609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260676+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:00.260676+0000","last_peered":"2023-12-14T18:26:00.260676+0000","last_clean":"2023-12-14T18:26:00.260676+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:00.260676+0000","last_undegraded":"2023-12-14T18:26:00.260676+0000","last_fullsized":"2023-12-14T18:26:00.260676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:12:07.158191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096316+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:00.096316+0000","last_peered":"2023-12-14T18:26:00.096316+0000","last_clean":"2023-12-14T18:26:00.096316+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:00.096316+0000","last_undegraded":"2023-12-14T18:26:00.096316+0000","last_fullsized":"2023-12-14T18:26:00.096316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:43.864201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.566288+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:20.566288+0000","last_peered":"2023-12-14T18:26:20.566288+0000","last_clean":"2023-12-14T18:26:20.566288+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:20.566288+0000","last_undegraded":"2023-12-14T18:26:20.566288+0000","last_fullsized":"2023-12-14T18:26:20.566288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:49:16.660803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260683+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:00.260683+0000","last_peered":"2023-12-14T18:26:00.260683+0000","last_clean":"2023-12-14T18:26:00.260683+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:00.260683+0000","last_undegraded":"2023-12-14T18:26:00.260683+0000","last_fullsized":"2023-12-14T18:26:00.260683+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:27:42.833359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783900+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:25:59.783900+0000","last_peered":"2023-12-14T18:25:59.783900+0000","last_clean":"2023-12-14T18:25:59.783900+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:25:59.783900+0000","last_undegraded":"2023-12-14T18:25:59.783900+0000","last_fullsized":"2023-12-14T18:25:59.783900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:59.055992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.567068+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:20.567068+0000","last_peered":"2023-12-14T18:26:20.567068+0000","last_clean":"2023-12-14T18:26:20.567068+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:20.567068+0000","last_undegraded":"2023-12-14T18:26:20.567068+0000","last_fullsized":"2023-12-14T18:26:20.567068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:54:18.694073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260875+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:00.260875+0000","last_peered":"2023-12-14T18:26:00.260875+0000","last_clean":"2023-12-14T18:26:00.260875+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:00.260875+0000","last_undegraded":"2023-12-14T18:26:00.260875+0000","last_fullsized":"2023-12-14T18:26:00.260875+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:53:27.261881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096368+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:00.096368+0000","last_peered":"2023-12-14T18:26:00.096368+0000","last_clean":"2023-12-14T18:26:00.096368+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:00.096368+0000","last_undegraded":"2023-12-14T18:26:00.096368+0000","last_fullsized":"2023-12-14T18:26:00.096368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:22.345101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783853+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:25:59.783853+0000","last_peered":"2023-12-14T18:25:59.783853+0000","last_clean":"2023-12-14T18:25:59.783853+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:25:59.783853+0000","last_undegraded":"2023-12-14T18:25:59.783853+0000","last_fullsized":"2023-12-14T18:25:59.783853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:47.086736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261151+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:00.261151+0000","last_peered":"2023-12-14T18:26:00.261151+0000","last_clean":"2023-12-14T18:26:00.261151+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:00.261151+0000","last_undegraded":"2023-12-14T18:26:00.261151+0000","last_fullsized":"2023-12-14T18:26:00.261151+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:54:54.197194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.096435+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:00.096435+0000","last_peered":"2023-12-14T18:26:00.096435+0000","last_clean":"2023-12-14T18:26:00.096435+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:00.096435+0000","last_undegraded":"2023-12-14T18:26:00.096435+0000","last_fullsized":"2023-12-14T18:26:00.096435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:29:39.716647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:20.565419+0000","last_change":"2023-12-14T18:25:54.850903+0000","last_active":"2023-12-14T18:26:20.565419+0000","last_peered":"2023-12-14T18:26:20.565419+0000","last_clean":"2023-12-14T18:26:20.565419+0000","last_became_active":"2023-12-14T18:25:54.850502+0000","last_became_peered":"2023-12-14T18:25:54.850502+0000","last_unstale":"2023-12-14T18:26:20.565419+0000","last_undegraded":"2023-12-14T18:26:20.565419+0000","last_fullsized":"2023-12-14T18:26:20.565419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:58:08.788921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261131+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:00.261131+0000","last_peered":"2023-12-14T18:26:00.261131+0000","last_clean":"2023-12-14T18:26:00.261131+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:00.261131+0000","last_undegraded":"2023-12-14T18:26:00.261131+0000","last_fullsized":"2023-12-14T18:26:00.261131+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:42:23.981776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260129+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:00.260129+0000","last_peered":"2023-12-14T18:26:00.260129+0000","last_clean":"2023-12-14T18:26:00.260129+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:00.260129+0000","last_undegraded":"2023-12-14T18:26:00.260129+0000","last_fullsized":"2023-12-14T18:26:00.260129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:16:10.873684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:21.582563+0000","last_change":"2023-12-14T18:25:54.860133+0000","last_active":"2023-12-14T18:26:21.582563+0000","last_peered":"2023-12-14T18:26:21.582563+0000","last_clean":"2023-12-14T18:26:21.582563+0000","last_became_active":"2023-12-14T18:25:54.859782+0000","last_became_peered":"2023-12-14T18:25:54.859782+0000","last_unstale":"2023-12-14T18:26:21.582563+0000","last_undegraded":"2023-12-14T18:26:21.582563+0000","last_fullsized":"2023-12-14T18:26:21.582563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:03:24.896495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261523+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:00.261523+0000","last_peered":"2023-12-14T18:26:00.261523+0000","last_clean":"2023-12-14T18:26:00.261523+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:00.261523+0000","last_undegraded":"2023-12-14T18:26:00.261523+0000","last_fullsized":"2023-12-14T18:26:00.261523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:57:46.923639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.259985+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:00.259985+0000","last_peered":"2023-12-14T18:26:00.259985+0000","last_clean":"2023-12-14T18:26:00.259985+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:00.259985+0000","last_undegraded":"2023-12-14T18:26:00.259985+0000","last_fullsized":"2023-12-14T18:26:00.259985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:17.333368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261488+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:00.261488+0000","last_peered":"2023-12-14T18:26:00.261488+0000","last_clean":"2023-12-14T18:26:00.261488+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:00.261488+0000","last_undegraded":"2023-12-14T18:26:00.261488+0000","last_fullsized":"2023-12-14T18:26:00.261488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:32.435550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261370+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:00.261370+0000","last_peered":"2023-12-14T18:26:00.261370+0000","last_clean":"2023-12-14T18:26:00.261370+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:00.261370+0000","last_undegraded":"2023-12-14T18:26:00.261370+0000","last_fullsized":"2023-12-14T18:26:00.261370+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:55:17.989682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260121+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:00.260121+0000","last_peered":"2023-12-14T18:26:00.260121+0000","last_clean":"2023-12-14T18:26:00.260121+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:00.260121+0000","last_undegraded":"2023-12-14T18:26:00.260121+0000","last_fullsized":"2023-12-14T18:26:00.260121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:06.010260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689833+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:00.689833+0000","last_peered":"2023-12-14T18:26:00.689833+0000","last_clean":"2023-12-14T18:26:00.689833+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:00.689833+0000","last_undegraded":"2023-12-14T18:26:00.689833+0000","last_fullsized":"2023-12-14T18:26:00.689833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:23:23.345143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261510+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:00.261510+0000","last_peered":"2023-12-14T18:26:00.261510+0000","last_clean":"2023-12-14T18:26:00.261510+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:00.261510+0000","last_undegraded":"2023-12-14T18:26:00.261510+0000","last_fullsized":"2023-12-14T18:26:00.261510+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:00:54.564106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261020+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:00.261020+0000","last_peered":"2023-12-14T18:26:00.261020+0000","last_clean":"2023-12-14T18:26:00.261020+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:00.261020+0000","last_undegraded":"2023-12-14T18:26:00.261020+0000","last_fullsized":"2023-12-14T18:26:00.261020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:10:55.378712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784089+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:25:59.784089+0000","last_peered":"2023-12-14T18:25:59.784089+0000","last_clean":"2023-12-14T18:25:59.784089+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:25:59.784089+0000","last_undegraded":"2023-12-14T18:25:59.784089+0000","last_fullsized":"2023-12-14T18:25:59.784089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:37.784010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.783831+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:25:59.783831+0000","last_peered":"2023-12-14T18:25:59.783831+0000","last_clean":"2023-12-14T18:25:59.783831+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:25:59.783831+0000","last_undegraded":"2023-12-14T18:25:59.783831+0000","last_fullsized":"2023-12-14T18:25:59.783831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:34:55.579739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260069+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:00.260069+0000","last_peered":"2023-12-14T18:26:00.260069+0000","last_clean":"2023-12-14T18:26:00.260069+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:00.260069+0000","last_undegraded":"2023-12-14T18:26:00.260069+0000","last_fullsized":"2023-12-14T18:26:00.260069+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:27:04.926460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689932+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:00.689932+0000","last_peered":"2023-12-14T18:26:00.689932+0000","last_clean":"2023-12-14T18:26:00.689932+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:00.689932+0000","last_undegraded":"2023-12-14T18:26:00.689932+0000","last_fullsized":"2023-12-14T18:26:00.689932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:59:17.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261452+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:00.261452+0000","last_peered":"2023-12-14T18:26:00.261452+0000","last_clean":"2023-12-14T18:26:00.261452+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:00.261452+0000","last_undegraded":"2023-12-14T18:26:00.261452+0000","last_fullsized":"2023-12-14T18:26:00.261452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:33.344146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:59.784499+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:25:59.784499+0000","last_peered":"2023-12-14T18:25:59.784499+0000","last_clean":"2023-12-14T18:25:59.784499+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:25:59.784499+0000","last_undegraded":"2023-12-14T18:25:59.784499+0000","last_fullsized":"2023-12-14T18:25:59.784499+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:38.320340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.261896+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:00.261896+0000","last_peered":"2023-12-14T18:26:00.261896+0000","last_clean":"2023-12-14T18:26:00.261896+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:00.261896+0000","last_undegraded":"2023-12-14T18:26:00.261896+0000","last_fullsized":"2023-12-14T18:26:00.261896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:49:32.561679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260230+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:00.260230+0000","last_peered":"2023-12-14T18:26:00.260230+0000","last_clean":"2023-12-14T18:26:00.260230+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:00.260230+0000","last_undegraded":"2023-12-14T18:26:00.260230+0000","last_fullsized":"2023-12-14T18:26:00.260230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:20.517919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260923+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:00.260923+0000","last_peered":"2023-12-14T18:26:00.260923+0000","last_clean":"2023-12-14T18:26:00.260923+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:00.260923+0000","last_undegraded":"2023-12-14T18:26:00.260923+0000","last_fullsized":"2023-12-14T18:26:00.260923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:47:43.144080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260248+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:00.260248+0000","last_peered":"2023-12-14T18:26:00.260248+0000","last_clean":"2023-12-14T18:26:00.260248+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:00.260248+0000","last_undegraded":"2023-12-14T18:26:00.260248+0000","last_fullsized":"2023-12-14T18:26:00.260248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:42:31.252913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.689577+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:00.689577+0000","last_peered":"2023-12-14T18:26:00.689577+0000","last_clean":"2023-12-14T18:26:00.689577+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:00.689577+0000","last_undegraded":"2023-12-14T18:26:00.689577+0000","last_fullsized":"2023-12-14T18:26:00.689577+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:50:54.626392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:26:00.260687+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:00.260687+0000","last_peered":"2023-12-14T18:26:00.260687+0000","last_clean":"2023-12-14T18:26:00.260687+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:00.260687+0000","last_undegraded":"2023-12-14T18:26:00.260687+0000","last_fullsized":"2023-12-14T18:26:00.260687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:49:08.491183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41938,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":19,"num_read_kb":34,"num_write":122,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":307200,"data_stored":91539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6028,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739124,"statfs":{"total":95995035648,"available":95988862976,"internally_reserved":0,"allocated":921600,"data_stored":460250,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":10,"up_from":68,"seq":292057776147,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6392,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738760,"statfs":{"total":95995035648,"available":95988490240,"internally_reserved":0,"allocated":1032192,"data_stored":1073041,"data_compressed":47404,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.503}]},{"osd":5,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":1.145}]},{"osd":9,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.4960000000000004,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.60499999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":9,"up_from":63,"seq":270582939669,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6224,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738928,"statfs":{"total":95995035648,"available":95988662272,"internally_reserved":0,"allocated":925696,"data_stored":460832,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.312}]},{"osd":1,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.66200000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.70699999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.45600000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.72999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":8,"up_from":55,"seq":236223201306,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6368,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738784,"statfs":{"total":95995035648,"available":95988514816,"internally_reserved":0,"allocated":942080,"data_stored":460988,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.67200000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.628},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.89900000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.84899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.878}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6468,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738684,"statfs":{"total":95995035648,"available":95988412416,"internally_reserved":0,"allocated":978944,"data_stored":1051710,"data_compressed":47404,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.41099999999999998,"15min":0.39700000000000002},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.40799999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.84599999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.752,"5min":1.8,"15min":1.8},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.36199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.63}]},{"osd":5,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.88300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.59599999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84899999999999998,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.55000000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Thu Dec 14 18:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.96499999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.872},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":6,"up_from":43,"seq":184683593761,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6348,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738804,"statfs":{"total":95995035648,"available":95988535296,"internally_reserved":0,"allocated":921600,"data_stored":460250,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.32100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.42899999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59899999999999998,"15min":0.58699999999999997},"min":{"1min":0.52800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.749}]},{"osd":3,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.627}]},{"osd":4,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70999999999999996},"min":{"1min":0.622,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.76600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85599999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.91000000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":966656,"data_stored":498491,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.502,"15min":0.498},"min":{"1min":0.39700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.50600000000000001,"15min":0.49199999999999999},"min":{"1min":0.39600000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.495}]},{"osd":2,"last update":"Thu Dec 14 18:25:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.496,"15min":0.48499999999999999},"min":{"1min":0.39600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.48999999999999999,"15min":0.46600000000000003},"min":{"1min":0.41999999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.61099999999999999,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.3150000000000004,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.61299999999999999,"15min":0.54700000000000004},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.312,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70899999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.52200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.44900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.78600000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.48199999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.752}]},{"osd":8,"last update":"Thu Dec 14 18:25:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.85299999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":3.3570000000000002,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":5.1159999999999997,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.81799999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.92100000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7272,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737880,"statfs":{"total":95995035648,"available":95987589120,"internally_reserved":0,"allocated":950272,"data_stored":461728,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.52900000000000003,"15min":0.5},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.41999999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54700000000000004,"15min":0.54000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.627}]},{"osd":3,"last update":"Thu Dec 14 18:25:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.308,"5min":0.251,"15min":0.251},"max":{"1min":0.70399999999999996,"5min":0.751,"15min":0.751},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.32700000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:26:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":1.091,"15min":1.091},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.60099999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93100000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.44},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71299999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.69999999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.747},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.79500000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7280,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737872,"statfs":{"total":95995035648,"available":95987580928,"internally_reserved":0,"allocated":958464,"data_stored":465345,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.36599999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.39300000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90400000000000003,"5min":1.484,"15min":1.484},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.34100000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71699999999999997,"5min":0.75,"15min":0.75},"last":0.748}]},{"osd":5,"last update":"Thu Dec 14 18:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.373,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.376},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.66800000000000004,"5min":1.514,"15min":1.514},"last":0.505}]},{"osd":6,"last update":"Thu Dec 14 18:25:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.45600000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92500000000000004,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63200000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70199999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215147,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":460081,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38200000000000001},"min":{"1min":0.36499999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.40899999999999997},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.503},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.64600000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.747},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.62}]},{"osd":5,"last update":"Thu Dec 14 18:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.629,"15min":0.621},"min":{"1min":0.52800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95199999999999996,"5min":1.093,"15min":1.093},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.623,"15min":0.62},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.40899999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52200000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.441},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.47999999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.68600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986217,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":937984,"data_stored":464942,"data_compressed":11700,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.29099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.33400000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71799999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.504},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":1.121,"15min":1.121},"last":0.379}]},{"osd":5,"last update":"Thu Dec 14 18:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.42599999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.435},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.48299999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.879,"15min":0.879},"last":0.46100000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.73999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.52200000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59999999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822693,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6612,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738540,"statfs":{"total":95995035648,"available":95988264960,"internally_reserved":0,"allocated":995328,"data_stored":1051715,"data_compressed":47404,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.75700000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51800000000000002,"15min":0.52400000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.50900000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.746,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.374,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.192,"5min":0.192,"15min":0.192},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.40999999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66400000000000003,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.42799999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.371,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.622},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.41099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.80100000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67600000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58099999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71399999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:26:25.970 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-14T18:26:25.971 INFO:tasks.cephadm:Setting up client nodes... 2023-12-14T18:26:25.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-14T18:26:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:25 smithi176 ceph-mon[43506]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:25 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/853200192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:25 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/533831389' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:26 smithi031 ceph-mon[42085]: pgmap v244: 129 pgs: 129 active+clean; 618 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2023-12-14T18:26:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:26 smithi031 ceph-mon[42085]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:26:26.838 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:26 smithi031 ceph-mon[42085]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:26:26.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:26 smithi204 ceph-mon[43866]: pgmap v244: 129 pgs: 129 active+clean; 618 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2023-12-14T18:26:26.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:26 smithi204 ceph-mon[43866]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:26:26.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:26 smithi204 ceph-mon[43866]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:26:26.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:26 smithi176 ceph-mon[43506]: pgmap v244: 129 pgs: 129 active+clean; 618 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5.4 KiB/s wr, 7 op/s 2023-12-14T18:26:26.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:26 smithi176 ceph-mon[43506]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:26:26.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:26 smithi176 ceph-mon[43506]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 -- 172.21.15.31:0/1785176510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 msgr2=0x7fb060100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 --2- 172.21.15.31:0/1785176510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb060100830 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb048009990 tx=0x7fb048009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 -- 172.21.15.31:0/1785176510 shutdown_connections 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 --2- 172.21.15.31:0/1785176510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb060100d70 0x7fb060107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 --2- 172.21.15.31:0/1785176510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb0600ff1e0 0x7fb0600ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 --2- 172.21.15.31:0/1785176510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb060100830 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 -- 172.21.15.31:0/1785176510 >> 172.21.15.31:0/1785176510 conn(0x7fb0600fa7f0 msgr2=0x7fb0600fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:27.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 -- 172.21.15.31:0/1785176510 shutdown_connections 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.356+0000 7fb065ea8700 1 -- 172.21.15.31:0/1785176510 wait complete. 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 Processor -- start 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 -- start start 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0600ff1e0 0x7fb060069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb06019ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb060100d70 0x7fb06019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06019db80 con 0x7fb0601003e0 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06019dcf0 con 0x7fb060100d70 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.357+0000 7fb065ea8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06019deb0 con 0x7fb0600ff1e0 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb06019ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb06019ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35368/0 (socket says 172.21.15.31:35368) 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 -- 172.21.15.31:0/3857797660 learned_addr learned my addr 172.21.15.31:0/3857797660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05f7fe700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0600ff1e0 0x7fb060069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05ffff700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb060100d70 0x7fb06019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 -- 172.21.15.31:0/3857797660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0600ff1e0 msgr2=0x7fb060069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:27.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0600ff1e0 0x7fb060069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 -- 172.21.15.31:0/3857797660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb060100d70 msgr2=0x7fb06019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb060100d70 0x7fb06019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 -- 172.21.15.31:0/3857797660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480095f0 con 0x7fb0601003e0 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05effd700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb06019ada0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb048000c00 tx=0x7fb048009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04801d070 con 0x7fb0601003e0 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb048022470 con 0x7fb0601003e0 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06010ab40 con 0x7fb0601003e0 2023-12-14T18:26:27.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048017650 con 0x7fb0601003e0 2023-12-14T18:26:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.358+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06010a9b0 con 0x7fb0601003e0 2023-12-14T18:26:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.360+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb060106820 con 0x7fb0601003e0 2023-12-14T18:26:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.360+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb048022850 con 0x7fb0601003e0 2023-12-14T18:26:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.361+0000 7fb05cff9700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb04c0a11e0 0x7fb04c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.361+0000 7fb05f7fe700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb04c0a11e0 0x7fb04c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:27.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.361+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0480cdfe0 con 0x7fb0601003e0 2023-12-14T18:26:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.362+0000 7fb05f7fe700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb04c0a11e0 0x7fb04c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb060100240 tx=0x7fb05800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:27.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.365+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb048097580 con 0x7fb0601003e0 2023-12-14T18:26:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.665+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb06002cfe0 con 0x7fb0601003e0 2023-12-14T18:26:27.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.669+0000 7fb05cff9700 1 -- 172.21.15.31:0/3857797660 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb048027050 con 0x7fb0601003e0 2023-12-14T18:26:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-12-14T18:26:27.671 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDTSHtlwKyyJxAAF2uXBLwxuYzeo8jkJ2Nunw== 2023-12-14T18:26:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb04c0a11e0 msgr2=0x7fb04c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb04c0a11e0 0x7fb04c0a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb060100240 tx=0x7fb05800b040 comp rx=0 tx=0).stop 2023-12-14T18:26:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 msgr2=0x7fb06019ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb06019ada0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb048000c00 tx=0x7fb048009580 comp rx=0 tx=0).stop 2023-12-14T18:26:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 shutdown_connections 2023-12-14T18:26:27.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb060100d70 0x7fb06019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb04c0a11e0 0x7fb04c0a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0600ff1e0 0x7fb060069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 --2- 172.21.15.31:0/3857797660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0601003e0 0x7fb06019ada0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 >> 172.21.15.31:0/3857797660 conn(0x7fb0600fa7f0 msgr2=0x7fb060103960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 shutdown_connections 2023-12-14T18:26:27.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:27.672+0000 7fb065ea8700 1 -- 172.21.15.31:0/3857797660 wait complete. 2023-12-14T18:26:28.058 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:26:28.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-14T18:26:28.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-14T18:26:28.110 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-14T18:26:28.110 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-14T18:26:28.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph mgr dump --format=json 2023-12-14T18:26:28.507 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:28.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:28 smithi031 ceph-mon[42085]: pgmap v245: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2023-12-14T18:26:28.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:28 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3857797660' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-14T18:26:28.953 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:28 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3857797660' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-14T18:26:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:28 smithi176 ceph-mon[43506]: pgmap v245: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2023-12-14T18:26:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:28 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3857797660' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-14T18:26:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:28 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3857797660' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-14T18:26:29.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:28 smithi204 ceph-mon[43866]: pgmap v245: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2023-12-14T18:26:29.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:28 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3857797660' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-14T18:26:29.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:28 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3857797660' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-14T18:26:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 -- 172.21.15.31:0/3572444121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ffde0 msgr2=0x7f800c103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 --2- 172.21.15.31:0/3572444121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ffde0 0x7f800c103db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f80000099c0 tx=0x7f8000009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 -- 172.21.15.31:0/3572444121 shutdown_connections 2023-12-14T18:26:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 --2- 172.21.15.31:0/3572444121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c1044e0 0x7f800c10c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 --2- 172.21.15.31:0/3572444121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c0ff440 0x7f800c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 --2- 172.21.15.31:0/3572444121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ffde0 0x7f800c103db0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 -- 172.21.15.31:0/3572444121 >> 172.21.15.31:0/3572444121 conn(0x7f800c0facc0 msgr2=0x7f800c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 -- 172.21.15.31:0/3572444121 shutdown_connections 2023-12-14T18:26:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.530+0000 7f8011c97700 1 -- 172.21.15.31:0/3572444121 wait complete. 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.531+0000 7f8011c97700 1 Processor -- start 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.531+0000 7f8011c97700 1 -- start start 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8011c97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 0x7f800c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8011c97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c0ffde0 0x7f800c1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8011c97700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c1044e0 0x7f800c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8011c97700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19a240 con 0x7f800c0ff440 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8011c97700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19a3b0 con 0x7f800c1044e0 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8011c97700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19a570 con 0x7f800c0ffde0 2023-12-14T18:26:29.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 0x7f800c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 0x7f800c195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35384/0 (socket says 172.21.15.31:35384) 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 -- 172.21.15.31:0/3732063384 learned_addr learned my addr 172.21.15.31:0/3732063384 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800affd700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c0ffde0 0x7f800c1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800bfff700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c1044e0 0x7f800c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 -- 172.21.15.31:0/3732063384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c0ffde0 msgr2=0x7f800c1957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c0ffde0 0x7f800c1957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 -- 172.21.15.31:0/3732063384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c1044e0 msgr2=0x7f800c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c1044e0 0x7f800c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 -- 172.21.15.31:0/3732063384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000095f0 con 0x7f800c0ff440 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800affd700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c0ffde0 0x7f800c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800bfff700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c1044e0 0x7f800c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f800b7fe700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 0x7f800c195280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00d950 tx=0x7f7ffc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:29.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.532+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc009b40 con 0x7f800c0ff440 2023-12-14T18:26:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.533+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ffc009ca0 con 0x7f800c0ff440 2023-12-14T18:26:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.533+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c10f210 con 0x7f800c0ff440 2023-12-14T18:26:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.533+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc00f620 con 0x7f800c0ff440 2023-12-14T18:26:29.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.533+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c10f080 con 0x7f800c0ff440 2023-12-14T18:26:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.534+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c10f3a0 con 0x7f800c0ff440 2023-12-14T18:26:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.535+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7ffc010460 con 0x7f800c0ff440 2023-12-14T18:26:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.536+0000 7f8008ff9700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ff40a1260 0x7f7ff40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.536+0000 7f800affd700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ff40a1260 0x7f7ff40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.536+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7ffc0ccfe0 con 0x7f800c0ff440 2023-12-14T18:26:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.536+0000 7f800affd700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ff40a1260 0x7f7ff40a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8000000c00 tx=0x7f8000005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:29.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.539+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7ffc0963c0 con 0x7f800c0ff440 2023-12-14T18:26:29.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.815+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f800c066ea0 con 0x7f800c0ff440 2023-12-14T18:26:29.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.820+0000 7f8008ff9700 1 -- 172.21.15.31:0/3732063384 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260126 (secure 0 0 0) 0x7f7ffc016090 con 0x7f800c0ff440 2023-12-14T18:26:29.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ff40a1260 msgr2=0x7f7ff40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ff40a1260 0x7f7ff40a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8000000c00 tx=0x7f8000005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 msgr2=0x7f800c195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 0x7f800c195280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00d950 tx=0x7f7ffc00dd10 comp rx=0 tx=0).stop 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 shutdown_connections 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f800c1044e0 0x7f800c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f800c0ff440 0x7f800c195280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ff40a1260 0x7f7ff40a3710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 --2- 172.21.15.31:0/3732063384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f800c0ffde0 0x7f800c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 >> 172.21.15.31:0/3732063384 conn(0x7f800c0facc0 msgr2=0x7f800c0fb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:29.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 shutdown_connections 2023-12-14T18:26:29.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:29.824+0000 7f8011c97700 1 -- 172.21.15.31:0/3732063384 wait complete. 2023-12-14T18:26:30.200 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2688246057},{"type":"v1","addr":"172.21.15.31:6801","nonce":2688246057}]},"active_addr":"172.21.15.31:6801/2688246057","active_change":"2023-12-14T18:20:47.403823+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3645411014}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2160431679}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2924181707}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3108903497}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3199333831}]}]} 2023-12-14T18:26:30.204 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-14T18:26:30.204 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-14T18:26:30.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-14T18:26:30.558 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:30.866 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:30 smithi031 ceph-mon[42085]: pgmap v246: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2023-12-14T18:26:30.866 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:30 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3732063384' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-14T18:26:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:30 smithi176 ceph-mon[43506]: pgmap v246: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2023-12-14T18:26:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:30 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3732063384' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-14T18:26:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:30 smithi204 ceph-mon[43866]: pgmap v246: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2023-12-14T18:26:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:30 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3732063384' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-14T18:26:31.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 -- 172.21.15.31:0/247854298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 msgr2=0x7fba4010ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:31.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 --2- 172.21.15.31:0/247854298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba4010ccd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fba3800b0a0 tx=0x7fba3800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 -- 172.21.15.31:0/247854298 shutdown_connections 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 --2- 172.21.15.31:0/247854298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba4010ccd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 --2- 172.21.15.31:0/247854298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba40101540 0x7fba40101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 --2- 172.21.15.31:0/247854298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba40101e50 0x7fba4010a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 -- 172.21.15.31:0/247854298 >> 172.21.15.31:0/247854298 conn(0x7fba400770a0 msgr2=0x7fba400774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 -- 172.21.15.31:0/247854298 shutdown_connections 2023-12-14T18:26:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.594+0000 7fba48575700 1 -- 172.21.15.31:0/247854298 wait complete. 2023-12-14T18:26:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 Processor -- start 2023-12-14T18:26:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 -- start start 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba40101540 0x7fba400ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba40101e50 0x7fba400ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba400ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba401a4980 con 0x7fba4010a890 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba401a4af0 con 0x7fba40101e50 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba48575700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba401a4cb0 con 0x7fba40101540 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba46b12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba400ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba46b12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba400ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35412/0 (socket says 172.21.15.31:35412) 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.595+0000 7fba46b12700 1 -- 172.21.15.31:0/2499846732 learned_addr learned my addr 172.21.15.31:0/2499846732 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46311700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba40101540 0x7fba400ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba45b10700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba40101e50 0x7fba400ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46b12700 1 -- 172.21.15.31:0/2499846732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba40101540 msgr2=0x7fba400ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46b12700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba40101540 0x7fba400ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46b12700 1 -- 172.21.15.31:0/2499846732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba40101e50 msgr2=0x7fba400ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46b12700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba40101e50 0x7fba400ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46b12700 1 -- 172.21.15.31:0/2499846732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba38009d00 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba46b12700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba400ffc60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fba38009fd0 tx=0x7fba3800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800e040 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba38004e60 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba40110350 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38005280 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.596+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba401101c0 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.597+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba401104e0 con 0x7fba4010a890 2023-12-14T18:26:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.599+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fba3801d470 con 0x7fba4010a890 2023-12-14T18:26:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.599+0000 7fba377fe700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba2c0a1260 0x7fba2c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.599+0000 7fba46311700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba2c0a1260 0x7fba2c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.599+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fba380cdd80 con 0x7fba4010a890 2023-12-14T18:26:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.600+0000 7fba46311700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba2c0a1260 0x7fba2c0a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009c60 tx=0x7fba3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.604+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba38097210 con 0x7fba4010a890 2023-12-14T18:26:31.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.814+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba401006d0 con 0x7fba4010a890 2023-12-14T18:26:31.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.815+0000 7fba377fe700 1 -- 172.21.15.31:0/2499846732 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7fba3809a830 con 0x7fba4010a890 2023-12-14T18:26:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:31.817 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:31.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.817+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba2c0a1260 msgr2=0x7fba2c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.817+0000 7fba48575700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba2c0a1260 0x7fba2c0a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009c60 tx=0x7fba3c009260 comp rx=0 tx=0).stop 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.817+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 msgr2=0x7fba400ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.817+0000 7fba48575700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba400ffc60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fba38009fd0 tx=0x7fba3800bce0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 shutdown_connections 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba4010a890 0x7fba400ffc60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba2c0a1260 0x7fba2c0a3710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba40101540 0x7fba400ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 --2- 172.21.15.31:0/2499846732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba40101e50 0x7fba400ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 >> 172.21.15.31:0/2499846732 conn(0x7fba400770a0 msgr2=0x7fba400fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 shutdown_connections 2023-12-14T18:26:31.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:31.818+0000 7fba48575700 1 -- 172.21.15.31:0/2499846732 wait complete. 2023-12-14T18:26:32.207 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-14T18:26:32.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-14T18:26:32.565 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:32 smithi031 ceph-mon[42085]: pgmap v247: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2023-12-14T18:26:32.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:32 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2499846732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:32 smithi176 ceph-mon[43506]: pgmap v247: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2023-12-14T18:26:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:32 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2499846732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:32 smithi204 ceph-mon[43866]: pgmap v247: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 13 op/s 2023-12-14T18:26:33.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:32 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2499846732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.613+0000 7fa57f5c7700 1 -- 172.21.15.31:0/3583601440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ff470 msgr2=0x7fa5780ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.613+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/3583601440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ff470 0x7fa5780ff840 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa5740099c0 tx=0x7fa574009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 -- 172.21.15.31:0/3583601440 shutdown_connections 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/3583601440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa578104510 0x7fa57810c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/3583601440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ff470 0x7fa5780ff840 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/3583601440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5780ffe10 0x7fa578103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 -- 172.21.15.31:0/3583601440 >> 172.21.15.31:0/3583601440 conn(0x7fa5780770d0 msgr2=0x7fa5780774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:33.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 -- 172.21.15.31:0/3583601440 shutdown_connections 2023-12-14T18:26:33.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 -- 172.21.15.31:0/3583601440 wait complete. 2023-12-14T18:26:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 Processor -- start 2023-12-14T18:26:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.614+0000 7fa57f5c7700 1 -- start start 2023-12-14T18:26:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57f5c7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5780ff470 0x7fa578100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57f5c7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 0x7fa5781013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57f5c7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa578104510 0x7fa578101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57f5c7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5781a47b0 con 0x7fa5780ffe10 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57f5c7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5781a4920 con 0x7fa5780ff470 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57f5c7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5781a4ae0 con 0x7fa578104510 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 0x7fa5781013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 0x7fa5781013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35422/0 (socket says 172.21.15.31:35422) 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 -- 172.21.15.31:0/1288704690 learned_addr learned my addr 172.21.15.31:0/1288704690 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57db64700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa578104510 0x7fa578101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57d363700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5780ff470 0x7fa578100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 -- 172.21.15.31:0/1288704690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa578104510 msgr2=0x7fa578101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa578104510 0x7fa578101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 -- 172.21.15.31:0/1288704690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5780ff470 msgr2=0x7fa578100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:33.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5780ff470 0x7fa578100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.615+0000 7fa57cb62700 1 -- 172.21.15.31:0/1288704690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740095f0 con 0x7fa5780ffe10 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.616+0000 7fa57cb62700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 0x7fa5781013c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa56800d980 tx=0x7fa56800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.616+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800f890 con 0x7fa5780ffe10 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.616+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57810f250 con 0x7fa5780ffe10 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.616+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa568010460 con 0x7fa5780ffe10 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.616+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800e610 con 0x7fa5780ffe10 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.616+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57810f0c0 con 0x7fa5780ffe10 2023-12-14T18:26:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.617+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57810f3e0 con 0x7fa5780ffe10 2023-12-14T18:26:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.618+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa5680105d0 con 0x7fa5780ffe10 2023-12-14T18:26:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.619+0000 7fa56e7fc700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5640a11e0 0x7fa5640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.619+0000 7fa57d363700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5640a11e0 0x7fa5640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.619+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa5680cc6f0 con 0x7fa5780ffe10 2023-12-14T18:26:33.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.619+0000 7fa57d363700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5640a11e0 0x7fa5640a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa574000c00 tx=0x7fa574003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.623+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa568095a90 con 0x7fa5780ffe10 2023-12-14T18:26:33.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.836+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5781023b0 con 0x7fa5780ffe10 2023-12-14T18:26:33.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.838+0000 7fa56e7fc700 1 -- 172.21.15.31:0/1288704690 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23983 (secure 0 0 0) 0x7fa5680990b0 con 0x7fa5780ffe10 2023-12-14T18:26:33.840 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:26:33.840 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":85,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:25:59.779415+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5640a11e0 msgr2=0x7fa5640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5640a11e0 0x7fa5640a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa574000c00 tx=0x7fa574003680 comp rx=0 tx=0).stop 2023-12-14T18:26:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 msgr2=0x7fa5781013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 0x7fa5781013c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa56800d980 tx=0x7fa56800dc90 comp rx=0 tx=0).stop 2023-12-14T18:26:33.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 shutdown_connections 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5640a11e0 0x7fa5640a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa578104510 0x7fa578101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5780ff470 0x7fa578100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 --2- 172.21.15.31:0/1288704690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ffe10 0x7fa5781013c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 >> 172.21.15.31:0/1288704690 conn(0x7fa5780770d0 msgr2=0x7fa578103020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 shutdown_connections 2023-12-14T18:26:33.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:33.840+0000 7fa57f5c7700 1 -- 172.21.15.31:0/1288704690 wait complete. 2023-12-14T18:26:34.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-14T18:26:34.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-14T18:26:34.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-14T18:26:34.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-14T18:26:34.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-14T18:26:34.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-14T18:26:34.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-14T18:26:34.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-14T18:26:34.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-14T18:26:34.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-14T18:26:34.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-14T18:26:34.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-14T18:26:34.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:34 smithi031 ceph-mon[42085]: pgmap v248: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2023-12-14T18:26:34.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:34 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1288704690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:34 smithi204 ceph-mon[43866]: pgmap v248: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2023-12-14T18:26:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:34 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1288704690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:34 smithi176 ceph-mon[43506]: pgmap v248: 129 pgs: 129 active+clean; 627 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2023-12-14T18:26:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:34 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1288704690' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:35.090 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:35.234 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:36.605 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 -- 172.21.15.31:0/4245399321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 msgr2=0x7f89c010a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 --2- 172.21.15.31:0/4245399321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c010a350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009990 tx=0x7f89a8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 -- 172.21.15.31:0/4245399321 shutdown_connections 2023-12-14T18:26:36.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 --2- 172.21.15.31:0/4245399321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c010a890 0x7f89c010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 --2- 172.21.15.31:0/4245399321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c0101540 0x7f89c0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 --2- 172.21.15.31:0/4245399321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c010a350 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 -- 172.21.15.31:0/4245399321 >> 172.21.15.31:0/4245399321 conn(0x7f89c00770a0 msgr2=0x7f89c00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 -- 172.21.15.31:0/4245399321 shutdown_connections 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.670+0000 7f89c5e07700 1 -- 172.21.15.31:0/4245399321 wait complete. 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 Processor -- start 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 -- start start 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c0101540 0x7f89c01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:36.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c010a890 0x7f89c019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019d570 con 0x7f89c0101e50 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019d6e0 con 0x7f89c0101540 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89c5e07700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019d8a0 con 0x7f89c010a890 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89b6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89b6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47994/0 (socket says 172.21.15.31:47994) 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89b6ffd700 1 -- 172.21.15.31:0/2362724639 learned_addr learned my addr 172.21.15.31:0/2362724639 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89bffff700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c010a890 0x7f89c019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.671+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c0101540 0x7f89c01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:36.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89b6ffd700 1 -- 172.21.15.31:0/2362724639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c010a890 msgr2=0x7f89c019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89b6ffd700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c010a890 0x7f89c019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89b6ffd700 1 -- 172.21.15.31:0/2362724639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c0101540 msgr2=0x7f89c01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89b6ffd700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c0101540 0x7f89c01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89b6ffd700 1 -- 172.21.15.31:0/2362724639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80095f0 con 0x7f89c0101e50 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89b6ffd700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c0198af0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f89a8006010 tx=0x7f89a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a801d070 con 0x7f89c0101e50 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89a8022470 con 0x7f89c0101e50 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c0110310 con 0x7f89c0101e50 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8017650 con 0x7f89c0101e50 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.672+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c0110180 con 0x7f89c0101e50 2023-12-14T18:26:36.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.673+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89c0100b90 con 0x7f89c0101e50 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.675+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f89a80225e0 con 0x7f89c0101e50 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.675+0000 7f89bd7fa700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89ac0a11e0 0x7f89ac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.675+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f89a80ce1d0 con 0x7f89c0101e50 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.675+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89ac0a11e0 0x7f89ac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.676+0000 7f89bd7fa700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] conn(0x7f89ac0abd50 0x7f89ac0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.676+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 --> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f89ac0ae850 con 0x7f89ac0abd50 2023-12-14T18:26:36.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.676+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89a80ce670 con 0x7f89c0101e50 2023-12-14T18:26:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.676+0000 7f89bffff700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] conn(0x7f89ac0abd50 0x7f89ac0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.676+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89ac0a11e0 0x7f89ac0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007be0 tx=0x7f89b00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.676+0000 7f89bffff700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] conn(0x7f89ac0abd50 0x7f89ac0ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.677+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== osd.3 v2:172.21.15.31:6810/3446506620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f89ac0ae850 con 0x7f89ac0abd50 2023-12-14T18:26:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 --> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f89c01104a0 con 0x7f89ac0abd50 2023-12-14T18:26:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89bd7fa700 1 -- 172.21.15.31:0/2362724639 <== osd.3 v2:172.21.15.31:6810/3446506620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f89c01104a0 con 0x7f89ac0abd50 2023-12-14T18:26:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] conn(0x7f89ac0abd50 msgr2=0x7f89ac0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] conn(0x7f89ac0abd50 0x7f89ac0ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89ac0a11e0 msgr2=0x7f89ac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89ac0a11e0 0x7f89ac0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f89b0007be0 tx=0x7f89b00073d0 comp rx=0 tx=0).stop 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 msgr2=0x7f89c0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89c5e07700 1 --2- 172.21.15.31:0/2362724639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c0101e50 0x7f89c0198af0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f89a8006010 tx=0x7f89a800f690 comp rx=0 tx=0).stop 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.694+0000 7f89bffff700 1 -- 172.21.15.31:0/2362724639 reap_dead start 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.695+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 shutdown_connections 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.695+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 >> 172.21.15.31:0/2362724639 conn(0x7f89c00770a0 msgr2=0x7f89c01001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.695+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 shutdown_connections 2023-12-14T18:26:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:36.695+0000 7f89c5e07700 1 -- 172.21.15.31:0/2362724639 wait complete. 2023-12-14T18:26:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:36 smithi204 ceph-mon[43866]: pgmap v249: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-12-14T18:26:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:36 smithi031 ceph-mon[42085]: pgmap v249: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-12-14T18:26:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:36 smithi176 ceph-mon[43506]: pgmap v249: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.2 KiB/s wr, 13 op/s 2023-12-14T18:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.485+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2596942115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0104500 msgr2=0x7fb1a010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:37.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.485+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2596942115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0104500 0x7fb1a010c080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb19800b0d0 tx=0x7fb19800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.485+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2596942115 shutdown_connections 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2596942115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0104500 0x7fb1a010c080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2596942115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a00ff460 0x7fb1a00ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2596942115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a00ffe00 0x7fb1a0103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2596942115 >> 172.21.15.31:0/2596942115 conn(0x7fb1a00770a0 msgr2=0x7fb1a00774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2596942115 shutdown_connections 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2596942115 wait complete. 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 Processor -- start 2023-12-14T18:26:37.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.486+0000 7fb1a79fa700 1 -- start start 2023-12-14T18:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a79fa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00ff460 0x7fb1a0196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a79fa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a00ffe00 0x7fb1a0196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a79fa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a0104500 0x7fb1a019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:37.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a79fa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a019b4e0 con 0x7fb1a00ff460 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a79fa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a019b650 con 0x7fb1a0104500 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a79fa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a019b810 con 0x7fb1a00ffe00 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00ff460 0x7fb1a0196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00ff460 0x7fb1a0196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48012/0 (socket says 172.21.15.31:48012) 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 -- 172.21.15.31:0/2180158453 learned_addr learned my addr 172.21.15.31:0/2180158453 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5f97700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a0104500 0x7fb1a019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a4f95700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a00ffe00 0x7fb1a0196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 -- 172.21.15.31:0/2180158453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a00ffe00 msgr2=0x7fb1a0196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a00ffe00 0x7fb1a0196a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 -- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a0104500 msgr2=0x7fb1a019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1a0104500 0x7fb1a019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 -- 172.21.15.31:0/2180158453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009d00 con 0x7fb1a00ff460 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1a5796700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00ff460 0x7fb1a0196520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00d950 tx=0x7fb19c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.487+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00f890 con 0x7fb1a00ff460 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.488+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb19c010460 con 0x7fb1a00ff460 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.488+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00e610 con 0x7fb1a00ff460 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.488+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a010f240 con 0x7fb1a00ff460 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.488+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a010f0b0 con 0x7fb1a00ff460 2023-12-14T18:26:37.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.489+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb1a0069040 con 0x7fb1a00ff460 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.490+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb19c0108d0 con 0x7fb1a00ff460 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1967fc700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb18c0a11e0 0x7fb18c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb19c0ccf80 con 0x7fb1a00ff460 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1a4f95700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb18c0a11e0 0x7fb18c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1967fc700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] conn(0x7fb18c0abd50 0x7fb18c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 --> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb18c0ae850 con 0x7fb18c0abd50 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb19c0cd390 con 0x7fb1a00ff460 2023-12-14T18:26:37.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1a4f95700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb18c0a11e0 0x7fb18c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0197b40 tx=0x7fb19000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.491+0000 7fb1a5f97700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] conn(0x7fb18c0abd50 0x7fb18c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.492+0000 7fb1a5f97700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] conn(0x7fb18c0abd50 0x7fb18c0ae180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:37.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.492+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== osd.10 v2:172.21.15.176:6824/3741647642 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb18c0ae850 con 0x7fb18c0abd50 2023-12-14T18:26:37.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.511+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 --> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb1a010f3d0 con 0x7fb18c0abd50 2023-12-14T18:26:37.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1967fc700 1 -- 172.21.15.31:0/2180158453 <== osd.10 v2:172.21.15.176:6824/3741647642 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb1a010f3d0 con 0x7fb18c0abd50 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] conn(0x7fb18c0abd50 msgr2=0x7fb18c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] conn(0x7fb18c0abd50 0x7fb18c0ae180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb18c0a11e0 msgr2=0x7fb18c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb18c0a11e0 0x7fb18c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0197b40 tx=0x7fb19000b410 comp rx=0 tx=0).stop 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00ff460 msgr2=0x7fb1a0196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 --2- 172.21.15.31:0/2180158453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00ff460 0x7fb1a0196520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb19c00d950 tx=0x7fb19c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a5f97700 1 -- 172.21.15.31:0/2180158453 reap_dead start 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 shutdown_connections 2023-12-14T18:26:37.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 >> 172.21.15.31:0/2180158453 conn(0x7fb1a00770a0 msgr2=0x7fb1a0068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 shutdown_connections 2023-12-14T18:26:37.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:37.512+0000 7fb1a79fa700 1 -- 172.21.15.31:0/2180158453 wait complete. 2023-12-14T18:26:37.854 INFO:teuthology.orchestra.run.smithi031.stdout:103079215151 2023-12-14T18:26:37.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-14T18:26:37.874 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 -- 172.21.15.31:0/1442997639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 msgr2=0x7fa35410a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 --2- 172.21.15.31:0/1442997639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa35410a350 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa344009990 tx=0x7fa344009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 -- 172.21.15.31:0/1442997639 shutdown_connections 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 --2- 172.21.15.31:0/1442997639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa35410a890 0x7fa35410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 --2- 172.21.15.31:0/1442997639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa354101540 0x7fa354101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 --2- 172.21.15.31:0/1442997639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa35410a350 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 -- 172.21.15.31:0/1442997639 >> 172.21.15.31:0/1442997639 conn(0x7fa3540770a0 msgr2=0x7fa3540774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 -- 172.21.15.31:0/1442997639 shutdown_connections 2023-12-14T18:26:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.141+0000 7fa35c01d700 1 -- 172.21.15.31:0/1442997639 wait complete. 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 Processor -- start 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 -- start start 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa354101540 0x7fa354196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa354196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa35410a890 0x7fa35419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419b400 con 0x7fa354101e50 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419b570 con 0x7fa354101540 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa35c01d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419b730 con 0x7fa35410a890 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa3595b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa354196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa3595b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa354196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48024/0 (socket says 172.21.15.31:48024) 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.142+0000 7fa3595b8700 1 -- 172.21.15.31:0/125577260 learned_addr learned my addr 172.21.15.31:0/125577260 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa35a5ba700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa35410a890 0x7fa35419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa359db9700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa354101540 0x7fa354196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa3595b8700 1 -- 172.21.15.31:0/125577260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa35410a890 msgr2=0x7fa35419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa3595b8700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa35410a890 0x7fa35419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa3595b8700 1 -- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa354101540 msgr2=0x7fa354196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa3595b8700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa354101540 0x7fa354196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa3595b8700 1 -- 172.21.15.31:0/125577260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440095f0 con 0x7fa354101e50 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa3595b8700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa354196980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098e0 tx=0x7fa34400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34401d070 con 0x7fa354101e50 2023-12-14T18:26:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa354110310 con 0x7fa354101e50 2023-12-14T18:26:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa344005510 con 0x7fa354101e50 2023-12-14T18:26:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344004c90 con 0x7fa354101e50 2023-12-14T18:26:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.143+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa354110180 con 0x7fa354101e50 2023-12-14T18:26:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.144+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3540fd8e0 con 0x7fa354101e50 2023-12-14T18:26:38.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.147+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa34400fc10 con 0x7fa354101e50 2023-12-14T18:26:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa34affd700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3400a11e0 0x7fa3400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa3440ce680 con 0x7fa354101e50 2023-12-14T18:26:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa359db9700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3400a11e0 0x7fa3400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa34affd700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] conn(0x7fa3400abd50 0x7fa3400ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 --> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3400ae850 con 0x7fa3400abd50 2023-12-14T18:26:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3440ce9f0 con 0x7fa354101e50 2023-12-14T18:26:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa35a5ba700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] conn(0x7fa3400abd50 0x7fa3400ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.148+0000 7fa359db9700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3400a11e0 0x7fa3400a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa350005fd0 tx=0x7fa350005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.149+0000 7fa35a5ba700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] conn(0x7fa3400abd50 0x7fa3400ae180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.150+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== osd.4 v2:172.21.15.176:6808/2531574099 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa3400ae850 con 0x7fa3400abd50 2023-12-14T18:26:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.175+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 --> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa3541104a0 con 0x7fa3400abd50 2023-12-14T18:26:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa34affd700 1 -- 172.21.15.31:0/125577260 <== osd.4 v2:172.21.15.176:6808/2531574099 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa3541104a0 con 0x7fa3400abd50 2023-12-14T18:26:38.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] conn(0x7fa3400abd50 msgr2=0x7fa3400ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:38.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35c01d700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] conn(0x7fa3400abd50 0x7fa3400ae180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:38.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3400a11e0 msgr2=0x7fa3400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:38.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35c01d700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa3400a11e0 0x7fa3400a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa350005fd0 tx=0x7fa350005f60 comp rx=0 tx=0).stop 2023-12-14T18:26:38.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 msgr2=0x7fa354196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:38.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35c01d700 1 --2- 172.21.15.31:0/125577260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa354101e50 0x7fa354196980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098e0 tx=0x7fa34400f720 comp rx=0 tx=0).stop 2023-12-14T18:26:38.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.176+0000 7fa35a5ba700 1 -- 172.21.15.31:0/125577260 reap_dead start 2023-12-14T18:26:38.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.177+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 shutdown_connections 2023-12-14T18:26:38.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.177+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 >> 172.21.15.31:0/125577260 conn(0x7fa3540770a0 msgr2=0x7fa3540fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:38.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.177+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 shutdown_connections 2023-12-14T18:26:38.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:38.177+0000 7fa35c01d700 1 -- 172.21.15.31:0/125577260 wait complete. 2023-12-14T18:26:38.479 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:38 smithi204 ceph-mon[43866]: pgmap v250: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2023-12-14T18:26:38.931 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:38 smithi031 ceph-mon[42085]: pgmap v250: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2023-12-14T18:26:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:38 smithi176 ceph-mon[43506]: pgmap v250: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2023-12-14T18:26:39.929 INFO:teuthology.orchestra.run.smithi031.stdout:292057776150 2023-12-14T18:26:39.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-14T18:26:40.646 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:40.683 INFO:teuthology.orchestra.run.smithi031.stdout:133143986220 2023-12-14T18:26:40.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-14T18:26:40.700 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:40 smithi031 ceph-mon[42085]: pgmap v251: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2023-12-14T18:26:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:40.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.832+0000 7f3cc2705700 1 -- 172.21.15.31:0/3247342474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 msgr2=0x7f3cbc10c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.832+0000 7f3cc2705700 1 --2- 172.21.15.31:0/3247342474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 0x7f3cbc10c060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cb400b0d0 tx=0x7f3cb400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 -- 172.21.15.31:0/3247342474 shutdown_connections 2023-12-14T18:26:40.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 --2- 172.21.15.31:0/3247342474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 0x7f3cbc10c060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 --2- 172.21.15.31:0/3247342474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cbc1041c0 0x7f3cbc104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 --2- 172.21.15.31:0/3247342474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cbc0ff980 0x7f3cbc0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 -- 172.21.15.31:0/3247342474 >> 172.21.15.31:0/3247342474 conn(0x7f3cbc0758a0 msgr2=0x7f3cbc075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:40.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 -- 172.21.15.31:0/3247342474 shutdown_connections 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 -- 172.21.15.31:0/3247342474 wait complete. 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.833+0000 7f3cc2705700 1 Processor -- start 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 -- start start 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cbc0ff980 0x7f3cbc10ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 0x7f3cbc10b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cbc1041c0 0x7f3cbc1b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc10ba00 con 0x7f3cbc1041c0 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc06a510 con 0x7f3cbc0ff980 2023-12-14T18:26:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc2705700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc06a6d0 con 0x7f3cbc100330 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc0ca2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cbc1041c0 0x7f3cbc1b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc0ca2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cbc1041c0 0x7f3cbc1b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48066/0 (socket says 172.21.15.31:48066) 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cc0ca2700 1 -- 172.21.15.31:0/1692932423 learned_addr learned my addr 172.21.15.31:0/1692932423 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.834+0000 7f3cb3fff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 0x7f3cbc10b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cbbfff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cbc0ff980 0x7f3cbc10ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb3fff700 1 -- 172.21.15.31:0/1692932423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cbc0ff980 msgr2=0x7f3cbc10ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb3fff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cbc0ff980 0x7f3cbc10ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb3fff700 1 -- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cbc1041c0 msgr2=0x7f3cbc1b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb3fff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cbc1041c0 0x7f3cbc1b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb3fff700 1 -- 172.21.15.31:0/1692932423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4009d00 con 0x7f3cbc100330 2023-12-14T18:26:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cbbfff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3cbc0ff980 0x7f3cbc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:40.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:40 smithi204 ceph-mon[43866]: pgmap v251: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb3fff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 0x7f3cbc10b1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400b450 tx=0x7f3ca400b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4011830 con 0x7f3cbc100330 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca4011e50 con 0x7f3cbc100330 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4010e70 con 0x7f3cbc100330 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cbc1104a0 con 0x7f3cbc100330 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.835+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cbc110310 con 0x7f3cbc100330 2023-12-14T18:26:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.838+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3cbc0fcd50 con 0x7f3cbc100330 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.839+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ca4010510 con 0x7f3cbc100330 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.839+0000 7f3cb9ffb700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ca80a1290 0x7f3ca80a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.839+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3ca40cc4c0 con 0x7f3cbc100330 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.840+0000 7f3cbbfff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ca80a1290 0x7f3ca80a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.840+0000 7f3cb9ffb700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] conn(0x7f3ca80abe00 0x7f3ca80ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.840+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 --> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ca80ae900 con 0x7f3ca80abe00 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.840+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca40ceb40 con 0x7f3cbc100330 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.840+0000 7f3cbbfff700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ca80a1290 0x7f3ca80a3740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009cb0 tx=0x7f3cac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.841+0000 7f3cc0ca2700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] conn(0x7f3ca80abe00 0x7f3ca80ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.842+0000 7f3cc0ca2700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] conn(0x7f3ca80abe00 0x7f3ca80ae230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.842+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== osd.11 v2:172.21.15.204:6824/2063130933 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3ca80ae900 con 0x7f3ca80abe00 2023-12-14T18:26:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.861+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 --> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3cbc110630 con 0x7f3ca80abe00 2023-12-14T18:26:40.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cb9ffb700 1 -- 172.21.15.31:0/1692932423 <== osd.11 v2:172.21.15.204:6824/2063130933 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3cbc110630 con 0x7f3ca80abe00 2023-12-14T18:26:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] conn(0x7f3ca80abe00 msgr2=0x7f3ca80ae230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] conn(0x7f3ca80abe00 0x7f3ca80ae230 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ca80a1290 msgr2=0x7f3ca80a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ca80a1290 0x7f3ca80a3740 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009cb0 tx=0x7f3cac009260 comp rx=0 tx=0).stop 2023-12-14T18:26:40.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 msgr2=0x7f3cbc10b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 --2- 172.21.15.31:0/1692932423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cbc100330 0x7f3cbc10b1b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400b450 tx=0x7f3ca400b760 comp rx=0 tx=0).stop 2023-12-14T18:26:40.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc0ca2700 1 -- 172.21.15.31:0/1692932423 reap_dead start 2023-12-14T18:26:40.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 shutdown_connections 2023-12-14T18:26:40.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 >> 172.21.15.31:0/1692932423 conn(0x7f3cbc0758a0 msgr2=0x7f3cbc0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:40.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 shutdown_connections 2023-12-14T18:26:40.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:40.862+0000 7f3cc2705700 1 -- 172.21.15.31:0/1692932423 wait complete. 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 -- 172.21.15.31:0/116488449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 msgr2=0x7efec810ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 --2- 172.21.15.31:0/116488449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec810ccd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efec000b0a0 tx=0x7efec000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 -- 172.21.15.31:0/116488449 shutdown_connections 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 --2- 172.21.15.31:0/116488449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec810ccd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 --2- 172.21.15.31:0/116488449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efec8101540 0x7efec8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 --2- 172.21.15.31:0/116488449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec8101e50 0x7efec810a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 -- 172.21.15.31:0/116488449 >> 172.21.15.31:0/116488449 conn(0x7efec80770a0 msgr2=0x7efec80774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 -- 172.21.15.31:0/116488449 shutdown_connections 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.589+0000 7efecc925700 1 -- 172.21.15.31:0/116488449 wait complete. 2023-12-14T18:26:40.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 Processor -- start 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 -- start start 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efec8101540 0x7efec8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec8101e50 0x7efec8196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec819b4d0 con 0x7efec810a890 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec819b640 con 0x7efec8101540 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efecc925700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec819b800 con 0x7efec8101e50 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48042/0 (socket says 172.21.15.31:48042) 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 -- 172.21.15.31:0/559673082 learned_addr learned my addr 172.21.15.31:0/559673082 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec659c700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efec8101540 0x7efec8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec5d9b700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec8101e50 0x7efec8196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 -- 172.21.15.31:0/559673082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec8101e50 msgr2=0x7efec8196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec8101e50 0x7efec8196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 -- 172.21.15.31:0/559673082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efec8101540 msgr2=0x7efec8196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efec8101540 0x7efec8196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.590+0000 7efec6d9d700 1 -- 172.21.15.31:0/559673082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0009d00 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.591+0000 7efec6d9d700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec819aec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efec000b830 tx=0x7efec00079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.591+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000e040 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.591+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efec0004ce0 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.591+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec8110300 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.591+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec001d480 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.591+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec8110170 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.592+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efec80fd760 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.593+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efec0003a00 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efeb77fe700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efeb00a1260 0x7efeb00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efec659c700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efeb00a1260 0x7efeb00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efec00ce320 con 0x7efec810a890 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efeb77fe700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] conn(0x7efeb00abdd0 0x7efeb00ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 --> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efeb00ae8d0 con 0x7efeb00abdd0 2023-12-14T18:26:40.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7efec00d0900 con 0x7efec810a890 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efec659c700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efeb00a1260 0x7efeb00a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005950 tx=0x7efeb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.594+0000 7efec5d9b700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] conn(0x7efeb00abdd0 0x7efeb00ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.595+0000 7efec5d9b700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] conn(0x7efeb00abdd0 0x7efeb00ae200 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.595+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== osd.0 v2:172.21.15.31:6802/4133067944 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efeb00ae8d0 con 0x7efeb00abdd0 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.612+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 --> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efec8110490 con 0x7efeb00abdd0 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efeb77fe700 1 -- 172.21.15.31:0/559673082 <== osd.0 v2:172.21.15.31:6802/4133067944 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efec8110490 con 0x7efeb00abdd0 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] conn(0x7efeb00abdd0 msgr2=0x7efeb00ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] conn(0x7efeb00abdd0 0x7efeb00ae200 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efeb00a1260 msgr2=0x7efeb00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efeb00a1260 0x7efeb00a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005950 tx=0x7efeb800b410 comp rx=0 tx=0).stop 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 msgr2=0x7efec819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 --2- 172.21.15.31:0/559673082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efec810a890 0x7efec819aec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efec000b830 tx=0x7efec00079b0 comp rx=0 tx=0).stop 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efec6d9d700 1 -- 172.21.15.31:0/559673082 reap_dead start 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 shutdown_connections 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 >> 172.21.15.31:0/559673082 conn(0x7efec80770a0 msgr2=0x7efec80fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 shutdown_connections 2023-12-14T18:26:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:39.613+0000 7efecc925700 1 -- 172.21.15.31:0/559673082 wait complete. 2023-12-14T18:26:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:40 smithi176 ceph-mon[43506]: pgmap v251: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2023-12-14T18:26:41.909 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:42.145 INFO:teuthology.orchestra.run.smithi031.stdout:326417514515 2023-12-14T18:26:42.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-14T18:26:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 -- 172.21.15.31:0/1588811720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106370 msgr2=0x7fa150106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 --2- 172.21.15.31:0/1588811720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106370 0x7fa150106740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009990 tx=0x7fa14c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 -- 172.21.15.31:0/1588811720 shutdown_connections 2023-12-14T18:26:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 --2- 172.21.15.31:0/1588811720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100e40 0x7fa15010d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 --2- 172.21.15.31:0/1588811720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150106370 0x7fa150106740 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 --2- 172.21.15.31:0/1588811720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa150100490 0x7fa150100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.199+0000 7fa157a0e700 1 -- 172.21.15.31:0/1588811720 >> 172.21.15.31:0/1588811720 conn(0x7fa150078430 msgr2=0x7fa150078830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.200+0000 7fa157a0e700 1 -- 172.21.15.31:0/1588811720 shutdown_connections 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.200+0000 7fa157a0e700 1 -- 172.21.15.31:0/1588811720 wait complete. 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.200+0000 7fa157a0e700 1 Processor -- start 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.200+0000 7fa157a0e700 1 -- start start 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa157a0e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100490 0x7fa150196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa157a0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150100e40 0x7fa150196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa157a0e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa150106370 0x7fa15019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa157a0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15019b3c0 con 0x7fa150100e40 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa157a0e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15019b530 con 0x7fa150100490 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa157a0e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15019b6f0 con 0x7fa150106370 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100490 0x7fa150196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa154fa9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150100e40 0x7fa150196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100490 0x7fa150196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41940/0 (socket says 172.21.15.31:41940) 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 -- 172.21.15.31:0/2839424132 learned_addr learned my addr 172.21.15.31:0/2839424132 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:42.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa155fab700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa150106370 0x7fa15019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 -- 172.21.15.31:0/2839424132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa150106370 msgr2=0x7fa15019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa150106370 0x7fa15019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 -- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150100e40 msgr2=0x7fa150196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa150100e40 0x7fa150196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.201+0000 7fa1557aa700 1 -- 172.21.15.31:0/2839424132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0095f0 con 0x7fa150100490 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.202+0000 7fa1557aa700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100490 0x7fa150196400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00b680 tx=0x7fa14c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.202+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c01d070 con 0x7fa150100490 2023-12-14T18:26:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.203+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa150110350 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.203+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1501101c0 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.203+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa14c021410 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.203+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c017650 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.203+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa134000ff0 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.206+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa14c00fb10 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.206+0000 7fa1467fc700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa13c0a1260 0x7fa13c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.207+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa14c0ce3b0 con 0x7fa150100490 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.207+0000 7fa1467fc700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] conn(0x7fa13c0abdd0 0x7fa13c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.207+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 --> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa13c0ae8d0 con 0x7fa13c0abdd0 2023-12-14T18:26:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.207+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa14c0ce720 con 0x7fa150100490 2023-12-14T18:26:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.207+0000 7fa154fa9700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa13c0a1260 0x7fa13c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.207+0000 7fa155fab700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] conn(0x7fa13c0abdd0 0x7fa13c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.208+0000 7fa154fa9700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa13c0a1260 0x7fa13c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa150197a20 tx=0x7fa140009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:42.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.208+0000 7fa155fab700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] conn(0x7fa13c0abdd0 0x7fa13c0ae200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.209+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== osd.1 v2:172.21.15.176:6800/3594574093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa13c0ae8d0 con 0x7fa13c0abdd0 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.227+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 --> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa134002d70 con 0x7fa13c0abdd0 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.228+0000 7fa1467fc700 1 -- 172.21.15.31:0/2839424132 <== osd.1 v2:172.21.15.176:6800/3594574093 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa134002d70 con 0x7fa13c0abdd0 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] conn(0x7fa13c0abdd0 msgr2=0x7fa13c0ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] conn(0x7fa13c0abdd0 0x7fa13c0ae200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa13c0a1260 msgr2=0x7fa13c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa13c0a1260 0x7fa13c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa150197a20 tx=0x7fa140009260 comp rx=0 tx=0).stop 2023-12-14T18:26:42.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100490 msgr2=0x7fa150196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 --2- 172.21.15.31:0/2839424132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa150100490 0x7fa150196400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00b680 tx=0x7fa14c004730 comp rx=0 tx=0).stop 2023-12-14T18:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa155fab700 1 -- 172.21.15.31:0/2839424132 reap_dead start 2023-12-14T18:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 shutdown_connections 2023-12-14T18:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 >> 172.21.15.31:0/2839424132 conn(0x7fa150078430 msgr2=0x7fa1500fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 shutdown_connections 2023-12-14T18:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.229+0000 7fa157a0e700 1 -- 172.21.15.31:0/2839424132 wait complete. 2023-12-14T18:26:42.670 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:42 smithi031 ceph-mon[42085]: pgmap v252: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-12-14T18:26:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 -- 172.21.15.31:0/2646938011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 msgr2=0x7f49d810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 --2- 172.21.15.31:0/2646938011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d810a350 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009990 tx=0x7f49c8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 -- 172.21.15.31:0/2646938011 shutdown_connections 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 --2- 172.21.15.31:0/2646938011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d810a890 0x7f49d810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 --2- 172.21.15.31:0/2646938011 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49d8101540 0x7f49d8101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 --2- 172.21.15.31:0/2646938011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d810a350 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 -- 172.21.15.31:0/2646938011 >> 172.21.15.31:0/2646938011 conn(0x7f49d80770a0 msgr2=0x7f49d80774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 -- 172.21.15.31:0/2646938011 shutdown_connections 2023-12-14T18:26:42.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.366+0000 7f49e028d700 1 -- 172.21.15.31:0/2646938011 wait complete. 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 Processor -- start 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 -- start start 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d8101540 0x7f49d80ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d80ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49d810a890 0x7f49d80ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a48b0 con 0x7f49d8101e50 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a4a20 con 0x7f49d810a890 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49e028d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81a4be0 con 0x7f49d8101540 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49dd828700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d80ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49dd828700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d80ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48098/0 (socket says 172.21.15.31:48098) 2023-12-14T18:26:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49dd828700 1 -- 172.21.15.31:0/3815951793 learned_addr learned my addr 172.21.15.31:0/3815951793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49de029700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d8101540 0x7f49d80ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49de82a700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49d810a890 0x7f49d80ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49dd828700 1 -- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d8101540 msgr2=0x7f49d80ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49dd828700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d8101540 0x7f49d80ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.367+0000 7f49dd828700 1 -- 172.21.15.31:0/3815951793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49d810a890 msgr2=0x7f49d80ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49dd828700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49d810a890 0x7f49d80ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49dd828700 1 -- 172.21.15.31:0/3815951793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c80095f0 con 0x7f49d8101e50 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49de029700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d8101540 0x7f49d80ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49de82a700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49d810a890 0x7f49d80ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49dd828700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d80ff6e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49c80098e0 tx=0x7f49c800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c801d070 con 0x7f49d8101e50 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8110310 con 0x7f49d8101e50 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49c8005510 con 0x7f49d8101e50 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8004c90 con 0x7f49d8101e50 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.368+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d8110180 con 0x7f49d8101e50 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.369+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49d80fd860 con 0x7f49d8101e50 2023-12-14T18:26:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.370+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f49c800fc10 con 0x7f49d8101e50 2023-12-14T18:26:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.371+0000 7f49cf7fe700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49c40a11e0 0x7f49c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.371+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f49c80cdaa0 con 0x7f49d8101e50 2023-12-14T18:26:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.371+0000 7f49de029700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49c40a11e0 0x7f49c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.371+0000 7f49cf7fe700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] conn(0x7f49c40abd50 0x7f49c40ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.371+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 --> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49c40ae850 con 0x7f49c40abd50 2023-12-14T18:26:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.371+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c80d22c0 con 0x7f49d8101e50 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.372+0000 7f49de029700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49c40a11e0 0x7f49c40a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f49d4005950 tx=0x7f49d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.372+0000 7f49de82a700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] conn(0x7f49c40abd50 0x7f49c40ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.372+0000 7f49de82a700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] conn(0x7f49c40abd50 0x7f49c40ae180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.373+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== osd.8 v2:172.21.15.204:6816/2332045081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f49c40ae850 con 0x7f49c40abd50 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.390+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 --> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49d81104a0 con 0x7f49c40abd50 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49cf7fe700 1 -- 172.21.15.31:0/3815951793 <== osd.8 v2:172.21.15.204:6816/2332045081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f49d81104a0 con 0x7f49c40abd50 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] conn(0x7f49c40abd50 msgr2=0x7f49c40ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] conn(0x7f49c40abd50 0x7f49c40ae180 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49c40a11e0 msgr2=0x7f49c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f49c40a11e0 0x7f49c40a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f49d4005950 tx=0x7f49d40058e0 comp rx=0 tx=0).stop 2023-12-14T18:26:42.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 msgr2=0x7f49d80ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 --2- 172.21.15.31:0/3815951793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d8101e50 0x7f49d80ff6e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f49c80098e0 tx=0x7f49c800f720 comp rx=0 tx=0).stop 2023-12-14T18:26:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49de82a700 1 -- 172.21.15.31:0/3815951793 reap_dead start 2023-12-14T18:26:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 shutdown_connections 2023-12-14T18:26:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 >> 172.21.15.31:0/3815951793 conn(0x7f49d80770a0 msgr2=0x7f49d80fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 shutdown_connections 2023-12-14T18:26:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:42.391+0000 7f49e028d700 1 -- 172.21.15.31:0/3815951793 wait complete. 2023-12-14T18:26:42.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:42 smithi204 ceph-mon[43866]: pgmap v252: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-12-14T18:26:42.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:42 smithi176 ceph-mon[43506]: pgmap v252: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-12-14T18:26:44.005 INFO:teuthology.orchestra.run.smithi031.stdout:55834574904 2023-12-14T18:26:44.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-14T18:26:44.400 INFO:teuthology.orchestra.run.smithi031.stdout:34359738426 2023-12-14T18:26:44.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-14T18:26:44.429 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:44 smithi031 ceph-mon[42085]: pgmap v253: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-12-14T18:26:44.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:44 smithi204 ceph-mon[43866]: pgmap v253: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-12-14T18:26:44.699 INFO:teuthology.orchestra.run.smithi031.stdout:236223201311 2023-12-14T18:26:44.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-14T18:26:44.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:44 smithi176 ceph-mon[43506]: pgmap v253: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2023-12-14T18:26:44.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 -- 172.21.15.31:0/1664778655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9808106d60 msgr2=0x7f98080feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 --2- 172.21.15.31:0/1664778655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9808106d60 0x7f98080feb10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009990 tx=0x7f97f8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 -- 172.21.15.31:0/1664778655 shutdown_connections 2023-12-14T18:26:44.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 --2- 172.21.15.31:0/1664778655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98080ff240 0x7f980810d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 --2- 172.21.15.31:0/1664778655 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98081063c0 0x7f9808106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 --2- 172.21.15.31:0/1664778655 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9808106d60 0x7f98080feb10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 -- 172.21.15.31:0/1664778655 >> 172.21.15.31:0/1664778655 conn(0x7f9808074930 msgr2=0x7f9808074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:44.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 -- 172.21.15.31:0/1664778655 shutdown_connections 2023-12-14T18:26:44.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.859+0000 7f980f7a9700 1 -- 172.21.15.31:0/1664778655 wait complete. 2023-12-14T18:26:44.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.860+0000 7f980f7a9700 1 Processor -- start 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.860+0000 7f980f7a9700 1 -- start start 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980f7a9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98080ff240 0x7f9808196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980f7a9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98081063c0 0x7f9808196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980f7a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f980819add0 0x7f980819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980f7a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819b7b0 con 0x7f980819add0 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980f7a9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819b920 con 0x7f98080ff240 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980f7a9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980819bae0 con 0x7f98081063c0 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f980819add0 0x7f980819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f980819add0 0x7f980819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60428/0 (socket says 172.21.15.31:60428) 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 -- 172.21.15.31:0/3185745650 learned_addr learned my addr 172.21.15.31:0/3185745650 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980cd44700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98081063c0 0x7f9808196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980d545700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98080ff240 0x7f9808196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 -- 172.21.15.31:0/3185745650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98081063c0 msgr2=0x7f9808196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98081063c0 0x7f9808196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 -- 172.21.15.31:0/3185745650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98080ff240 msgr2=0x7f9808196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98080ff240 0x7f9808196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.861+0000 7f980dd46700 1 -- 172.21.15.31:0/3185745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80095f0 con 0x7f980819add0 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.862+0000 7f980dd46700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f980819add0 0x7f980819b240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9800007ee0 tx=0x7f980000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.862+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800017070 con 0x7f980819add0 2023-12-14T18:26:44.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.862+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9808110370 con 0x7f980819add0 2023-12-14T18:26:44.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.862+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980000f040 con 0x7f980819add0 2023-12-14T18:26:44.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.862+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800005580 con 0x7f980819add0 2023-12-14T18:26:44.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.862+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98081101e0 con 0x7f980819add0 2023-12-14T18:26:44.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.863+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9808104a80 con 0x7f980819add0 2023-12-14T18:26:44.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.865+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f980000db20 con 0x7f980819add0 2023-12-14T18:26:44.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f97fe7fc700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f97f40a11e0 0x7f97f40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f98000ccb80 con 0x7f980819add0 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f980d545700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f97f40a11e0 0x7f97f40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f97fe7fc700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] conn(0x7f97f40abd50 0x7f97f40ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 --> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97f40ae850 con 0x7f97f40abd50 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98000ccef0 con 0x7f980819add0 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.866+0000 7f980cd44700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] conn(0x7f97f40abd50 0x7f97f40ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.867+0000 7f980d545700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f97f40a11e0 0x7f97f40a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9804010d00 tx=0x7f9804012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.867+0000 7f980cd44700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] conn(0x7f97f40abd50 0x7f97f40ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.868+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== osd.9 v2:172.21.15.31:6826/1064380347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f97f40ae850 con 0x7f97f40abd50 2023-12-14T18:26:44.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.885+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 --> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9808110500 con 0x7f97f40abd50 2023-12-14T18:26:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f97fe7fc700 1 -- 172.21.15.31:0/3185745650 <== osd.9 v2:172.21.15.31:6826/1064380347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9808110500 con 0x7f97f40abd50 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] conn(0x7f97f40abd50 msgr2=0x7f97f40ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] conn(0x7f97f40abd50 0x7f97f40ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f97f40a11e0 msgr2=0x7f97f40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f97f40a11e0 0x7f97f40a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9804010d00 tx=0x7f9804012040 comp rx=0 tx=0).stop 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f980819add0 msgr2=0x7f980819b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 --2- 172.21.15.31:0/3185745650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f980819add0 0x7f980819b240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9800007ee0 tx=0x7f980000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980dd46700 1 -- 172.21.15.31:0/3185745650 reap_dead start 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 shutdown_connections 2023-12-14T18:26:44.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 >> 172.21.15.31:0/3185745650 conn(0x7f9808074930 msgr2=0x7f9808104030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:44.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 shutdown_connections 2023-12-14T18:26:44.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.886+0000 7f980f7a9700 1 -- 172.21.15.31:0/3185745650 wait complete. 2023-12-14T18:26:44.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 -- 172.21.15.31:0/1448854417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 msgr2=0x7f6e0010ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 --2- 172.21.15.31:0/1448854417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e0010ccd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b0a0 tx=0x7f6df800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 -- 172.21.15.31:0/1448854417 shutdown_connections 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 --2- 172.21.15.31:0/1448854417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e0010ccd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 --2- 172.21.15.31:0/1448854417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e00101540 0x7f6e00101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 --2- 172.21.15.31:0/1448854417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101e50 0x7f6e0010a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 -- 172.21.15.31:0/1448854417 >> 172.21.15.31:0/1448854417 conn(0x7f6e000770a0 msgr2=0x7f6e000774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 -- 172.21.15.31:0/1448854417 shutdown_connections 2023-12-14T18:26:44.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.952+0000 7f6e04d53700 1 -- 172.21.15.31:0/1448854417 wait complete. 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 Processor -- start 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 -- start start 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e00101540 0x7f6e000ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101e50 0x7f6e000ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e000ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001a4980 con 0x7f6e0010a890 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001a4af0 con 0x7f6e00101540 2023-12-14T18:26:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.953+0000 7f6e04d53700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e001a4cb0 con 0x7f6e00101e50 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e000ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e000ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60458/0 (socket says 172.21.15.31:60458) 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 -- 172.21.15.31:0/653191654 learned_addr learned my addr 172.21.15.31:0/653191654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfdd9b700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101e50 0x7f6e000ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfe59c700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e00101540 0x7f6e000ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 -- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101e50 msgr2=0x7f6e000ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e00101e50 0x7f6e000ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 -- 172.21.15.31:0/653191654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e00101540 msgr2=0x7f6e000ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e00101540 0x7f6e000ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 -- 172.21.15.31:0/653191654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009d00 con 0x7f6e0010a890 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6dfed9d700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e000ffc60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b780 tx=0x7f6df8007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df800e040 con 0x7f6e0010a890 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6df801d470 con 0x7f6e0010a890 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e00110300 con 0x7f6e0010a890 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80052a0 con 0x7f6e0010a890 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.954+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e00110170 con 0x7f6e0010a890 2023-12-14T18:26:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.955+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e000fd720 con 0x7f6e0010a890 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.957+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6df801d5e0 con 0x7f6e0010a890 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6def7fe700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6de80a1260 0x7f6de80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6dfe59c700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6de80a1260 0x7f6de80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6df80ce8a0 con 0x7f6e0010a890 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6def7fe700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] conn(0x7f6de80abdd0 0x7f6de80ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 --> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6de80ae8d0 con 0x7f6de80abdd0 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6df80cec10 con 0x7f6e0010a890 2023-12-14T18:26:44.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6dfe59c700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6de80a1260 0x7f6de80a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009c60 tx=0x7f6df0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:44.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.958+0000 7f6dfdd9b700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] conn(0x7f6de80abdd0 0x7f6de80ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:44.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.959+0000 7f6dfdd9b700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] conn(0x7f6de80abdd0 0x7f6de80ae200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:44.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.960+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== osd.5 v2:172.21.15.204:6808/1402310939 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6de80ae8d0 con 0x7f6de80abdd0 2023-12-14T18:26:44.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.984+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 --> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e00110490 con 0x7f6de80abdd0 2023-12-14T18:26:44.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.984+0000 7f6def7fe700 1 -- 172.21.15.31:0/653191654 <== osd.5 v2:172.21.15.204:6808/1402310939 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6e00110490 con 0x7f6de80abdd0 2023-12-14T18:26:44.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] conn(0x7f6de80abdd0 msgr2=0x7f6de80ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] conn(0x7f6de80abdd0 0x7f6de80ae200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6de80a1260 msgr2=0x7f6de80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6de80a1260 0x7f6de80a3710 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009c60 tx=0x7f6df0009260 comp rx=0 tx=0).stop 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 msgr2=0x7f6e000ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 --2- 172.21.15.31:0/653191654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e0010a890 0x7f6e000ffc60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6df800b780 tx=0x7f6df8007990 comp rx=0 tx=0).stop 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6dfed9d700 1 -- 172.21.15.31:0/653191654 reap_dead start 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 shutdown_connections 2023-12-14T18:26:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.986+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 >> 172.21.15.31:0/653191654 conn(0x7f6e000770a0 msgr2=0x7f6e000fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.987+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 shutdown_connections 2023-12-14T18:26:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:44.987+0000 7f6e04d53700 1 -- 172.21.15.31:0/653191654 wait complete. 2023-12-14T18:26:45.413 INFO:teuthology.orchestra.run.smithi031.stdout:270582939674 2023-12-14T18:26:45.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-14T18:26:45.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 -- 172.21.15.31:0/2179513374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ff460 msgr2=0x7f391c0ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:45.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 --2- 172.21.15.31:0/2179513374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ff460 0x7f391c0ff830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f390c0099c0 tx=0x7f390c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 -- 172.21.15.31:0/2179513374 shutdown_connections 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 --2- 172.21.15.31:0/2179513374 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f391c104500 0x7f391c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 --2- 172.21.15.31:0/2179513374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ff460 0x7f391c0ff830 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 --2- 172.21.15.31:0/2179513374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c0ffe00 0x7f391c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.091+0000 7f3922934700 1 -- 172.21.15.31:0/2179513374 >> 172.21.15.31:0/2179513374 conn(0x7f391c0770a0 msgr2=0x7f391c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.092+0000 7f3922934700 1 -- 172.21.15.31:0/2179513374 shutdown_connections 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.092+0000 7f3922934700 1 -- 172.21.15.31:0/2179513374 wait complete. 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.092+0000 7f3922934700 1 Processor -- start 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.092+0000 7f3922934700 1 -- start start 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3922934700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c0ff460 0x7f391c10c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:45.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3922934700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ffe00 0x7f391c10cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3922934700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f391c104500 0x7f391c10d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3922934700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1a4860 con 0x7f391c0ffe00 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3922934700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1a49d0 con 0x7f391c104500 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3922934700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1a4b90 con 0x7f391c0ff460 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ffe00 0x7f391c10cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ffe00 0x7f391c10cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60480/0 (socket says 172.21.15.31:60480) 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 -- 172.21.15.31:0/3091271168 learned_addr learned my addr 172.21.15.31:0/3091271168 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3920ed1700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f391c104500 0x7f391c10d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391bfff700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c0ff460 0x7f391c10c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 -- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c0ff460 msgr2=0x7f391c10c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f391c0ff460 0x7f391c10c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 -- 172.21.15.31:0/3091271168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f391c104500 msgr2=0x7f391c10d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f391c104500 0x7f391c10d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 -- 172.21.15.31:0/3091271168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c0095f0 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f3920ed1700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f391c104500 0x7f391c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.093+0000 7f391b7fe700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ffe00 0x7f391c10cd10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f391000c9e0 tx=0x7f391000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.094+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910007aa0 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.094+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c1103b0 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.094+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3910007c00 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.094+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910018660 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.094+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c110220 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.095+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f391c0fd610 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.096+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f391000f450 con 0x7f391c0ffe00 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.096+0000 7f39197fa700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f39040a11e0 0x7f39040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:45.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f39100cd0c0 con 0x7f391c0ffe00 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f39197fa700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] conn(0x7f39040abd50 0x7f39040ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 --> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39040ae850 con 0x7f39040abd50 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39100cd4d0 con 0x7f391c0ffe00 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f391bfff700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f39040a11e0 0x7f39040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f3920ed1700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] conn(0x7f39040abd50 0x7f39040ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.097+0000 7f391bfff700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f39040a11e0 0x7f39040a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f390c009fd0 tx=0x7f390c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.098+0000 7f3920ed1700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] conn(0x7f39040abd50 0x7f39040ae180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.098+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== osd.2 v2:172.21.15.204:6800/4107722229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f39040ae850 con 0x7f39040abd50 2023-12-14T18:26:45.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.115+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 --> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f391c110540 con 0x7f39040abd50 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.116+0000 7f39197fa700 1 -- 172.21.15.31:0/3091271168 <== osd.2 v2:172.21.15.204:6800/4107722229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f391c110540 con 0x7f39040abd50 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] conn(0x7f39040abd50 msgr2=0x7f39040ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] conn(0x7f39040abd50 0x7f39040ae180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f39040a11e0 msgr2=0x7f39040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f39040a11e0 0x7f39040a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f390c009fd0 tx=0x7f390c01a040 comp rx=0 tx=0).stop 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ffe00 msgr2=0x7f391c10cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 --2- 172.21.15.31:0/3091271168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f391c0ffe00 0x7f391c10cd10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f391000c9e0 tx=0x7f391000cda0 comp rx=0 tx=0).stop 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3920ed1700 1 -- 172.21.15.31:0/3091271168 reap_dead start 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 shutdown_connections 2023-12-14T18:26:45.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 >> 172.21.15.31:0/3091271168 conn(0x7f391c0770a0 msgr2=0x7f391c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:45.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 shutdown_connections 2023-12-14T18:26:45.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:45.117+0000 7f3922934700 1 -- 172.21.15.31:0/3091271168 wait complete. 2023-12-14T18:26:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.106+0000 7f7e1e890700 1 -- 172.21.15.31:0/4196000657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18100a50 msgr2=0x7f7e18100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.106+0000 7f7e1e890700 1 --2- 172.21.15.31:0/4196000657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18100a50 0x7f7e18100ec0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009990 tx=0x7f7e0c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 -- 172.21.15.31:0/4196000657 shutdown_connections 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 --2- 172.21.15.31:0/4196000657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e18101400 0x7f7e1810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 --2- 172.21.15.31:0/4196000657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e18106380 0x7f7e18106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 --2- 172.21.15.31:0/4196000657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18100a50 0x7f7e18100ec0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 -- 172.21.15.31:0/4196000657 >> 172.21.15.31:0/4196000657 conn(0x7f7e18077530 msgr2=0x7f7e18077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 -- 172.21.15.31:0/4196000657 shutdown_connections 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 -- 172.21.15.31:0/4196000657 wait complete. 2023-12-14T18:26:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 Processor -- start 2023-12-14T18:26:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.107+0000 7f7e1e890700 1 -- start start 2023-12-14T18:26:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1e890700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e18100a50 0x7f7e181985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1e890700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18101400 0x7f7e18198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1e890700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e18106380 0x7f7e1819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1e890700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1819d590 con 0x7f7e18101400 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1e890700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1819d700 con 0x7f7e18100a50 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1e890700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1819d8c0 con 0x7f7e18106380 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18101400 0x7f7e18198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18101400 0x7f7e18198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60492/0 (socket says 172.21.15.31:60492) 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 -- 172.21.15.31:0/2615696232 learned_addr learned my addr 172.21.15.31:0/2615696232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e17fff700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e18100a50 0x7f7e181985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e1ce2d700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e18106380 0x7f7e1819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 -- 172.21.15.31:0/2615696232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e18106380 msgr2=0x7f7e1819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e18106380 0x7f7e1819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 -- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e18100a50 msgr2=0x7f7e181985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.108+0000 7f7e177fe700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e18100a50 0x7f7e181985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e177fe700 1 -- 172.21.15.31:0/2615696232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0095f0 con 0x7f7e18101400 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e177fe700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18101400 0x7f7e18198b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0095c0 tx=0x7f7e0c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01d070 con 0x7f7e18101400 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e18110360 con 0x7f7e18101400 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0c0054f0 con 0x7f7e18101400 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c004a70 con 0x7f7e18101400 2023-12-14T18:26:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.109+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e181101d0 con 0x7f7e18101400 2023-12-14T18:26:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.111+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e180611b0 con 0x7f7e18101400 2023-12-14T18:26:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.111+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7e0c004bd0 con 0x7f7e18101400 2023-12-14T18:26:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.112+0000 7f7e157fa700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e000a11e0 0x7f7e000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e17fff700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e000a11e0 0x7f7e000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7e0c0cda00 con 0x7f7e18101400 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e157fa700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] conn(0x7f7e000abd50 0x7f7e000ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 --> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e000ae850 con 0x7f7e000abd50 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c0d22c0 con 0x7f7e18101400 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e17fff700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e000a11e0 0x7f7e000a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005950 tx=0x7f7e0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.113+0000 7f7e1ce2d700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] conn(0x7f7e000abd50 0x7f7e000ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.114+0000 7f7e1ce2d700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] conn(0x7f7e000abd50 0x7f7e000ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.115+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== osd.7 v2:172.21.15.176:6816/800429312 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7e000ae850 con 0x7f7e000abd50 2023-12-14T18:26:46.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.135+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 --> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7e181104f0 con 0x7f7e000abd50 2023-12-14T18:26:46.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.136+0000 7f7e157fa700 1 -- 172.21.15.31:0/2615696232 <== osd.7 v2:172.21.15.176:6816/800429312 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7e181104f0 con 0x7f7e000abd50 2023-12-14T18:26:46.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] conn(0x7f7e000abd50 msgr2=0x7f7e000ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] conn(0x7f7e000abd50 0x7f7e000ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e000a11e0 msgr2=0x7f7e000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7e000a11e0 0x7f7e000a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7e08005950 tx=0x7f7e0800b410 comp rx=0 tx=0).stop 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18101400 msgr2=0x7f7e18198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 --2- 172.21.15.31:0/2615696232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e18101400 0x7f7e18198b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0095c0 tx=0x7f7e0c00f710 comp rx=0 tx=0).stop 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1ce2d700 1 -- 172.21.15.31:0/2615696232 reap_dead start 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 shutdown_connections 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 >> 172.21.15.31:0/2615696232 conn(0x7f7e18077530 msgr2=0x7f7e1810b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.137+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 shutdown_connections 2023-12-14T18:26:46.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.138+0000 7f7e1e890700 1 -- 172.21.15.31:0/2615696232 wait complete. 2023-12-14T18:26:46.146 INFO:teuthology.orchestra.run.smithi031.stdout:154618822698 2023-12-14T18:26:46.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-14T18:26:46.397 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 -- 172.21.15.31:0/1643124542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd00ff470 msgr2=0x7fedd00ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 --2- 172.21.15.31:0/1643124542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd00ff470 0x7fedd00ff840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009990 tx=0x7fedcc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 -- 172.21.15.31:0/1643124542 shutdown_connections 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 --2- 172.21.15.31:0/1643124542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedd0104510 0x7fedd010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 --2- 172.21.15.31:0/1643124542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd00ff470 0x7fedd00ff840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 --2- 172.21.15.31:0/1643124542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedd00ffe10 0x7fedd0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 -- 172.21.15.31:0/1643124542 >> 172.21.15.31:0/1643124542 conn(0x7fedd00770d0 msgr2=0x7fedd00774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 -- 172.21.15.31:0/1643124542 shutdown_connections 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.391+0000 7fedd7499700 1 -- 172.21.15.31:0/1643124542 wait complete. 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.392+0000 7fedd7499700 1 Processor -- start 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.392+0000 7fedd7499700 1 -- start start 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd7499700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedd00ff470 0x7fedd0196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd7499700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedd00ffe10 0x7fedd0196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd7499700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd0104510 0x7fedd019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd7499700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd019b3e0 con 0x7fedd0104510 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd7499700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd019b550 con 0x7fedd00ff470 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd7499700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd019b710 con 0x7fedd00ffe10 2023-12-14T18:26:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5a36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd0104510 0x7fedd019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5235700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedd00ff470 0x7fedd0196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5a36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd0104510 0x7fedd019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60504/0 (socket says 172.21.15.31:60504) 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5a36700 1 -- 172.21.15.31:0/2596030895 learned_addr learned my addr 172.21.15.31:0/2596030895 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd4a34700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedd00ffe10 0x7fedd0196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5235700 1 -- 172.21.15.31:0/2596030895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedd00ffe10 msgr2=0x7fedd0196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5235700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedd00ffe10 0x7fedd0196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5235700 1 -- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd0104510 msgr2=0x7fedd019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5235700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd0104510 0x7fedd019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5235700 1 -- 172.21.15.31:0/2596030895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc0009520 con 0x7fedd00ff470 2023-12-14T18:26:46.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.393+0000 7fedd5a36700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedd0104510 0x7fedd019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.394+0000 7fedd5235700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedd00ff470 0x7fedd0196420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fedcc000c00 tx=0x7fedcc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.394+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01d070 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.394+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc0095f0 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.394+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedcc022470 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.394+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc017650 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.394+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd01103e0 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.395+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedd00fd740 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.397+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fedcc0225e0 con 0x7fedd00ff470 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.398+0000 7fedc67fc700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fedbc0a11e0 0x7fedbc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.398+0000 7fedd4a34700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fedbc0a11e0 0x7fedbc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.398+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fedcc0cdfe0 con 0x7fedd00ff470 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.398+0000 7fedc67fc700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] conn(0x7fedbc0abd50 0x7fedbc0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.398+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 --> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedbc0ae850 con 0x7fedbc0abd50 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.398+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedcc0ce350 con 0x7fedd00ff470 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.399+0000 7fedd4a34700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fedbc0a11e0 0x7fedbc0a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedd0197a40 tx=0x7fedc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.399+0000 7fedd5a36700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] conn(0x7fedbc0abd50 0x7fedbc0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.399+0000 7fedd5a36700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] conn(0x7fedbc0abd50 0x7fedbc0ae180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.400+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== osd.6 v2:172.21.15.31:6818/2676550981 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fedbc0ae850 con 0x7fedbc0abd50 2023-12-14T18:26:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.418+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 --> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fedd0110250 con 0x7fedbc0abd50 2023-12-14T18:26:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedc67fc700 1 -- 172.21.15.31:0/2596030895 <== osd.6 v2:172.21.15.31:6818/2676550981 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fedd0110250 con 0x7fedbc0abd50 2023-12-14T18:26:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] conn(0x7fedbc0abd50 msgr2=0x7fedbc0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedd7499700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] conn(0x7fedbc0abd50 0x7fedbc0ae180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fedbc0a11e0 msgr2=0x7fedbc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedd7499700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fedbc0a11e0 0x7fedbc0a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedd0197a40 tx=0x7fedc0009260 comp rx=0 tx=0).stop 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedd00ff470 msgr2=0x7fedd0196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.419+0000 7fedd7499700 1 --2- 172.21.15.31:0/2596030895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedd00ff470 0x7fedd0196420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fedcc000c00 tx=0x7fedcc009580 comp rx=0 tx=0).stop 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.420+0000 7fedd5a36700 1 -- 172.21.15.31:0/2596030895 reap_dead start 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.420+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 shutdown_connections 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.420+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 >> 172.21.15.31:0/2596030895 conn(0x7fedd00770d0 msgr2=0x7fedd00fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.421+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 shutdown_connections 2023-12-14T18:26:46.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:46.421+0000 7fedd7499700 1 -- 172.21.15.31:0/2596030895 wait complete. 2023-12-14T18:26:46.684 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:46 smithi031 ceph-mon[42085]: pgmap v254: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-12-14T18:26:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:46 smithi204 ceph-mon[43866]: pgmap v254: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-12-14T18:26:46.879 INFO:teuthology.orchestra.run.smithi031.stdout:206158430242 2023-12-14T18:26:46.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-14T18:26:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:46 smithi176 ceph-mon[43506]: pgmap v254: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-12-14T18:26:47.077 INFO:teuthology.orchestra.run.smithi031.stdout:184683593767 2023-12-14T18:26:47.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-14T18:26:47.175 INFO:teuthology.orchestra.run.smithi031.stdout:77309411381 2023-12-14T18:26:47.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-14T18:26:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-12-14T18:26:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:47 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2023-12-14T18:26:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-12-14T18:26:47.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:47 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2023-12-14T18:26:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:47 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-12-14T18:26:48.125 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:48.125 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:48.125 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:48.126 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:48.131 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:48.132 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.559+0000 7fde848a4700 1 -- 172.21.15.31:0/1715267604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde80104500 msgr2=0x7fde8010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.559+0000 7fde848a4700 1 --2- 172.21.15.31:0/1715267604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde80104500 0x7fde8010c080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fde7800b0d0 tx=0x7fde7800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.560+0000 7fde848a4700 1 -- 172.21.15.31:0/1715267604 shutdown_connections 2023-12-14T18:26:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.560+0000 7fde848a4700 1 --2- 172.21.15.31:0/1715267604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde80104500 0x7fde8010c080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.560+0000 7fde848a4700 1 --2- 172.21.15.31:0/1715267604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde800ff460 0x7fde800ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.560+0000 7fde848a4700 1 --2- 172.21.15.31:0/1715267604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde800ffe00 0x7fde80103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.560+0000 7fde848a4700 1 -- 172.21.15.31:0/1715267604 >> 172.21.15.31:0/1715267604 conn(0x7fde800770a0 msgr2=0x7fde800774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.561+0000 7fde848a4700 1 -- 172.21.15.31:0/1715267604 shutdown_connections 2023-12-14T18:26:48.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.562+0000 7fde848a4700 1 -- 172.21.15.31:0/1715267604 wait complete. 2023-12-14T18:26:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.563+0000 7fde848a4700 1 Processor -- start 2023-12-14T18:26:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.563+0000 7fde848a4700 1 -- start start 2023-12-14T18:26:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde848a4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde800ff460 0x7fde8010c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde848a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde800ffe00 0x7fde8010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde848a4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 0x7fde8010d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde848a4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde801a8e00 con 0x7fde800ffe00 2023-12-14T18:26:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde848a4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde801a8f40 con 0x7fde80104500 2023-12-14T18:26:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde848a4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde801a90b0 con 0x7fde800ff460 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde7ed9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 0x7fde8010d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde7ed9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 0x7fde8010d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35280/0 (socket says 172.21.15.31:35280) 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde7ed9d700 1 -- 172.21.15.31:0/2179845031 learned_addr learned my addr 172.21.15.31:0/2179845031 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.564+0000 7fde7dd9b700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde800ffe00 0x7fde8010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde7ed9d700 1 -- 172.21.15.31:0/2179845031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde800ff460 msgr2=0x7fde8010c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde7ed9d700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde800ff460 0x7fde8010c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde7ed9d700 1 -- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde800ffe00 msgr2=0x7fde8010cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde7ed9d700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde800ffe00 0x7fde8010cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde7ed9d700 1 -- 172.21.15.31:0/2179845031 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde78009d00 con 0x7fde80104500 2023-12-14T18:26:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde7ed9d700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 0x7fde8010d270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde78009bb0 tx=0x7fde78007af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800e040 con 0x7fde80104500 2023-12-14T18:26:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80110420 con 0x7fde80104500 2023-12-14T18:26:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.565+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80110290 con 0x7fde80104500 2023-12-14T18:26:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.566+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde7801dd50 con 0x7fde80104500 2023-12-14T18:26:48.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.566+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7801bd80 con 0x7fde80104500 2023-12-14T18:26:48.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.567+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde600052f0 con 0x7fde80104500 2023-12-14T18:26:48.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.569+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fde7801b450 con 0x7fde80104500 2023-12-14T18:26:48.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.569+0000 7fde6f7fe700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fde680a1260 0x7fde680a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:48.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.570+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fde780cee50 con 0x7fde80104500 2023-12-14T18:26:48.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.570+0000 7fde7e59c700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fde680a1260 0x7fde680a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:48.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.570+0000 7fde7e59c700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fde680a1260 0x7fde680a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde70005fd0 tx=0x7fde70005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:48.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.574+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fde780982e0 con 0x7fde80104500 2023-12-14T18:26:48.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: pgmap v255: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-14T18:26:48.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-14T18:26:48.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-12-14T18:26:48.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-12-14T18:26:48.641 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-12-14T18:26:48.642 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:48 smithi031 ceph-mon[42085]: osdmap e86: 12 total, 12 up, 12 in 2023-12-14T18:26:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.800+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fde60005160 con 0x7fde80104500 2023-12-14T18:26:48.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.801+0000 7fde6f7fe700 1 -- 172.21.15.31:0/2179845031 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde78021020 con 0x7fde80104500 2023-12-14T18:26:48.804 INFO:teuthology.orchestra.run.smithi031.stdout:103079215153 2023-12-14T18:26:48.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fde680a1260 msgr2=0x7fde680a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:48.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fde680a1260 0x7fde680a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde70005fd0 tx=0x7fde70005f60 comp rx=0 tx=0).stop 2023-12-14T18:26:48.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 msgr2=0x7fde8010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:48.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 0x7fde8010d270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde78009bb0 tx=0x7fde78007af0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 shutdown_connections 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde80104500 0x7fde8010d270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fde680a1260 0x7fde680a3710 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde70005fd0 tx=0x7fde70005f60 comp rx=0 tx=0).stop 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde800ff460 0x7fde8010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 --2- 172.21.15.31:0/2179845031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde800ffe00 0x7fde8010cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 >> 172.21.15.31:0/2179845031 conn(0x7fde800770a0 msgr2=0x7fde800fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 shutdown_connections 2023-12-14T18:26:48.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:48.805+0000 7fde848a4700 1 -- 172.21.15.31:0/2179845031 wait complete. 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: pgmap v255: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-12-14T18:26:48.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-12-14T18:26:48.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:48 smithi204 ceph-mon[43866]: osdmap e86: 12 total, 12 up, 12 in 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: pgmap v255: 129 pgs: 129 active+clean; 637 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 B/s wr, 6 op/s 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 10]}]': finished 2023-12-14T18:26:48.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2023-12-14T18:26:48.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-12-14T18:26:48.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:48 smithi176 ceph-mon[43506]: osdmap e86: 12 total, 12 up, 12 in 2023-12-14T18:26:49.408 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:49.409 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:49 smithi204 ceph-mon[43866]: osdmap e87: 12 total, 12 up, 12 in 2023-12-14T18:26:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:49 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2179845031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-14T18:26:49.865 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:49 smithi031 ceph-mon[42085]: osdmap e87: 12 total, 12 up, 12 in 2023-12-14T18:26:49.865 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:49 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2179845031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-14T18:26:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:49 smithi176 ceph-mon[43506]: osdmap e87: 12 total, 12 up, 12 in 2023-12-14T18:26:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:49 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2179845031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-14T18:26:50.822 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:50 smithi031 ceph-mon[42085]: pgmap v258: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2023-12-14T18:26:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:50 smithi204 ceph-mon[43866]: pgmap v258: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2023-12-14T18:26:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:50 smithi176 ceph-mon[43506]: pgmap v258: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2023-12-14T18:26:51.152 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:51.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.398+0000 7fc72fb4d700 1 -- 172.21.15.31:0/183069353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101e50 msgr2=0x7fc72810a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.398+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/183069353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101e50 0x7fc72810a350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc718009990 tx=0x7fc718009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 -- 172.21.15.31:0/183069353 shutdown_connections 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/183069353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72810a890 0x7fc72810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/183069353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101540 0x7fc728101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/183069353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101e50 0x7fc72810a350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 -- 172.21.15.31:0/183069353 >> 172.21.15.31:0/183069353 conn(0x7fc7280770a0 msgr2=0x7fc7280774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 -- 172.21.15.31:0/183069353 shutdown_connections 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 -- 172.21.15.31:0/183069353 wait complete. 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 Processor -- start 2023-12-14T18:26:51.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.399+0000 7fc72fb4d700 1 -- start start 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 0x7fc7281963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101e50 0x7fc7281968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72810a890 0x7fc72819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72819b370 con 0x7fc728101540 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72819b4e0 con 0x7fc728101e50 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72819b6a0 con 0x7fc72810a890 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 0x7fc7281963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 0x7fc7281963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60544/0 (socket says 172.21.15.31:60544) 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 -- 172.21.15.31:0/4131692709 learned_addr learned my addr 172.21.15.31:0/4131692709 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:51.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d0e8700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101e50 0x7fc7281968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 -- 172.21.15.31:0/4131692709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72810a890 msgr2=0x7fc72819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72810a890 0x7fc72819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 -- 172.21.15.31:0/4131692709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101e50 msgr2=0x7fc7281968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101e50 0x7fc7281968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 -- 172.21.15.31:0/4131692709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180095f0 con 0x7fc728101540 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d0e8700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101e50 0x7fc7281968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72d8e9700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 0x7fc7281963b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d980 tx=0x7fc72400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72400bb60 con 0x7fc728101540 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc724007640 con 0x7fc728101540 2023-12-14T18:26:51.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728110360 con 0x7fc728101540 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724004e90 con 0x7fc728101540 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.400+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7281101d0 con 0x7fc728101540 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.402+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7281104f0 con 0x7fc728101540 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.403+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc72400bcc0 con 0x7fc728101540 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.403+0000 7fc71effd700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc7140a1260 0x7fc7140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.403+0000 7fc72d0e8700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc7140a1260 0x7fc7140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.403+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7240ccbc0 con 0x7fc728101540 2023-12-14T18:26:51.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.404+0000 7fc72d0e8700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc7140a1260 0x7fc7140a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b5c0 tx=0x7fc718005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:51.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.408+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc724095ea0 con 0x7fc728101540 2023-12-14T18:26:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.642+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc728066ea0 con 0x7fc728101540 2023-12-14T18:26:51.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.643+0000 7fc71effd700 1 -- 172.21.15.31:0/4131692709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc724016020 con 0x7fc728101540 2023-12-14T18:26:51.645 INFO:teuthology.orchestra.run.smithi031.stdout:133143986223 2023-12-14T18:26:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc7140a1260 msgr2=0x7fc7140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc7140a1260 0x7fc7140a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b5c0 tx=0x7fc718005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 msgr2=0x7fc7281963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 0x7fc7281963b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d980 tx=0x7fc72400dd40 comp rx=0 tx=0).stop 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 shutdown_connections 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc7140a1260 0x7fc7140a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc72810a890 0x7fc72819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc728101540 0x7fc7281963b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 --2- 172.21.15.31:0/4131692709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc728101e50 0x7fc7281968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 >> 172.21.15.31:0/4131692709 conn(0x7fc7280770a0 msgr2=0x7fc7280fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 shutdown_connections 2023-12-14T18:26:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:51.645+0000 7fc72fb4d700 1 -- 172.21.15.31:0/4131692709 wait complete. 2023-12-14T18:26:51.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215151 got 103079215153 for osd.3 2023-12-14T18:26:51.906 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:52.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 -- 172.21.15.31:0/1441985845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 msgr2=0x7fc60010d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 --2- 172.21.15.31:0/1441985845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc60010d160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800b0d0 tx=0x7fc5f800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 -- 172.21.15.31:0/1441985845 shutdown_connections 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 --2- 172.21.15.31:0/1441985845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc60010d160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 --2- 172.21.15.31:0/1441985845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc600100540 0x7fc600100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 --2- 172.21.15.31:0/1441985845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc600100ee0 0x7fc600104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 -- 172.21.15.31:0/1441985845 >> 172.21.15.31:0/1441985845 conn(0x7fc6000fbdc0 msgr2=0x7fc6000fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 -- 172.21.15.31:0/1441985845 shutdown_connections 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 -- 172.21.15.31:0/1441985845 wait complete. 2023-12-14T18:26:52.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.444+0000 7fc60612f700 1 Processor -- start 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 -- start start 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc600100540 0x7fc60019e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc600100ee0 0x7fc60019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc6001985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600198c10 con 0x7fc6001055e0 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600198d80 con 0x7fc600100540 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc60612f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600198f40 con 0x7fc600100ee0 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc5fffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc6001985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc5ff7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc600100540 0x7fc60019e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc5fffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc6001985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60562/0 (socket says 172.21.15.31:60562) 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc5fffff700 1 -- 172.21.15.31:0/1503761902 learned_addr learned my addr 172.21.15.31:0/1503761902 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.445+0000 7fc5feffd700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc600100ee0 0x7fc60019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fffff700 1 -- 172.21.15.31:0/1503761902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc600100ee0 msgr2=0x7fc60019ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fffff700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc600100ee0 0x7fc60019ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fffff700 1 -- 172.21.15.31:0/1503761902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc600100540 msgr2=0x7fc60019e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fffff700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc600100540 0x7fc60019e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fffff700 1 -- 172.21.15.31:0/1503761902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8009d00 con 0x7fc6001055e0 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fffff700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc6001985a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007890 tx=0x7fc5f80078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800e040 con 0x7fc6001055e0 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc600110370 con 0x7fc6001055e0 2023-12-14T18:26:52.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5f800bd30 con 0x7fc6001055e0 2023-12-14T18:26:52.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f801d470 con 0x7fc6001055e0 2023-12-14T18:26:52.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.446+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6001101e0 con 0x7fc6001055e0 2023-12-14T18:26:52.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.447+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600110500 con 0x7fc6001055e0 2023-12-14T18:26:52.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.448+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc5f8004a50 con 0x7fc6001055e0 2023-12-14T18:26:52.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.449+0000 7fc5fcff9700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5e80a11e0 0x7fc5e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.449+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5f80cdbf0 con 0x7fc6001055e0 2023-12-14T18:26:52.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.449+0000 7fc5ff7fe700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5e80a11e0 0x7fc5e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.450+0000 7fc5ff7fe700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5e80a11e0 0x7fc5e80a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009c60 tx=0x7fc5f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.454+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc5f8096f00 con 0x7fc6001055e0 2023-12-14T18:26:52.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.674+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fc600066ea0 con 0x7fc6001055e0 2023-12-14T18:26:52.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.674+0000 7fc5fcff9700 1 -- 172.21.15.31:0/1503761902 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fc5f809a520 con 0x7fc6001055e0 2023-12-14T18:26:52.677 INFO:teuthology.orchestra.run.smithi031.stdout:326417514517 2023-12-14T18:26:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5e80a11e0 msgr2=0x7fc5e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5e80a11e0 0x7fc5e80a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009c60 tx=0x7fc5f0009260 comp rx=0 tx=0).stop 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 msgr2=0x7fc6001985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc6001985a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007890 tx=0x7fc5f80078c0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 shutdown_connections 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6001055e0 0x7fc6001985a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc600100540 0x7fc60019e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5e80a11e0 0x7fc5e80a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 --2- 172.21.15.31:0/1503761902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc600100ee0 0x7fc60019ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 >> 172.21.15.31:0/1503761902 conn(0x7fc6000fbdc0 msgr2=0x7fc60010b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 shutdown_connections 2023-12-14T18:26:52.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.677+0000 7fc60612f700 1 -- 172.21.15.31:0/1503761902 wait complete. 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/23354615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101500 msgr2=0x7f1fe41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/23354615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101500 0x7f1fe41018d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009990 tx=0x7f1fd4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/23354615 shutdown_connections 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/23354615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fe410a850 0x7f1fe410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/23354615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101500 0x7f1fe41018d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/23354615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe4101e10 0x7f1fe410a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/23354615 >> 172.21.15.31:0/23354615 conn(0x7f1fe40facc0 msgr2=0x7f1fe40fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/23354615 shutdown_connections 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.592+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/23354615 wait complete. 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 Processor -- start 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 -- start start 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fe4101500 0x7f1fe40ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101e10 0x7f1fe40ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe410a850 0x7f1fe40ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41a4840 con 0x7f1fe4101e10 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41a49b0 con 0x7f1fe410a850 2023-12-14T18:26:52.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.593+0000 7f1fe9c9a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe41a4b70 con 0x7f1fe4101500 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe410a850 0x7f1fe40ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe410a850 0x7f1fe40ffc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35312/0 (socket says 172.21.15.31:35312) 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe3fff700 1 -- 172.21.15.31:0/3942270459 learned_addr learned my addr 172.21.15.31:0/3942270459 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101e10 0x7f1fe40ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 -- 172.21.15.31:0/3942270459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fe4101500 msgr2=0x7f1fe40ff160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fe4101500 0x7f1fe40ff160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 -- 172.21.15.31:0/3942270459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe410a850 msgr2=0x7f1fe40ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe410a850 0x7f1fe40ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 -- 172.21.15.31:0/3942270459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd40095f0 con 0x7f1fe4101e10 2023-12-14T18:26:52.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.594+0000 7f1fe2ffd700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101e10 0x7f1fe40ff6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800e9f0 tx=0x7f1fd800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.595+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd800eee0 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.595+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe4110330 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.595+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe41101a0 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.595+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fd800cbd0 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.595+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8018680 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.596+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd0005320 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.598+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1fd800cd40 con 0x7f1fe4101e10 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.599+0000 7f1fe0ff9700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1fcc0a11e0 0x7f1fcc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.599+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1fd8014070 con 0x7f1fe4101e10 2023-12-14T18:26:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.600+0000 7f1fe37fe700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1fcc0a11e0 0x7f1fcc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.601+0000 7f1fe37fe700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1fcc0a11e0 0x7f1fcc0a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400b2e0 tx=0x7f1fd4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.602+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1fd80963e0 con 0x7f1fe4101e10 2023-12-14T18:26:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.817+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f1fd0005190 con 0x7f1fe4101e10 2023-12-14T18:26:52.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.817+0000 7f1fe0ff9700 1 -- 172.21.15.31:0/3942270459 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1fd8026090 con 0x7f1fe4101e10 2023-12-14T18:26:52.820 INFO:teuthology.orchestra.run.smithi031.stdout:292057776153 2023-12-14T18:26:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1fcc0a11e0 msgr2=0x7f1fcc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1fcc0a11e0 0x7f1fcc0a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400b2e0 tx=0x7f1fd4000bc0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101e10 msgr2=0x7f1fe40ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101e10 0x7f1fe40ff6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800e9f0 tx=0x7f1fd800ed00 comp rx=0 tx=0).stop 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 shutdown_connections 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1fe410a850 0x7f1fe40ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1fcc0a11e0 0x7f1fcc0a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1fe4101500 0x7f1fe40ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 --2- 172.21.15.31:0/3942270459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe4101e10 0x7f1fe40ff6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 >> 172.21.15.31:0/3942270459 conn(0x7f1fe40facc0 msgr2=0x7f1fe410c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 shutdown_connections 2023-12-14T18:26:52.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:52.820+0000 7f1fe9c9a700 1 -- 172.21.15.31:0/3942270459 wait complete. 2023-12-14T18:26:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:52 smithi204 ceph-mon[43866]: pgmap v259: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:52 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4131692709' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-14T18:26:52.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:52 smithi031 ceph-mon[42085]: pgmap v259: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:52.848 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:52 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4131692709' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-14T18:26:52.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:52 smithi176 ceph-mon[43506]: pgmap v259: 129 pgs: 2 peering, 127 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 6 op/s; 72 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:52.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:52 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4131692709' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-14T18:26:53.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:53 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1503761902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-14T18:26:53.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:53 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3942270459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-14T18:26:53.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:53 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:53 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1503761902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-14T18:26:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:53 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3942270459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-14T18:26:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:53 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:53.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 -- 172.21.15.31:0/1443288427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 msgr2=0x7fb1d40ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:53.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 --2- 172.21.15.31:0/1443288427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d40ff4a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009990 tx=0x7fb1d0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 -- 172.21.15.31:0/1443288427 shutdown_connections 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 --2- 172.21.15.31:0/1443288427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d410cc80 0x7fb1d410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 --2- 172.21.15.31:0/1443288427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d40ff4a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 --2- 172.21.15.31:0/1443288427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d40ff9e0 0x7fb1d410c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 -- 172.21.15.31:0/1443288427 >> 172.21.15.31:0/1443288427 conn(0x7fb1d40770d0 msgr2=0x7fb1d40774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 -- 172.21.15.31:0/1443288427 shutdown_connections 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.855+0000 7fb1db6de700 1 -- 172.21.15.31:0/1443288427 wait complete. 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 Processor -- start 2023-12-14T18:26:53.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 -- start start 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d40ff9e0 0x7fb1d4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d410cc80 0x7fb1d419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d419d580 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d419d6f0 con 0x7fb1d410cc80 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.856+0000 7fb1db6de700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d419d8b0 con 0x7fb1d40ff9e0 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d8c79700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d40ff9e0 0x7fb1d4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:53.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60606/0 (socket says 172.21.15.31:60606) 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 -- 172.21.15.31:0/842859526 learned_addr learned my addr 172.21.15.31:0/842859526 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d9c7b700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d410cc80 0x7fb1d419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 -- 172.21.15.31:0/842859526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d40ff9e0 msgr2=0x7fb1d4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d40ff9e0 0x7fb1d4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 -- 172.21.15.31:0/842859526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d410cc80 msgr2=0x7fb1d419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d410cc80 0x7fb1d419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 -- 172.21.15.31:0/842859526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00095f0 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d8c79700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d40ff9e0 0x7fb1d4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:53.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1d947a700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d41985c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0000c00 tx=0x7fb1d0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d001d070 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d0021410 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d4112560 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0017650 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.857+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d41123d0 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.858+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d41126f0 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.860+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb1d0021580 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.860+0000 7fb1ca7fc700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb1c00a11e0 0x7fb1c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.860+0000 7fb1d8c79700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb1c00a11e0 0x7fb1c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:53.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.860+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb1d00ce030 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.861+0000 7fb1d8c79700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb1c00a11e0 0x7fb1c00a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4199be0 tx=0x7fb1c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:53.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:53.865+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb1d00973a0 con 0x7fb1d40ff0d0 2023-12-14T18:26:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:53 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1503761902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-14T18:26:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:53 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3942270459' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-14T18:26:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:53 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.089+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fb1d4066ea0 con 0x7fb1d40ff0d0 2023-12-14T18:26:54.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.090+0000 7fb1ca7fc700 1 -- 172.21.15.31:0/842859526 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1d009a9c0 con 0x7fb1d40ff0d0 2023-12-14T18:26:54.092 INFO:teuthology.orchestra.run.smithi031.stdout:236223201312 2023-12-14T18:26:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb1c00a11e0 msgr2=0x7fb1c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb1c00a11e0 0x7fb1c00a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4199be0 tx=0x7fb1c4009260 comp rx=0 tx=0).stop 2023-12-14T18:26:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 msgr2=0x7fb1d41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d41985c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0000c00 tx=0x7fb1d0004790 comp rx=0 tx=0).stop 2023-12-14T18:26:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 shutdown_connections 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1d410cc80 0x7fb1d419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1d40ff0d0 0x7fb1d41985c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb1c00a11e0 0x7fb1c00a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 --2- 172.21.15.31:0/842859526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1d40ff9e0 0x7fb1d4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 >> 172.21.15.31:0/842859526 conn(0x7fb1d40770d0 msgr2=0x7fb1d410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 shutdown_connections 2023-12-14T18:26:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.092+0000 7fb1db6de700 1 -- 172.21.15.31:0/842859526 wait complete. 2023-12-14T18:26:54.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776150 got 292057776153 for osd.10 2023-12-14T18:26:54.143 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:54.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 -- 172.21.15.31:0/3052241369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ff460 msgr2=0x7fbe480ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/3052241369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ff460 0x7fbe480ff830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe380099c0 tx=0x7fbe38009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 -- 172.21.15.31:0/3052241369 shutdown_connections 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/3052241369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe48104500 0x7fbe4810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/3052241369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ff460 0x7fbe480ff830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/3052241369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe480ffe00 0x7fbe48103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 -- 172.21.15.31:0/3052241369 >> 172.21.15.31:0/3052241369 conn(0x7fbe480770a0 msgr2=0x7fbe480774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 -- 172.21.15.31:0/3052241369 shutdown_connections 2023-12-14T18:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.599+0000 7fbe4d07b700 1 -- 172.21.15.31:0/3052241369 wait complete. 2023-12-14T18:26:54.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 Processor -- start 2023-12-14T18:26:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 -- start start 2023-12-14T18:26:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe480ff460 0x7fbe481963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 0x7fbe48196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe48104500 0x7fbe4819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:54.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4819b3b0 con 0x7fbe480ffe00 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4819b520 con 0x7fbe48104500 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4d07b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4819b6e0 con 0x7fbe480ff460 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 0x7fbe48196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 0x7fbe48196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48982/0 (socket says 172.21.15.31:48982) 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4659c700 1 -- 172.21.15.31:0/2894586211 learned_addr learned my addr 172.21.15.31:0/2894586211 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe46d9d700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe480ff460 0x7fbe481963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.600+0000 7fbe4759e700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe48104500 0x7fbe4819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4659c700 1 -- 172.21.15.31:0/2894586211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe480ff460 msgr2=0x7fbe481963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4659c700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe480ff460 0x7fbe481963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4659c700 1 -- 172.21.15.31:0/2894586211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe48104500 msgr2=0x7fbe4819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4659c700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe48104500 0x7fbe4819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4659c700 1 -- 172.21.15.31:0/2894586211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380095f0 con 0x7fbe480ffe00 2023-12-14T18:26:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4659c700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 0x7fbe48196930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00e970 tx=0x7fbe3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c009790 con 0x7fbe480ffe00 2023-12-14T18:26:54.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4810f240 con 0x7fbe480ffe00 2023-12-14T18:26:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4810f0b0 con 0x7fbe480ffe00 2023-12-14T18:26:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3c00c7e0 con 0x7fbe480ffe00 2023-12-14T18:26:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.601+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0054e0 con 0x7fbe480ffe00 2023-12-14T18:26:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.602+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4810f3d0 con 0x7fbe480ffe00 2023-12-14T18:26:54.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.604+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbe3c00c950 con 0x7fbe480ffe00 2023-12-14T18:26:54.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.604+0000 7fbe2ffff700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe300a1260 0x7fbe300a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:54.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.604+0000 7fbe46d9d700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe300a1260 0x7fbe300a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:54.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.604+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbe3c014070 con 0x7fbe480ffe00 2023-12-14T18:26:54.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.605+0000 7fbe46d9d700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe300a1260 0x7fbe300a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009990 tx=0x7fbe38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:54.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.613+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe3c096280 con 0x7fbe480ffe00 2023-12-14T18:26:54.708 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:54 smithi031 ceph-mon[42085]: pgmap v260: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 383 B/s wr, 3 op/s; 193 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:54.708 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:54 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/842859526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-14T18:26:54.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:54 smithi204 ceph-mon[43866]: pgmap v260: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 383 B/s wr, 3 op/s; 193 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:54.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:54 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/842859526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-14T18:26:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.837+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbe48066ea0 con 0x7fbe480ffe00 2023-12-14T18:26:54.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.837+0000 7fbe2ffff700 1 -- 172.21.15.31:0/2894586211 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe3c0998a0 con 0x7fbe480ffe00 2023-12-14T18:26:54.841 INFO:teuthology.orchestra.run.smithi031.stdout:55834574906 2023-12-14T18:26:54.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe300a1260 msgr2=0x7fbe300a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe300a1260 0x7fbe300a3710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009990 tx=0x7fbe38005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 msgr2=0x7fbe48196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 0x7fbe48196930 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00e970 tx=0x7fbe3c00ec80 comp rx=0 tx=0).stop 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 shutdown_connections 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe48104500 0x7fbe4819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe300a1260 0x7fbe300a3710 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009990 tx=0x7fbe38005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe480ff460 0x7fbe481963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 --2- 172.21.15.31:0/2894586211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe480ffe00 0x7fbe48196930 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 >> 172.21.15.31:0/2894586211 conn(0x7fbe480770a0 msgr2=0x7fbe48103050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:54.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 shutdown_connections 2023-12-14T18:26:54.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.840+0000 7fbe4d07b700 1 -- 172.21.15.31:0/2894586211 wait complete. 2023-12-14T18:26:54.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:54 smithi176 ceph-mon[43506]: pgmap v260: 129 pgs: 1 peering, 128 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 383 B/s wr, 3 op/s; 193 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:54.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:54 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/842859526' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-14T18:26:55.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201311 got 236223201312 for osd.8 2023-12-14T18:26:55.436 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:55 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3839024230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-14T18:26:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:55 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:55 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2894586211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-14T18:26:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:55 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3839024230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-14T18:26:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:55 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:55 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2894586211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 -- 172.21.15.31:0/129222870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3848100ee0 msgr2=0x7f3848104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 --2- 172.21.15.31:0/129222870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3848100ee0 0x7f3848104eb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f38380099c0 tx=0x7f3838009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 -- 172.21.15.31:0/129222870 shutdown_connections 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 --2- 172.21.15.31:0/129222870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38481055e0 0x7f384810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 --2- 172.21.15.31:0/129222870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3848100540 0x7f3848100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 --2- 172.21.15.31:0/129222870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3848100ee0 0x7f3848104eb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 -- 172.21.15.31:0/129222870 >> 172.21.15.31:0/129222870 conn(0x7f38480fbdc0 msgr2=0x7f38480fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 -- 172.21.15.31:0/129222870 shutdown_connections 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.433+0000 7f384f5e6700 1 -- 172.21.15.31:0/129222870 wait complete. 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 Processor -- start 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 -- start start 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3848100540 0x7f38481963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3848100ee0 0x7f38481968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 0x7f384819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384819b360 con 0x7f38481055e0 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384819b4d0 con 0x7f3848100540 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384f5e6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384819b690 con 0x7f3848100ee0 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384db83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 0x7f384819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384db83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 0x7f384819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48964/0 (socket says 172.21.15.31:48964) 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384cb81700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3848100ee0 0x7f38481968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384db83700 1 -- 172.21.15.31:0/3839024230 learned_addr learned my addr 172.21.15.31:0/3839024230 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.434+0000 7f384d382700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3848100540 0x7f38481963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384db83700 1 -- 172.21.15.31:0/3839024230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3848100ee0 msgr2=0x7f38481968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384db83700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3848100ee0 0x7f38481968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384db83700 1 -- 172.21.15.31:0/3839024230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3848100540 msgr2=0x7f38481963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384db83700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3848100540 0x7f38481963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384db83700 1 -- 172.21.15.31:0/3839024230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38380095f0 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384cb81700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3848100ee0 0x7f38481968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384db83700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 0x7f384819ad50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3840007ee0 tx=0x7f384000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840017070 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3848110310 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f384000f040 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840005550 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.435+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848110180 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.436+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38481104a0 con 0x7f38481055e0 2023-12-14T18:26:56.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.438+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f384000db20 con 0x7f38481055e0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.438+0000 7f383e7fc700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38340a11e0 0x7f38340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.438+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f38400cc440 con 0x7f38481055e0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.438+0000 7f384d382700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38340a11e0 0x7f38340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.439+0000 7f384d382700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38340a11e0 0x7f38340a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3844009c60 tx=0x7f3844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.443+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3840095720 con 0x7f38481055e0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.690+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f3848066ea0 con 0x7f38481055e0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.691+0000 7f383e7fc700 1 -- 172.21.15.31:0/3839024230 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3840098d40 con 0x7f38481055e0 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.694+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38340a11e0 msgr2=0x7f38340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.694+0000 7f384f5e6700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38340a11e0 0x7f38340a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3844009c60 tx=0x7f3844009260 comp rx=0 tx=0).stop 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.694+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 msgr2=0x7f384819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.694+0000 7f384f5e6700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 0x7f384819ad50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3840007ee0 tx=0x7f384000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 shutdown_connections 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38481055e0 0x7f384819ad50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3848100540 0x7f38481963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38340a11e0 0x7f38340a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 --2- 172.21.15.31:0/3839024230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3848100ee0 0x7f38481968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 >> 172.21.15.31:0/3839024230 conn(0x7f38480fbdc0 msgr2=0x7f38480fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 shutdown_connections 2023-12-14T18:26:56.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:54.695+0000 7f384f5e6700 1 -- 172.21.15.31:0/3839024230 wait complete. 2023-12-14T18:26:56.076 INFO:teuthology.orchestra.run.smithi031.stdout:270582939676 2023-12-14T18:26:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:55 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3839024230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-14T18:26:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:55 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:26:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:55 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2894586211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-14T18:26:56.304 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574904 got 55834574906 for osd.1 2023-12-14T18:26:56.304 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:56.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/2656053284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100540 msgr2=0x7f9a78100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/2656053284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100540 0x7f9a78100910 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a680099c0 tx=0x7f9a68009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/2656053284 shutdown_connections 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/2656053284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a781055e0 0x7f9a7810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/2656053284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100540 0x7f9a78100910 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/2656053284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a78100ee0 0x7f9a78104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/2656053284 >> 172.21.15.31:0/2656053284 conn(0x7f9a780fbdc0 msgr2=0x7f9a780fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/2656053284 shutdown_connections 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.951+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/2656053284 wait complete. 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.952+0000 7f9a7dc1a700 1 Processor -- start 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.952+0000 7f9a7dc1a700 1 -- start start 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.952+0000 7f9a7dc1a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a78100540 0x7f9a78196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.952+0000 7f9a7dc1a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 0x7f9a78196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a7dc1a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a781055e0 0x7f9a7819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a7dc1a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819b3e0 con 0x7f9a78100ee0 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a7dc1a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819b550 con 0x7f9a78100540 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a7dc1a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819b710 con 0x7f9a781055e0 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 0x7f9a78196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 0x7f9a78196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48998/0 (socket says 172.21.15.31:48998) 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 -- 172.21.15.31:0/1960191827 learned_addr learned my addr 172.21.15.31:0/1960191827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a77fff700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a781055e0 0x7f9a7819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 -- 172.21.15.31:0/1960191827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a781055e0 msgr2=0x7f9a7819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a781055e0 0x7f9a7819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 -- 172.21.15.31:0/1960191827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a78100540 msgr2=0x7f9a78196420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a78100540 0x7f9a78196420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.953+0000 7f9a76ffd700 1 -- 172.21.15.31:0/1960191827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680095f0 con 0x7f9a78100ee0 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.954+0000 7f9a76ffd700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 0x7f9a78196960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00d980 tx=0x7f9a6c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.954+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c009b40 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.954+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78110320 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.954+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a78110190 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.954+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a6c00de60 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.954+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c020630 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.955+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a64005320 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.956+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9a6c009ca0 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.957+0000 7f9a74ff9700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a600a11e0 0x7f9a600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.957+0000 7f9a777fe700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a600a11e0 0x7f9a600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.957+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a6c0dd8c0 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.958+0000 7f9a777fe700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a600a11e0 0x7f9a600a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9a680098e0 tx=0x7f9a6800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:55.962+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9a6c0a6ba0 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.183+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9a64005190 con 0x7f9a78100ee0 2023-12-14T18:26:56.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.183+0000 7f9a74ff9700 1 -- 172.21.15.31:0/1960191827 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a6c0aa1c0 con 0x7f9a78100ee0 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stdout:154618822700 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a600a11e0 msgr2=0x7f9a600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a600a11e0 0x7f9a600a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9a680098e0 tx=0x7f9a6800b540 comp rx=0 tx=0).stop 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 msgr2=0x7f9a78196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 0x7f9a78196960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00d980 tx=0x7f9a6c00dd40 comp rx=0 tx=0).stop 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 shutdown_connections 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9a600a11e0 0x7f9a600a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a781055e0 0x7f9a7819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a78100540 0x7f9a78196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 --2- 172.21.15.31:0/1960191827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a78100ee0 0x7f9a78196960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 >> 172.21.15.31:0/1960191827 conn(0x7f9a780fbdc0 msgr2=0x7f9a78103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 shutdown_connections 2023-12-14T18:26:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.186+0000 7f9a7dc1a700 1 -- 172.21.15.31:0/1960191827 wait complete. 2023-12-14T18:26:56.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986220 got 133143986223 for osd.4 2023-12-14T18:26:56.670 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:56.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514515 got 326417514517 for osd.11 2023-12-14T18:26:56.810 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:56.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 -- 172.21.15.31:0/1717746981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 msgr2=0x7f3a6410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 --2- 172.21.15.31:0/1717746981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a6410a3c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3a58009990 tx=0x7f3a58009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 -- 172.21.15.31:0/1717746981 shutdown_connections 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 --2- 172.21.15.31:0/1717746981 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a6410a900 0x7f3a6410cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 --2- 172.21.15.31:0/1717746981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a641013e0 0x7f3a641017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 --2- 172.21.15.31:0/1717746981 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a6410a3c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 -- 172.21.15.31:0/1717746981 >> 172.21.15.31:0/1717746981 conn(0x7f3a640770d0 msgr2=0x7f3a640774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 -- 172.21.15.31:0/1717746981 shutdown_connections 2023-12-14T18:26:56.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.948+0000 7f3a69519700 1 -- 172.21.15.31:0/1717746981 wait complete. 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.949+0000 7f3a69519700 1 Processor -- start 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 -- start start 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641013e0 0x7f3a640ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a640ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a6410a900 0x7f3a640ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641a4860 con 0x7f3a64101cf0 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641a49d0 con 0x7f3a641013e0 2023-12-14T18:26:56.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a69519700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a641a4b90 con 0x7f3a6410a900 2023-12-14T18:26:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a627fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a640ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a62ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641013e0 0x7f3a640ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a627fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a640ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49028/0 (socket says 172.21.15.31:49028) 2023-12-14T18:26:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a627fc700 1 -- 172.21.15.31:0/1634738974 learned_addr learned my addr 172.21.15.31:0/1634738974 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:56.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a637fe700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a6410a900 0x7f3a640ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a62ffd700 1 -- 172.21.15.31:0/1634738974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a6410a900 msgr2=0x7f3a640ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a62ffd700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a6410a900 0x7f3a640ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a62ffd700 1 -- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 msgr2=0x7f3a640ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.950+0000 7f3a62ffd700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a640ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a62ffd700 1 -- 172.21.15.31:0/1634738974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580095f0 con 0x7f3a641013e0 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a627fc700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a640ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:56.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a62ffd700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641013e0 0x7f3a640ff210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400e9f0 tx=0x7f3a5400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54004220 con 0x7f3a641013e0 2023-12-14T18:26:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a64110390 con 0x7f3a641013e0 2023-12-14T18:26:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a54004cf0 con 0x7f3a641013e0 2023-12-14T18:26:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400ce70 con 0x7f3a641013e0 2023-12-14T18:26:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.951+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a64110200 con 0x7f3a641013e0 2023-12-14T18:26:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.952+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a64110520 con 0x7f3a641013e0 2023-12-14T18:26:56.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.954+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3a54004750 con 0x7f3a641013e0 2023-12-14T18:26:56.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.955+0000 7f3a4bfff700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a4c0a11e0 0x7f3a4c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.955+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3a54014070 con 0x7f3a641013e0 2023-12-14T18:26:56.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.955+0000 7f3a627fc700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a4c0a11e0 0x7f3a4c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.956+0000 7f3a627fc700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a4c0a11e0 0x7f3a4c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a64100a00 tx=0x7f3a58005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:56.957+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a54096560 con 0x7f3a641013e0 2023-12-14T18:26:56.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:56 smithi031 ceph-mon[42085]: pgmap v261: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s; 193 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:56.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:56 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1960191827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-14T18:26:56.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:56 smithi176 ceph-mon[43506]: pgmap v261: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s; 193 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:56.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:56 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1960191827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-14T18:26:57.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:56 smithi204 ceph-mon[43866]: pgmap v261: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 383 B/s wr, 5 op/s; 193 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:57.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:56 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1960191827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 -- 172.21.15.31:0/3618406510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c1013e0 msgr2=0x7f125c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 --2- 172.21.15.31:0/3618406510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c1013e0 0x7f125c1017b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1258009990 tx=0x7f1258009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 -- 172.21.15.31:0/3618406510 shutdown_connections 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 --2- 172.21.15.31:0/3618406510 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f125c10a900 0x7f125c10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 --2- 172.21.15.31:0/3618406510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c1013e0 0x7f125c1017b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 --2- 172.21.15.31:0/3618406510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c101cf0 0x7f125c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.103+0000 7f126433c700 1 -- 172.21.15.31:0/3618406510 >> 172.21.15.31:0/3618406510 conn(0x7f125c0770d0 msgr2=0x7f125c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.104+0000 7f126433c700 1 -- 172.21.15.31:0/3618406510 shutdown_connections 2023-12-14T18:26:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.104+0000 7f126433c700 1 -- 172.21.15.31:0/3618406510 wait complete. 2023-12-14T18:26:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.104+0000 7f126433c700 1 Processor -- start 2023-12-14T18:26:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.104+0000 7f126433c700 1 -- start start 2023-12-14T18:26:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f126433c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c1013e0 0x7f125c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f126433c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f125c101cf0 0x7f125c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f126433c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 0x7f125c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f126433c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c19d5f0 con 0x7f125c10a900 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f126433c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c19d760 con 0x7f125c101cf0 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f126433c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c19d920 con 0x7f125c1013e0 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 0x7f125c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 0x7f125c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49046/0 (socket says 172.21.15.31:49046) 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 -- 172.21.15.31:0/3289769856 learned_addr learned my addr 172.21.15.31:0/3289769856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12620d8700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c1013e0 0x7f125c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12618d7700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f125c101cf0 0x7f125c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 -- 172.21.15.31:0/3289769856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c1013e0 msgr2=0x7f125c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c1013e0 0x7f125c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 -- 172.21.15.31:0/3289769856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f125c101cf0 msgr2=0x7f125c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f125c101cf0 0x7f125c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 -- 172.21.15.31:0/3289769856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580095f0 con 0x7f125c10a900 2023-12-14T18:26:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.105+0000 7f12628d9700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 0x7f125c19cfe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1254007ce0 tx=0x7f125400d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.106+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254004db0 con 0x7f125c10a900 2023-12-14T18:26:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.106+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f125400f040 con 0x7f125c10a900 2023-12-14T18:26:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.106+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c110370 con 0x7f125c10a900 2023-12-14T18:26:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.106+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254005990 con 0x7f125c10a900 2023-12-14T18:26:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.107+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c1101e0 con 0x7f125c10a900 2023-12-14T18:26:57.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.108+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125c110500 con 0x7f125c10a900 2023-12-14T18:26:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.109+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1254005000 con 0x7f125c10a900 2023-12-14T18:26:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.110+0000 7f12537fe700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12480a11e0 0x7f12480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.110+0000 7f12620d8700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12480a11e0 0x7f12480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.110+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f12540ccdd0 con 0x7f125c10a900 2023-12-14T18:26:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.111+0000 7f12620d8700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12480a11e0 0x7f12480a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f125800b2e0 tx=0x7f1258005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.114+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f12540960b0 con 0x7f125c10a900 2023-12-14T18:26:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 -- 172.21.15.31:0/2515019187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 msgr2=0x7f9b4c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 --2- 172.21.15.31:0/2515019187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c100910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0099c0 tx=0x7f9b3c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 -- 172.21.15.31:0/2515019187 shutdown_connections 2023-12-14T18:26:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 --2- 172.21.15.31:0/2515019187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c1055e0 0x7f9b4c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 --2- 172.21.15.31:0/2515019187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c100910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 --2- 172.21.15.31:0/2515019187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c100ee0 0x7f9b4c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 -- 172.21.15.31:0/2515019187 >> 172.21.15.31:0/2515019187 conn(0x7f9b4c0fbdc0 msgr2=0x7f9b4c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 -- 172.21.15.31:0/2515019187 shutdown_connections 2023-12-14T18:26:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.167+0000 7f9b520ad700 1 -- 172.21.15.31:0/2515019187 wait complete. 2023-12-14T18:26:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.168+0000 7f9b520ad700 1 Processor -- start 2023-12-14T18:26:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.168+0000 7f9b520ad700 1 -- start start 2023-12-14T18:26:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b520ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b520ad700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c100ee0 0x7f9b4c1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b520ad700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c1055e0 0x7f9b4c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b520ad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c19b360 con 0x7f9b4c100540 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b520ad700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c19b4d0 con 0x7f9b4c1055e0 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b520ad700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c19b690 con 0x7f9b4c100ee0 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49072/0 (socket says 172.21.15.31:49072) 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 -- 172.21.15.31:0/3310026100 learned_addr learned my addr 172.21.15.31:0/3310026100 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4affd700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c100ee0 0x7f9b4c1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4bfff700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c1055e0 0x7f9b4c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 -- 172.21.15.31:0/3310026100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c100ee0 msgr2=0x7f9b4c1968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c100ee0 0x7f9b4c1968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 -- 172.21.15.31:0/3310026100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c1055e0 msgr2=0x7f9b4c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c1055e0 0x7f9b4c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4b7fe700 1 -- 172.21.15.31:0/3310026100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0095f0 con 0x7f9b4c100540 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4affd700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c100ee0 0x7f9b4c1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.169+0000 7f9b4bfff700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c1055e0 0x7f9b4c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.170+0000 7f9b4b7fe700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c1963a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009990 tx=0x7f9b3c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.170+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c01d070 con 0x7f9b4c100540 2023-12-14T18:26:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.170+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b3c004010 con 0x7f9b4c100540 2023-12-14T18:26:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.170+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c110320 con 0x7f9b4c100540 2023-12-14T18:26:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.170+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c022d40 con 0x7f9b4c100540 2023-12-14T18:26:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.170+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c110190 con 0x7f9b4c100540 2023-12-14T18:26:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.171+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c1104b0 con 0x7f9b4c100540 2023-12-14T18:26:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.174+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9b3c022470 con 0x7f9b4c100540 2023-12-14T18:26:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.175+0000 7f9b48ff9700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9b340a11e0 0x7f9b340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.175+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9b3c0ce5d0 con 0x7f9b4c100540 2023-12-14T18:26:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.176+0000 7f9b4affd700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9b340a11e0 0x7f9b340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.176+0000 7f9b4affd700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9b340a11e0 0x7f9b340a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b40005950 tx=0x7f9b400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.177+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9b3c097940 con 0x7f9b4c100540 2023-12-14T18:26:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.237+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3a64066ea0 con 0x7f3a641013e0 2023-12-14T18:26:57.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.237+0000 7f3a4bfff700 1 -- 172.21.15.31:0/1634738974 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a54004e60 con 0x7f3a641013e0 2023-12-14T18:26:57.239 INFO:teuthology.orchestra.run.smithi031.stdout:34359738429 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a4c0a11e0 msgr2=0x7f3a4c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a4c0a11e0 0x7f3a4c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a64100a00 tx=0x7f3a58005960 comp rx=0 tx=0).stop 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641013e0 msgr2=0x7f3a640ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641013e0 0x7f3a640ff210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400e9f0 tx=0x7f3a5400ed00 comp rx=0 tx=0).stop 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 shutdown_connections 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3a4c0a11e0 0x7f3a4c0a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a6410a900 0x7f3a640ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a641013e0 0x7f3a640ff210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 --2- 172.21.15.31:0/1634738974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a64101cf0 0x7f3a640ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 >> 172.21.15.31:0/1634738974 conn(0x7f3a640770d0 msgr2=0x7f3a640fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 shutdown_connections 2023-12-14T18:26:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.240+0000 7f3a69519700 1 -- 172.21.15.31:0/1634738974 wait complete. 2023-12-14T18:26:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.353+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f125c066ea0 con 0x7f125c10a900 2023-12-14T18:26:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.354+0000 7f12537fe700 1 -- 172.21.15.31:0/3289769856 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1254023020 con 0x7f125c10a900 2023-12-14T18:26:57.356 INFO:teuthology.orchestra.run.smithi031.stdout:206158430244 2023-12-14T18:26:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12480a11e0 msgr2=0x7f12480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12480a11e0 0x7f12480a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f125800b2e0 tx=0x7f1258005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 msgr2=0x7f125c19cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 0x7f125c19cfe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1254007ce0 tx=0x7f125400d0d0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 shutdown_connections 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f125c10a900 0x7f125c19cfe0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f12480a11e0 0x7f12480a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c1013e0 0x7f125c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 --2- 172.21.15.31:0/3289769856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f125c101cf0 0x7f125c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 >> 172.21.15.31:0/3289769856 conn(0x7f125c0770d0 msgr2=0x7f125c0fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 shutdown_connections 2023-12-14T18:26:57.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.357+0000 7f126433c700 1 -- 172.21.15.31:0/3289769856 wait complete. 2023-12-14T18:26:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.414+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9b4c197730 con 0x7f9b4c100540 2023-12-14T18:26:57.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.414+0000 7f9b48ff9700 1 -- 172.21.15.31:0/3310026100 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b3c09af60 con 0x7f9b4c100540 2023-12-14T18:26:57.416 INFO:teuthology.orchestra.run.smithi031.stdout:77309411383 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9b340a11e0 msgr2=0x7f9b340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9b340a11e0 0x7f9b340a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b40005950 tx=0x7f9b400058e0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 msgr2=0x7f9b4c1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c1963a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009990 tx=0x7f9b3c009580 comp rx=0 tx=0).stop 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 shutdown_connections 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b4c1055e0 0x7f9b4c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b4c100540 0x7f9b4c1963a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9b340a11e0 0x7f9b340a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 --2- 172.21.15.31:0/3310026100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4c100ee0 0x7f9b4c1968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 >> 172.21.15.31:0/3310026100 conn(0x7f9b4c0fbdc0 msgr2=0x7f9b4c0fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 shutdown_connections 2023-12-14T18:26:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.417+0000 7f9b520ad700 1 -- 172.21.15.31:0/3310026100 wait complete. 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 -- 172.21.15.31:0/821203555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 msgr2=0x7fcfa00ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/821203555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa00ff850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcf900099c0 tx=0x7fcf90009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 -- 172.21.15.31:0/821203555 shutdown_connections 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/821203555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa0104520 0x7fcfa010c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/821203555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa00ff850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/821203555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfa00ffe20 0x7fcfa0103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 -- 172.21.15.31:0/821203555 >> 172.21.15.31:0/821203555 conn(0x7fcfa00770d0 msgr2=0x7fcfa00774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 -- 172.21.15.31:0/821203555 shutdown_connections 2023-12-14T18:26:57.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.526+0000 7fcfa5a30700 1 -- 172.21.15.31:0/821203555 wait complete. 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 Processor -- start 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 -- start start 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa0195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfa00ffe20 0x7fcfa0195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa0104520 0x7fcfa0199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa019a2b0 con 0x7fcfa00ff480 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.527+0000 7fcfa5a30700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa019a420 con 0x7fcfa00ffe20 2023-12-14T18:26:57.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcfa5a30700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa019a5e0 con 0x7fcfa0104520 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa0195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa0195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49080/0 (socket says 172.21.15.31:49080) 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 -- 172.21.15.31:0/970282822 learned_addr learned my addr 172.21.15.31:0/970282822 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf967fc700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfa00ffe20 0x7fcfa0195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9f7fe700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa0104520 0x7fcfa0199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 -- 172.21.15.31:0/970282822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa0104520 msgr2=0x7fcfa0199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa0104520 0x7fcfa0199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 -- 172.21.15.31:0/970282822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfa00ffe20 msgr2=0x7fcfa0195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfa00ffe20 0x7fcfa0195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 -- 172.21.15.31:0/970282822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf900095f0 con 0x7fcfa00ff480 2023-12-14T18:26:57.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcf9effd700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa0195340 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009990 tx=0x7fcf90005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.528+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000fba0 con 0x7fcfa00ff480 2023-12-14T18:26:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.529+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa010f2b0 con 0x7fcfa00ff480 2023-12-14T18:26:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.529+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf9000fd00 con 0x7fcfa00ff480 2023-12-14T18:26:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.529+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005660 con 0x7fcfa00ff480 2023-12-14T18:26:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.529+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa010f120 con 0x7fcfa00ff480 2023-12-14T18:26:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.530+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf80005320 con 0x7fcfa00ff480 2023-12-14T18:26:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.531+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fcf90004800 con 0x7fcfa00ff480 2023-12-14T18:26:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.532+0000 7fcfa4a2e700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcf8c0a11e0 0x7fcf8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:57.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.532+0000 7fcf967fc700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcf8c0a11e0 0x7fcf8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:57.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.532+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf900d0680 con 0x7fcfa00ff480 2023-12-14T18:26:57.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.532+0000 7fcf967fc700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcf8c0a11e0 0x7fcf8c0a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0196910 tx=0x7fcf880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:57.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.536+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcf900999f0 con 0x7fcfa00ff480 2023-12-14T18:26:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.752+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fcf80005190 con 0x7fcfa00ff480 2023-12-14T18:26:57.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.752+0000 7fcfa4a2e700 1 -- 172.21.15.31:0/970282822 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf90005000 con 0x7fcfa00ff480 2023-12-14T18:26:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:184683593769 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcf8c0a11e0 msgr2=0x7fcf8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcf8c0a11e0 0x7fcf8c0a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0196910 tx=0x7fcf880073d0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 msgr2=0x7fcfa0195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa0195340 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009990 tx=0x7fcf90005e80 comp rx=0 tx=0).stop 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 shutdown_connections 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fcf8c0a11e0 0x7fcf8c0a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfa0104520 0x7fcfa0199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfa00ff480 0x7fcfa0195340 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 --2- 172.21.15.31:0/970282822 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcfa00ffe20 0x7fcfa0195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:57.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 >> 172.21.15.31:0/970282822 conn(0x7fcfa00770d0 msgr2=0x7fcfa00fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 shutdown_connections 2023-12-14T18:26:57.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:26:57.755+0000 7fcfa5a30700 1 -- 172.21.15.31:0/970282822 wait complete. 2023-12-14T18:26:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:57 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1634738974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-14T18:26:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:57 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3289769856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-14T18:26:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:57 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3310026100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-14T18:26:57.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:57 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1634738974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-14T18:26:57.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:57 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3289769856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-14T18:26:57.995 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:57 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3310026100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-14T18:26:58.018 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738426 got 34359738429 for osd.0 2023-12-14T18:26:58.018 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:58.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:57 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1634738974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-14T18:26:58.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:57 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3289769856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-14T18:26:58.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:57 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3310026100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-14T18:26:58.496 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822698 got 154618822700 for osd.5 2023-12-14T18:26:58.497 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:58.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593767 got 184683593769 for osd.6 2023-12-14T18:26:58.505 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:58.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:58 smithi031 ceph-mon[42085]: pgmap v262: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 309 B/s wr, 5 op/s; 155 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:58.891 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:26:58 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/970282822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-14T18:26:58.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939674 got 270582939676 for osd.9 2023-12-14T18:26:58.892 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:58.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:58 smithi176 ceph-mon[43506]: pgmap v262: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 309 B/s wr, 5 op/s; 155 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:58.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:26:58 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/970282822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-14T18:26:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:58 smithi204 ceph-mon[43866]: pgmap v262: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 309 B/s wr, 5 op/s; 155 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:26:59.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:26:58 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/970282822' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-14T18:26:59.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411381 got 77309411383 for osd.2 2023-12-14T18:26:59.128 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:59.148 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430242 got 206158430244 for osd.7 2023-12-14T18:26:59.148 DEBUG:teuthology.parallel:result is None 2023-12-14T18:26:59.148 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-14T18:26:59.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:26:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 -- 172.21.15.31:0/4252017896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc100ee0 msgr2=0x7f43fc104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 --2- 172.21.15.31:0/4252017896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc100ee0 0x7f43fc104eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0099c0 tx=0x7f43ec009cd0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 -- 172.21.15.31:0/4252017896 shutdown_connections 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 --2- 172.21.15.31:0/4252017896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc1055e0 0x7f43fc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 --2- 172.21.15.31:0/4252017896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43fc100540 0x7f43fc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 --2- 172.21.15.31:0/4252017896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc100ee0 0x7f43fc104eb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 -- 172.21.15.31:0/4252017896 >> 172.21.15.31:0/4252017896 conn(0x7f43fc0fbdc0 msgr2=0x7f43fc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 -- 172.21.15.31:0/4252017896 shutdown_connections 2023-12-14T18:27:00.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.662+0000 7f4403cdd700 1 -- 172.21.15.31:0/4252017896 wait complete. 2023-12-14T18:27:00.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.663+0000 7f4403cdd700 1 Processor -- start 2023-12-14T18:27:00.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.663+0000 7f4403cdd700 1 -- start start 2023-12-14T18:27:00.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.663+0000 7f4403cdd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43fc100540 0x7f43fc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:00.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4403cdd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc100ee0 0x7f43fc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4403cdd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 0x7f43fc0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4403cdd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1a4820 con 0x7f43fc1055e0 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4403cdd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1a4990 con 0x7f43fc100ee0 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4403cdd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1a4b50 con 0x7f43fc100540 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 0x7f43fc0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 0x7f43fc0738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49100/0 (socket says 172.21.15.31:49100) 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 -- 172.21.15.31:0/421856274 learned_addr learned my addr 172.21.15.31:0/421856274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4401278700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc100ee0 0x7f43fc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:00.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4401a79700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43fc100540 0x7f43fc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 -- 172.21.15.31:0/421856274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43fc100540 msgr2=0x7f43fc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43fc100540 0x7f43fc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 -- 172.21.15.31:0/421856274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc100ee0 msgr2=0x7f43fc073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc100ee0 0x7f43fc073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 -- 172.21.15.31:0/421856274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec0095f0 con 0x7f43fc1055e0 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f4401278700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc100ee0 0x7f43fc073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.664+0000 7f440227a700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 0x7f43fc0738b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f43f4007ee0 tx=0x7f43f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.665+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400dd20 con 0x7f43fc1055e0 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.665+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43f400f040 con 0x7f43fc1055e0 2023-12-14T18:27:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.665+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc110310 con 0x7f43fc1055e0 2023-12-14T18:27:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.665+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40048a0 con 0x7f43fc1055e0 2023-12-14T18:27:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.665+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc110180 con 0x7f43fc1055e0 2023-12-14T18:27:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.666+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc1104a0 con 0x7f43fc1055e0 2023-12-14T18:27:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.667+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f43f4003a00 con 0x7f43fc1055e0 2023-12-14T18:27:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.668+0000 7f43f2ffd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f43e80a11e0 0x7f43e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.668+0000 7f4401a79700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f43e80a11e0 0x7f43e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.668+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43f40cc620 con 0x7f43fc1055e0 2023-12-14T18:27:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.668+0000 7f4401a79700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f43e80a11e0 0x7f43e80a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f43f8007be0 tx=0x7f43f80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.672+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f43f404e730 con 0x7f43fc1055e0 2023-12-14T18:27:00.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.883+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43fc0fcf50 con 0x7f43e80a11e0 2023-12-14T18:27:00.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.904+0000 7f43f2ffd700 1 -- 172.21.15.31:0/421856274 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+425094 (secure 0 0 0) 0x7f43fc0fcf50 con 0x7f43e80a11e0 2023-12-14T18:27:00.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:00.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.908+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f43e80a11e0 msgr2=0x7f43e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.908+0000 7f4403cdd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f43e80a11e0 0x7f43e80a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f43f8007be0 tx=0x7f43f80073d0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 msgr2=0x7f43fc0738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 0x7f43fc0738b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f43f4007ee0 tx=0x7f43f400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 shutdown_connections 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43fc1055e0 0x7f43fc0738b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f43e80a11e0 0x7f43e80a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43fc100540 0x7f43fc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 --2- 172.21.15.31:0/421856274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f43fc100ee0 0x7f43fc073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:00.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 >> 172.21.15.31:0/421856274 conn(0x7f43fc0fbdc0 msgr2=0x7f43fc0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:00.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 shutdown_connections 2023-12-14T18:27:00.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:00.909+0000 7f4403cdd700 1 -- 172.21.15.31:0/421856274 wait complete. 2023-12-14T18:27:00.918 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:27:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:00 smithi176 ceph-mon[43506]: pgmap v263: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 281 B/s wr, 6 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:27:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:00 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:00 smithi204 ceph-mon[43866]: pgmap v263: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 281 B/s wr, 6 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:27:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:00 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.132 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:00 smithi031 ceph-mon[42085]: pgmap v263: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 281 B/s wr, 6 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:27:01.132 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.132 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:00 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.320 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2023-12-14T18:26:59.487728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654325,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":364,"num_read_kb":303,"num_write":340,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1182282,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80800,"kb_used_data":11648,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124861024,"statfs":{"total":1151940427776,"available":1151857688576,"internally_reserved":0,"allocated":11927552,"data_stored":7745251,"data_compressed":319020,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":37,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.932049"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562794+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:48.562794+0000","last_peered":"2023-12-14T18:26:48.562794+0000","last_clean":"2023-12-14T18:26:48.562794+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:48.562794+0000","last_undegraded":"2023-12-14T18:26:48.562794+0000","last_fullsized":"2023-12-14T18:26:48.562794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:33:46.076524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557880+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:48.557880+0000","last_peered":"2023-12-14T18:26:48.557880+0000","last_clean":"2023-12-14T18:26:48.557880+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:48.557880+0000","last_undegraded":"2023-12-14T18:26:48.557880+0000","last_fullsized":"2023-12-14T18:26:48.557880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:10.369082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557355+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:48.557355+0000","last_peered":"2023-12-14T18:26:48.557355+0000","last_clean":"2023-12-14T18:26:48.557355+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:48.557355+0000","last_undegraded":"2023-12-14T18:26:48.557355+0000","last_fullsized":"2023-12-14T18:26:48.557355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:23.331965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563177+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:48.563177+0000","last_peered":"2023-12-14T18:26:48.563177+0000","last_clean":"2023-12-14T18:26:48.563177+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:48.563177+0000","last_undegraded":"2023-12-14T18:26:48.563177+0000","last_fullsized":"2023-12-14T18:26:48.563177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:01.529418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567124+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:48.567124+0000","last_peered":"2023-12-14T18:26:48.567124+0000","last_clean":"2023-12-14T18:26:48.567124+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:48.567124+0000","last_undegraded":"2023-12-14T18:26:48.567124+0000","last_fullsized":"2023-12-14T18:26:48.567124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:53:32.988966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647501+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:26:48.647501+0000","last_peered":"2023-12-14T18:26:48.647501+0000","last_clean":"2023-12-14T18:26:48.647501+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:26:48.647501+0000","last_undegraded":"2023-12-14T18:26:48.647501+0000","last_fullsized":"2023-12-14T18:26:48.647501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:00:59.230921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561700+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:48.561700+0000","last_peered":"2023-12-14T18:26:48.561700+0000","last_clean":"2023-12-14T18:26:48.561700+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:48.561700+0000","last_undegraded":"2023-12-14T18:26:48.561700+0000","last_fullsized":"2023-12-14T18:26:48.561700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:47.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559287+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:48.559287+0000","last_peered":"2023-12-14T18:26:48.559287+0000","last_clean":"2023-12-14T18:26:48.559287+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:48.559287+0000","last_undegraded":"2023-12-14T18:26:48.559287+0000","last_fullsized":"2023-12-14T18:26:48.559287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:13:56.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560906+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:48.560906+0000","last_peered":"2023-12-14T18:26:48.560906+0000","last_clean":"2023-12-14T18:26:48.560906+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:48.560906+0000","last_undegraded":"2023-12-14T18:26:48.560906+0000","last_fullsized":"2023-12-14T18:26:48.560906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:38:42.914156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647960+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:26:48.647960+0000","last_peered":"2023-12-14T18:26:48.647960+0000","last_clean":"2023-12-14T18:26:48.647960+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:26:48.647960+0000","last_undegraded":"2023-12-14T18:26:48.647960+0000","last_fullsized":"2023-12-14T18:26:48.647960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:09:41.346067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562267+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:48.562267+0000","last_peered":"2023-12-14T18:26:48.562267+0000","last_clean":"2023-12-14T18:26:48.562267+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:48.562267+0000","last_undegraded":"2023-12-14T18:26:48.562267+0000","last_fullsized":"2023-12-14T18:26:48.562267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:26.374795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563022+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:48.563022+0000","last_peered":"2023-12-14T18:26:48.563022+0000","last_clean":"2023-12-14T18:26:48.563022+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:48.563022+0000","last_undegraded":"2023-12-14T18:26:48.563022+0000","last_fullsized":"2023-12-14T18:26:48.563022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:38.171386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563106+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:48.563106+0000","last_peered":"2023-12-14T18:26:48.563106+0000","last_clean":"2023-12-14T18:26:48.563106+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:48.563106+0000","last_undegraded":"2023-12-14T18:26:48.563106+0000","last_fullsized":"2023-12-14T18:26:48.563106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:18:44.617868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557419+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:48.557419+0000","last_peered":"2023-12-14T18:26:48.557419+0000","last_clean":"2023-12-14T18:26:48.557419+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:48.557419+0000","last_undegraded":"2023-12-14T18:26:48.557419+0000","last_fullsized":"2023-12-14T18:26:48.557419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:04:10.141803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558264+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:48.558264+0000","last_peered":"2023-12-14T18:26:48.558264+0000","last_clean":"2023-12-14T18:26:48.558264+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:48.558264+0000","last_undegraded":"2023-12-14T18:26:48.558264+0000","last_fullsized":"2023-12-14T18:26:48.558264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:53.022658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559788+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:48.559788+0000","last_peered":"2023-12-14T18:26:48.559788+0000","last_clean":"2023-12-14T18:26:48.559788+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:48.559788+0000","last_undegraded":"2023-12-14T18:26:48.559788+0000","last_fullsized":"2023-12-14T18:26:48.559788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:18.996023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562659+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:48.562659+0000","last_peered":"2023-12-14T18:26:48.562659+0000","last_clean":"2023-12-14T18:26:48.562659+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:48.562659+0000","last_undegraded":"2023-12-14T18:26:48.562659+0000","last_fullsized":"2023-12-14T18:26:48.562659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:00:27.549342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562608+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:26:48.562608+0000","last_peered":"2023-12-14T18:26:48.562608+0000","last_clean":"2023-12-14T18:26:48.562608+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:26:48.562608+0000","last_undegraded":"2023-12-14T18:26:48.562608+0000","last_fullsized":"2023-12-14T18:26:48.562608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:06:20.688646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566623+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:26:48.566623+0000","last_peered":"2023-12-14T18:26:48.566623+0000","last_clean":"2023-12-14T18:26:48.566623+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:26:48.566623+0000","last_undegraded":"2023-12-14T18:26:48.566623+0000","last_fullsized":"2023-12-14T18:26:48.566623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:58.296365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564244+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:26:48.564244+0000","last_peered":"2023-12-14T18:26:48.564244+0000","last_clean":"2023-12-14T18:26:48.564244+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:26:48.564244+0000","last_undegraded":"2023-12-14T18:26:48.564244+0000","last_fullsized":"2023-12-14T18:26:48.564244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:23.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563082+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:26:48.563082+0000","last_peered":"2023-12-14T18:26:48.563082+0000","last_clean":"2023-12-14T18:26:48.563082+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:26:48.563082+0000","last_undegraded":"2023-12-14T18:26:48.563082+0000","last_fullsized":"2023-12-14T18:26:48.563082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:35:18.652626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559422+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:48.559422+0000","last_peered":"2023-12-14T18:26:48.559422+0000","last_clean":"2023-12-14T18:26:48.559422+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:48.559422+0000","last_undegraded":"2023-12-14T18:26:48.559422+0000","last_fullsized":"2023-12-14T18:26:48.559422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:31.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562609+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:48.562609+0000","last_peered":"2023-12-14T18:26:48.562609+0000","last_clean":"2023-12-14T18:26:48.562609+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:48.562609+0000","last_undegraded":"2023-12-14T18:26:48.562609+0000","last_fullsized":"2023-12-14T18:26:48.562609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:43.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559749+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:48.559749+0000","last_peered":"2023-12-14T18:26:48.559749+0000","last_clean":"2023-12-14T18:26:48.559749+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:48.559749+0000","last_undegraded":"2023-12-14T18:26:48.559749+0000","last_fullsized":"2023-12-14T18:26:48.559749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:43.650498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558888+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:48.558888+0000","last_peered":"2023-12-14T18:26:48.558888+0000","last_clean":"2023-12-14T18:26:48.558888+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:48.558888+0000","last_undegraded":"2023-12-14T18:26:48.558888+0000","last_fullsized":"2023-12-14T18:26:48.558888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:50.284685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564072+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:48.564072+0000","last_peered":"2023-12-14T18:26:48.564072+0000","last_clean":"2023-12-14T18:26:48.564072+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:48.564072+0000","last_undegraded":"2023-12-14T18:26:48.564072+0000","last_fullsized":"2023-12-14T18:26:48.564072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:29:21.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568778+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:48.568778+0000","last_peered":"2023-12-14T18:26:48.568778+0000","last_clean":"2023-12-14T18:26:48.568778+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:48.568778+0000","last_undegraded":"2023-12-14T18:26:48.568778+0000","last_fullsized":"2023-12-14T18:26:48.568778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:38.658671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560627+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:48.560627+0000","last_peered":"2023-12-14T18:26:48.560627+0000","last_clean":"2023-12-14T18:26:48.560627+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:48.560627+0000","last_undegraded":"2023-12-14T18:26:48.560627+0000","last_fullsized":"2023-12-14T18:26:48.560627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:54:13.436594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562615+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:48.562615+0000","last_peered":"2023-12-14T18:26:48.562615+0000","last_clean":"2023-12-14T18:26:48.562615+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:48.562615+0000","last_undegraded":"2023-12-14T18:26:48.562615+0000","last_fullsized":"2023-12-14T18:26:48.562615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:22:13.915160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568436+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:48.568436+0000","last_peered":"2023-12-14T18:26:48.568436+0000","last_clean":"2023-12-14T18:26:48.568436+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:48.568436+0000","last_undegraded":"2023-12-14T18:26:48.568436+0000","last_fullsized":"2023-12-14T18:26:48.568436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:51:57.429285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563999+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:48.563999+0000","last_peered":"2023-12-14T18:26:48.563999+0000","last_clean":"2023-12-14T18:26:48.563999+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:48.563999+0000","last_undegraded":"2023-12-14T18:26:48.563999+0000","last_fullsized":"2023-12-14T18:26:48.563999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:21:53.454334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564084+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:26:48.564084+0000","last_peered":"2023-12-14T18:26:48.564084+0000","last_clean":"2023-12-14T18:26:48.564084+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:26:48.564084+0000","last_undegraded":"2023-12-14T18:26:48.564084+0000","last_fullsized":"2023-12-14T18:26:48.564084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:33.154296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:55.580449+0000","last_change":"2023-12-14T18:26:48.574808+0000","last_active":"2023-12-14T18:26:55.580449+0000","last_peered":"2023-12-14T18:26:55.580449+0000","last_clean":"2023-12-14T18:26:55.580449+0000","last_became_active":"2023-12-14T18:26:48.568640+0000","last_became_peered":"2023-12-14T18:26:48.568640+0000","last_unstale":"2023-12-14T18:26:55.580449+0000","last_undegraded":"2023-12-14T18:26:55.580449+0000","last_fullsized":"2023-12-14T18:26:55.580449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:16.685666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566784+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:48.566784+0000","last_peered":"2023-12-14T18:26:48.566784+0000","last_clean":"2023-12-14T18:26:48.566784+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:48.566784+0000","last_undegraded":"2023-12-14T18:26:48.566784+0000","last_fullsized":"2023-12-14T18:26:48.566784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.326220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.573798+0000","last_change":"2023-12-14T18:26:48.573798+0000","last_active":"2023-12-14T18:26:48.573798+0000","last_peered":"2023-12-14T18:26:48.573798+0000","last_clean":"2023-12-14T18:26:48.573798+0000","last_became_active":"2023-12-14T18:26:48.573308+0000","last_became_peered":"2023-12-14T18:26:48.573308+0000","last_unstale":"2023-12-14T18:26:48.573798+0000","last_undegraded":"2023-12-14T18:26:48.573798+0000","last_fullsized":"2023-12-14T18:26:48.573798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:28:46.273622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647410+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:48.647410+0000","last_peered":"2023-12-14T18:26:48.647410+0000","last_clean":"2023-12-14T18:26:48.647410+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:48.647410+0000","last_undegraded":"2023-12-14T18:26:48.647410+0000","last_fullsized":"2023-12-14T18:26:48.647410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:05:15.020350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564341+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:48.564341+0000","last_peered":"2023-12-14T18:26:48.564341+0000","last_clean":"2023-12-14T18:26:48.564341+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:48.564341+0000","last_undegraded":"2023-12-14T18:26:48.564341+0000","last_fullsized":"2023-12-14T18:26:48.564341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:43.990428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570306+0000","last_change":"2023-12-14T18:26:48.570306+0000","last_active":"2023-12-14T18:26:48.570306+0000","last_peered":"2023-12-14T18:26:48.570306+0000","last_clean":"2023-12-14T18:26:48.570306+0000","last_became_active":"2023-12-14T18:26:48.569803+0000","last_became_peered":"2023-12-14T18:26:48.569803+0000","last_unstale":"2023-12-14T18:26:48.570306+0000","last_undegraded":"2023-12-14T18:26:48.570306+0000","last_fullsized":"2023-12-14T18:26:48.570306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:33.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562436+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:48.562436+0000","last_peered":"2023-12-14T18:26:48.562436+0000","last_clean":"2023-12-14T18:26:48.562436+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:48.562436+0000","last_undegraded":"2023-12-14T18:26:48.562436+0000","last_fullsized":"2023-12-14T18:26:48.562436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:53.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559154+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:48.559154+0000","last_peered":"2023-12-14T18:26:48.559154+0000","last_clean":"2023-12-14T18:26:48.559154+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:48.559154+0000","last_undegraded":"2023-12-14T18:26:48.559154+0000","last_fullsized":"2023-12-14T18:26:48.559154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:49.740844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557724+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:48.557724+0000","last_peered":"2023-12-14T18:26:48.557724+0000","last_clean":"2023-12-14T18:26:48.557724+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:48.557724+0000","last_undegraded":"2023-12-14T18:26:48.557724+0000","last_fullsized":"2023-12-14T18:26:48.557724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:44:17.333118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567219+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:48.567219+0000","last_peered":"2023-12-14T18:26:48.567219+0000","last_clean":"2023-12-14T18:26:48.567219+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:48.567219+0000","last_undegraded":"2023-12-14T18:26:48.567219+0000","last_fullsized":"2023-12-14T18:26:48.567219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:45:09.792477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561943+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:26:48.561943+0000","last_peered":"2023-12-14T18:26:48.561943+0000","last_clean":"2023-12-14T18:26:48.561943+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:26:48.561943+0000","last_undegraded":"2023-12-14T18:26:48.561943+0000","last_fullsized":"2023-12-14T18:26:48.561943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:21:43.141339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559411+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:48.559411+0000","last_peered":"2023-12-14T18:26:48.559411+0000","last_clean":"2023-12-14T18:26:48.559411+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:48.559411+0000","last_undegraded":"2023-12-14T18:26:48.559411+0000","last_fullsized":"2023-12-14T18:26:48.559411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:36.307493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:57.582575+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:26:57.582575+0000","last_peered":"2023-12-14T18:26:57.582575+0000","last_clean":"2023-12-14T18:26:57.582575+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:26:57.582575+0000","last_undegraded":"2023-12-14T18:26:57.582575+0000","last_fullsized":"2023-12-14T18:26:57.582575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:20:23.673760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560703+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:48.560703+0000","last_peered":"2023-12-14T18:26:48.560703+0000","last_clean":"2023-12-14T18:26:48.560703+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:48.560703+0000","last_undegraded":"2023-12-14T18:26:48.560703+0000","last_fullsized":"2023-12-14T18:26:48.560703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:33:22.836099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561628+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:26:48.561628+0000","last_peered":"2023-12-14T18:26:48.561628+0000","last_clean":"2023-12-14T18:26:48.561628+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:26:48.561628+0000","last_undegraded":"2023-12-14T18:26:48.561628+0000","last_fullsized":"2023-12-14T18:26:48.561628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:47.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561906+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:48.561906+0000","last_peered":"2023-12-14T18:26:48.561906+0000","last_clean":"2023-12-14T18:26:48.561906+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:48.561906+0000","last_undegraded":"2023-12-14T18:26:48.561906+0000","last_fullsized":"2023-12-14T18:26:48.561906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:00.359853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558750+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:48.558750+0000","last_peered":"2023-12-14T18:26:48.558750+0000","last_clean":"2023-12-14T18:26:48.558750+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:48.558750+0000","last_undegraded":"2023-12-14T18:26:48.558750+0000","last_fullsized":"2023-12-14T18:26:48.558750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:49.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568565+0000","last_change":"2023-12-14T18:26:48.568565+0000","last_active":"2023-12-14T18:26:48.568565+0000","last_peered":"2023-12-14T18:26:48.568565+0000","last_clean":"2023-12-14T18:26:48.568565+0000","last_became_active":"2023-12-14T18:26:48.568090+0000","last_became_peered":"2023-12-14T18:26:48.568090+0000","last_unstale":"2023-12-14T18:26:48.568565+0000","last_undegraded":"2023-12-14T18:26:48.568565+0000","last_fullsized":"2023-12-14T18:26:48.568565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:14:38.160855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563856+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:48.563856+0000","last_peered":"2023-12-14T18:26:48.563856+0000","last_clean":"2023-12-14T18:26:48.563856+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:48.563856+0000","last_undegraded":"2023-12-14T18:26:48.563856+0000","last_fullsized":"2023-12-14T18:26:48.563856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:12.189176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560853+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:48.560853+0000","last_peered":"2023-12-14T18:26:48.560853+0000","last_clean":"2023-12-14T18:26:48.560853+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:48.560853+0000","last_undegraded":"2023-12-14T18:26:48.560853+0000","last_fullsized":"2023-12-14T18:26:48.560853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:55:18.680896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562308+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:26:48.562308+0000","last_peered":"2023-12-14T18:26:48.562308+0000","last_clean":"2023-12-14T18:26:48.562308+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:26:48.562308+0000","last_undegraded":"2023-12-14T18:26:48.562308+0000","last_fullsized":"2023-12-14T18:26:48.562308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:05:51.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":139,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:58.583889+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:26:58.583889+0000","last_peered":"2023-12-14T18:26:58.583889+0000","last_clean":"2023-12-14T18:26:58.583889+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:26:58.583889+0000","last_undegraded":"2023-12-14T18:26:58.583889+0000","last_fullsized":"2023-12-14T18:26:58.583889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:33.081169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59807,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":98,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568542+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:48.568542+0000","last_peered":"2023-12-14T18:26:48.568542+0000","last_clean":"2023-12-14T18:26:48.568542+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:48.568542+0000","last_undegraded":"2023-12-14T18:26:48.568542+0000","last_fullsized":"2023-12-14T18:26:48.568542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:30.839862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564457+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:48.564457+0000","last_peered":"2023-12-14T18:26:48.564457+0000","last_clean":"2023-12-14T18:26:48.564457+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:48.564457+0000","last_undegraded":"2023-12-14T18:26:48.564457+0000","last_fullsized":"2023-12-14T18:26:48.564457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:27.839169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565422+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:48.565422+0000","last_peered":"2023-12-14T18:26:48.565422+0000","last_clean":"2023-12-14T18:26:48.565422+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:48.565422+0000","last_undegraded":"2023-12-14T18:26:48.565422+0000","last_fullsized":"2023-12-14T18:26:48.565422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:34.931404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562476+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:48.562476+0000","last_peered":"2023-12-14T18:26:48.562476+0000","last_clean":"2023-12-14T18:26:48.562476+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:48.562476+0000","last_undegraded":"2023-12-14T18:26:48.562476+0000","last_fullsized":"2023-12-14T18:26:48.562476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:45.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561959+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:48.561959+0000","last_peered":"2023-12-14T18:26:48.561959+0000","last_clean":"2023-12-14T18:26:48.561959+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:48.561959+0000","last_undegraded":"2023-12-14T18:26:48.561959+0000","last_fullsized":"2023-12-14T18:26:48.561959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:54.542913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565425+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:48.565425+0000","last_peered":"2023-12-14T18:26:48.565425+0000","last_clean":"2023-12-14T18:26:48.565425+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:48.565425+0000","last_undegraded":"2023-12-14T18:26:48.565425+0000","last_fullsized":"2023-12-14T18:26:48.565425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:46.938078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566091+0000","last_change":"2023-12-14T18:26:48.566091+0000","last_active":"2023-12-14T18:26:48.566091+0000","last_peered":"2023-12-14T18:26:48.566091+0000","last_clean":"2023-12-14T18:26:48.566091+0000","last_became_active":"2023-12-14T18:26:48.565714+0000","last_became_peered":"2023-12-14T18:26:48.565714+0000","last_unstale":"2023-12-14T18:26:48.566091+0000","last_undegraded":"2023-12-14T18:26:48.566091+0000","last_fullsized":"2023-12-14T18:26:48.566091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:48.478859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562658+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:26:48.562658+0000","last_peered":"2023-12-14T18:26:48.562658+0000","last_clean":"2023-12-14T18:26:48.562658+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:26:48.562658+0000","last_undegraded":"2023-12-14T18:26:48.562658+0000","last_fullsized":"2023-12-14T18:26:48.562658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:22.379528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:52.658471+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:26:52.658471+0000","last_peered":"2023-12-14T18:26:52.658471+0000","last_clean":"2023-12-14T18:26:52.658471+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:26:52.658471+0000","last_undegraded":"2023-12-14T18:26:52.658471+0000","last_fullsized":"2023-12-14T18:26:52.658471+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:26:13.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561690+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:48.561690+0000","last_peered":"2023-12-14T18:26:48.561690+0000","last_clean":"2023-12-14T18:26:48.561690+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:48.561690+0000","last_undegraded":"2023-12-14T18:26:48.561690+0000","last_fullsized":"2023-12-14T18:26:48.561690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:10:17.702084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.649376+0000","last_change":"2023-12-14T18:26:48.649376+0000","last_active":"2023-12-14T18:26:48.649376+0000","last_peered":"2023-12-14T18:26:48.649376+0000","last_clean":"2023-12-14T18:26:48.649376+0000","last_became_active":"2023-12-14T18:26:48.648882+0000","last_became_peered":"2023-12-14T18:26:48.648882+0000","last_unstale":"2023-12-14T18:26:48.649376+0000","last_undegraded":"2023-12-14T18:26:48.649376+0000","last_fullsized":"2023-12-14T18:26:48.649376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:12.045140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566931+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:48.566931+0000","last_peered":"2023-12-14T18:26:48.566931+0000","last_clean":"2023-12-14T18:26:48.566931+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:48.566931+0000","last_undegraded":"2023-12-14T18:26:48.566931+0000","last_fullsized":"2023-12-14T18:26:48.566931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:47.382142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.572300+0000","last_change":"2023-12-14T18:26:48.572300+0000","last_active":"2023-12-14T18:26:48.572300+0000","last_peered":"2023-12-14T18:26:48.572300+0000","last_clean":"2023-12-14T18:26:48.572300+0000","last_became_active":"2023-12-14T18:26:48.571895+0000","last_became_peered":"2023-12-14T18:26:48.571895+0000","last_unstale":"2023-12-14T18:26:48.572300+0000","last_undegraded":"2023-12-14T18:26:48.572300+0000","last_fullsized":"2023-12-14T18:26:48.572300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:40:54.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566446+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:48.566446+0000","last_peered":"2023-12-14T18:26:48.566446+0000","last_clean":"2023-12-14T18:26:48.566446+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:48.566446+0000","last_undegraded":"2023-12-14T18:26:48.566446+0000","last_fullsized":"2023-12-14T18:26:48.566446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:44.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570871+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:48.570871+0000","last_peered":"2023-12-14T18:26:48.570871+0000","last_clean":"2023-12-14T18:26:48.570871+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:48.570871+0000","last_undegraded":"2023-12-14T18:26:48.570871+0000","last_fullsized":"2023-12-14T18:26:48.570871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:28.018194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561975+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:48.561975+0000","last_peered":"2023-12-14T18:26:48.561975+0000","last_clean":"2023-12-14T18:26:48.561975+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:48.561975+0000","last_undegraded":"2023-12-14T18:26:48.561975+0000","last_fullsized":"2023-12-14T18:26:48.561975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:37.576788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566649+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:48.566649+0000","last_peered":"2023-12-14T18:26:48.566649+0000","last_clean":"2023-12-14T18:26:48.566649+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:48.566649+0000","last_undegraded":"2023-12-14T18:26:48.566649+0000","last_fullsized":"2023-12-14T18:26:48.566649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:37.853928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562990+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:26:48.562990+0000","last_peered":"2023-12-14T18:26:48.562990+0000","last_clean":"2023-12-14T18:26:48.562990+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:26:48.562990+0000","last_undegraded":"2023-12-14T18:26:48.562990+0000","last_fullsized":"2023-12-14T18:26:48.562990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:32.215966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647772+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:26:48.647772+0000","last_peered":"2023-12-14T18:26:48.647772+0000","last_clean":"2023-12-14T18:26:48.647772+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:26:48.647772+0000","last_undegraded":"2023-12-14T18:26:48.647772+0000","last_fullsized":"2023-12-14T18:26:48.647772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:51:12.578633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563700+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:48.563700+0000","last_peered":"2023-12-14T18:26:48.563700+0000","last_clean":"2023-12-14T18:26:48.563700+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:48.563700+0000","last_undegraded":"2023-12-14T18:26:48.563700+0000","last_fullsized":"2023-12-14T18:26:48.563700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:58.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561937+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:48.561937+0000","last_peered":"2023-12-14T18:26:48.561937+0000","last_clean":"2023-12-14T18:26:48.561937+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:48.561937+0000","last_undegraded":"2023-12-14T18:26:48.561937+0000","last_fullsized":"2023-12-14T18:26:48.561937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:21.783482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647707+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:26:48.647707+0000","last_peered":"2023-12-14T18:26:48.647707+0000","last_clean":"2023-12-14T18:26:48.647707+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:26:48.647707+0000","last_undegraded":"2023-12-14T18:26:48.647707+0000","last_fullsized":"2023-12-14T18:26:48.647707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:26:41.629317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563228+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:48.563228+0000","last_peered":"2023-12-14T18:26:48.563228+0000","last_clean":"2023-12-14T18:26:48.563228+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:48.563228+0000","last_undegraded":"2023-12-14T18:26:48.563228+0000","last_fullsized":"2023-12-14T18:26:48.563228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:41:45.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566971+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:48.566971+0000","last_peered":"2023-12-14T18:26:48.566971+0000","last_clean":"2023-12-14T18:26:48.566971+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:48.566971+0000","last_undegraded":"2023-12-14T18:26:48.566971+0000","last_fullsized":"2023-12-14T18:26:48.566971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:54.638247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:54.660308+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:26:54.660308+0000","last_peered":"2023-12-14T18:26:54.660308+0000","last_clean":"2023-12-14T18:26:54.660308+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:26:54.660308+0000","last_undegraded":"2023-12-14T18:26:54.660308+0000","last_fullsized":"2023-12-14T18:26:54.660308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:07:36.295981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562086+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:48.562086+0000","last_peered":"2023-12-14T18:26:48.562086+0000","last_clean":"2023-12-14T18:26:48.562086+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:48.562086+0000","last_undegraded":"2023-12-14T18:26:48.562086+0000","last_fullsized":"2023-12-14T18:26:48.562086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:00.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559419+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:48.559419+0000","last_peered":"2023-12-14T18:26:48.559419+0000","last_clean":"2023-12-14T18:26:48.559419+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:48.559419+0000","last_undegraded":"2023-12-14T18:26:48.559419+0000","last_fullsized":"2023-12-14T18:26:48.559419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:01.825894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564209+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:48.564209+0000","last_peered":"2023-12-14T18:26:48.564209+0000","last_clean":"2023-12-14T18:26:48.564209+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:48.564209+0000","last_undegraded":"2023-12-14T18:26:48.564209+0000","last_fullsized":"2023-12-14T18:26:48.564209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:27:58.179129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561521+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:48.561521+0000","last_peered":"2023-12-14T18:26:48.561521+0000","last_clean":"2023-12-14T18:26:48.561521+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:48.561521+0000","last_undegraded":"2023-12-14T18:26:48.561521+0000","last_fullsized":"2023-12-14T18:26:48.561521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:22.740418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566535+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:48.566535+0000","last_peered":"2023-12-14T18:26:48.566535+0000","last_clean":"2023-12-14T18:26:48.566535+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:48.566535+0000","last_undegraded":"2023-12-14T18:26:48.566535+0000","last_fullsized":"2023-12-14T18:26:48.566535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:54:48.048416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562125+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:48.562125+0000","last_peered":"2023-12-14T18:26:48.562125+0000","last_clean":"2023-12-14T18:26:48.562125+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:48.562125+0000","last_undegraded":"2023-12-14T18:26:48.562125+0000","last_fullsized":"2023-12-14T18:26:48.562125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:36:49.964293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561575+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:48.561575+0000","last_peered":"2023-12-14T18:26:48.561575+0000","last_clean":"2023-12-14T18:26:48.561575+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:48.561575+0000","last_undegraded":"2023-12-14T18:26:48.561575+0000","last_fullsized":"2023-12-14T18:26:48.561575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:20:52.709127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562632+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:48.562632+0000","last_peered":"2023-12-14T18:26:48.562632+0000","last_clean":"2023-12-14T18:26:48.562632+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:48.562632+0000","last_undegraded":"2023-12-14T18:26:48.562632+0000","last_fullsized":"2023-12-14T18:26:48.562632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:26.536039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568593+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:48.568593+0000","last_peered":"2023-12-14T18:26:48.568593+0000","last_clean":"2023-12-14T18:26:48.568593+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:48.568593+0000","last_undegraded":"2023-12-14T18:26:48.568593+0000","last_fullsized":"2023-12-14T18:26:48.568593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:59:51.600710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564002+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:48.564002+0000","last_peered":"2023-12-14T18:26:48.564002+0000","last_clean":"2023-12-14T18:26:48.564002+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:48.564002+0000","last_undegraded":"2023-12-14T18:26:48.564002+0000","last_fullsized":"2023-12-14T18:26:48.564002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:06.768114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559014+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:48.559014+0000","last_peered":"2023-12-14T18:26:48.559014+0000","last_clean":"2023-12-14T18:26:48.559014+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:48.559014+0000","last_undegraded":"2023-12-14T18:26:48.559014+0000","last_fullsized":"2023-12-14T18:26:48.559014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:32.285691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.577550+0000","last_change":"2023-12-14T18:26:48.577550+0000","last_active":"2023-12-14T18:26:48.577550+0000","last_peered":"2023-12-14T18:26:48.577550+0000","last_clean":"2023-12-14T18:26:48.577550+0000","last_became_active":"2023-12-14T18:26:48.569607+0000","last_became_peered":"2023-12-14T18:26:48.569607+0000","last_unstale":"2023-12-14T18:26:48.577550+0000","last_undegraded":"2023-12-14T18:26:48.577550+0000","last_fullsized":"2023-12-14T18:26:48.577550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:23.766724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559798+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:48.559798+0000","last_peered":"2023-12-14T18:26:48.559798+0000","last_clean":"2023-12-14T18:26:48.559798+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:48.559798+0000","last_undegraded":"2023-12-14T18:26:48.559798+0000","last_fullsized":"2023-12-14T18:26:48.559798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:29.491959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563921+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:26:48.563921+0000","last_peered":"2023-12-14T18:26:48.563921+0000","last_clean":"2023-12-14T18:26:48.563921+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:26:48.563921+0000","last_undegraded":"2023-12-14T18:26:48.563921+0000","last_fullsized":"2023-12-14T18:26:48.563921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:12.984642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563541+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:48.563541+0000","last_peered":"2023-12-14T18:26:48.563541+0000","last_clean":"2023-12-14T18:26:48.563541+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:48.563541+0000","last_undegraded":"2023-12-14T18:26:48.563541+0000","last_fullsized":"2023-12-14T18:26:48.563541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:56.278549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557904+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:48.557904+0000","last_peered":"2023-12-14T18:26:48.557904+0000","last_clean":"2023-12-14T18:26:48.557904+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:48.557904+0000","last_undegraded":"2023-12-14T18:26:48.557904+0000","last_fullsized":"2023-12-14T18:26:48.557904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:36:53.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563087+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:48.563087+0000","last_peered":"2023-12-14T18:26:48.563087+0000","last_clean":"2023-12-14T18:26:48.563087+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:48.563087+0000","last_undegraded":"2023-12-14T18:26:48.563087+0000","last_fullsized":"2023-12-14T18:26:48.563087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:19.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559959+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:48.559959+0000","last_peered":"2023-12-14T18:26:48.559959+0000","last_clean":"2023-12-14T18:26:48.559959+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:48.559959+0000","last_undegraded":"2023-12-14T18:26:48.559959+0000","last_fullsized":"2023-12-14T18:26:48.559959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:05.103613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566175+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:48.566175+0000","last_peered":"2023-12-14T18:26:48.566175+0000","last_clean":"2023-12-14T18:26:48.566175+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:48.566175+0000","last_undegraded":"2023-12-14T18:26:48.566175+0000","last_fullsized":"2023-12-14T18:26:48.566175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:21.311660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562945+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:26:48.562945+0000","last_peered":"2023-12-14T18:26:48.562945+0000","last_clean":"2023-12-14T18:26:48.562945+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:26:48.562945+0000","last_undegraded":"2023-12-14T18:26:48.562945+0000","last_fullsized":"2023-12-14T18:26:48.562945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:04:53.461295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558908+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:48.558908+0000","last_peered":"2023-12-14T18:26:48.558908+0000","last_clean":"2023-12-14T18:26:48.558908+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:48.558908+0000","last_undegraded":"2023-12-14T18:26:48.558908+0000","last_fullsized":"2023-12-14T18:26:48.558908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:04:46.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561591+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:48.561591+0000","last_peered":"2023-12-14T18:26:48.561591+0000","last_clean":"2023-12-14T18:26:48.561591+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:48.561591+0000","last_undegraded":"2023-12-14T18:26:48.561591+0000","last_fullsized":"2023-12-14T18:26:48.561591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:52:50.350437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562857+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:48.562857+0000","last_peered":"2023-12-14T18:26:48.562857+0000","last_clean":"2023-12-14T18:26:48.562857+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:48.562857+0000","last_undegraded":"2023-12-14T18:26:48.562857+0000","last_fullsized":"2023-12-14T18:26:48.562857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:22.082703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565524+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:26:48.565524+0000","last_peered":"2023-12-14T18:26:48.565524+0000","last_clean":"2023-12-14T18:26:48.565524+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:26:48.565524+0000","last_undegraded":"2023-12-14T18:26:48.565524+0000","last_fullsized":"2023-12-14T18:26:48.565524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:51:55.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559293+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:48.559293+0000","last_peered":"2023-12-14T18:26:48.559293+0000","last_clean":"2023-12-14T18:26:48.559293+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:48.559293+0000","last_undegraded":"2023-12-14T18:26:48.559293+0000","last_fullsized":"2023-12-14T18:26:48.559293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:17:28.674237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563263+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:48.563263+0000","last_peered":"2023-12-14T18:26:48.563263+0000","last_clean":"2023-12-14T18:26:48.563263+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:48.563263+0000","last_undegraded":"2023-12-14T18:26:48.563263+0000","last_fullsized":"2023-12-14T18:26:48.563263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:08.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.578387+0000","last_change":"2023-12-14T18:26:48.577964+0000","last_active":"2023-12-14T18:26:48.578387+0000","last_peered":"2023-12-14T18:26:48.578387+0000","last_clean":"2023-12-14T18:26:48.578387+0000","last_became_active":"2023-12-14T18:26:48.569637+0000","last_became_peered":"2023-12-14T18:26:48.569637+0000","last_unstale":"2023-12-14T18:26:48.578387+0000","last_undegraded":"2023-12-14T18:26:48.578387+0000","last_fullsized":"2023-12-14T18:26:48.578387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:31:31.614815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562048+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:48.562048+0000","last_peered":"2023-12-14T18:26:48.562048+0000","last_clean":"2023-12-14T18:26:48.562048+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:48.562048+0000","last_undegraded":"2023-12-14T18:26:48.562048+0000","last_fullsized":"2023-12-14T18:26:48.562048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:43.573208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559867+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:48.559867+0000","last_peered":"2023-12-14T18:26:48.559867+0000","last_clean":"2023-12-14T18:26:48.559867+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:48.559867+0000","last_undegraded":"2023-12-14T18:26:48.559867+0000","last_fullsized":"2023-12-14T18:26:48.559867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:15:04.165007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.574468+0000","last_change":"2023-12-14T18:26:48.574083+0000","last_active":"2023-12-14T18:26:48.574468+0000","last_peered":"2023-12-14T18:26:48.574468+0000","last_clean":"2023-12-14T18:26:48.574468+0000","last_became_active":"2023-12-14T18:26:48.565205+0000","last_became_peered":"2023-12-14T18:26:48.565205+0000","last_unstale":"2023-12-14T18:26:48.574468+0000","last_undegraded":"2023-12-14T18:26:48.574468+0000","last_fullsized":"2023-12-14T18:26:48.574468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:19.409191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562445+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:48.562445+0000","last_peered":"2023-12-14T18:26:48.562445+0000","last_clean":"2023-12-14T18:26:48.562445+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:48.562445+0000","last_undegraded":"2023-12-14T18:26:48.562445+0000","last_fullsized":"2023-12-14T18:26:48.562445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:37.001669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557585+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:48.557585+0000","last_peered":"2023-12-14T18:26:48.557585+0000","last_clean":"2023-12-14T18:26:48.557585+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:48.557585+0000","last_undegraded":"2023-12-14T18:26:48.557585+0000","last_fullsized":"2023-12-14T18:26:48.557585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:54.017877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564140+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:48.564140+0000","last_peered":"2023-12-14T18:26:48.564140+0000","last_clean":"2023-12-14T18:26:48.564140+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:48.564140+0000","last_undegraded":"2023-12-14T18:26:48.564140+0000","last_fullsized":"2023-12-14T18:26:48.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:46:47.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562523+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:48.562523+0000","last_peered":"2023-12-14T18:26:48.562523+0000","last_clean":"2023-12-14T18:26:48.562523+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:48.562523+0000","last_undegraded":"2023-12-14T18:26:48.562523+0000","last_fullsized":"2023-12-14T18:26:48.562523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:52:04.129230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566378+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:48.566378+0000","last_peered":"2023-12-14T18:26:48.566378+0000","last_clean":"2023-12-14T18:26:48.566378+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:48.566378+0000","last_undegraded":"2023-12-14T18:26:48.566378+0000","last_fullsized":"2023-12-14T18:26:48.566378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:08:14.934634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564930+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:48.564930+0000","last_peered":"2023-12-14T18:26:48.564930+0000","last_clean":"2023-12-14T18:26:48.564930+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:48.564930+0000","last_undegraded":"2023-12-14T18:26:48.564930+0000","last_fullsized":"2023-12-14T18:26:48.564930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:28.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562150+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:48.562150+0000","last_peered":"2023-12-14T18:26:48.562150+0000","last_clean":"2023-12-14T18:26:48.562150+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:48.562150+0000","last_undegraded":"2023-12-14T18:26:48.562150+0000","last_fullsized":"2023-12-14T18:26:48.562150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:43:39.281554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562009+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:48.562009+0000","last_peered":"2023-12-14T18:26:48.562009+0000","last_clean":"2023-12-14T18:26:48.562009+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:48.562009+0000","last_undegraded":"2023-12-14T18:26:48.562009+0000","last_fullsized":"2023-12-14T18:26:48.562009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:49:45.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564454+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:26:48.564454+0000","last_peered":"2023-12-14T18:26:48.564454+0000","last_clean":"2023-12-14T18:26:48.564454+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:26:48.564454+0000","last_undegraded":"2023-12-14T18:26:48.564454+0000","last_fullsized":"2023-12-14T18:26:48.564454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:16.913094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563425+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:26:48.563425+0000","last_peered":"2023-12-14T18:26:48.563425+0000","last_clean":"2023-12-14T18:26:48.563425+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:26:48.563425+0000","last_undegraded":"2023-12-14T18:26:48.563425+0000","last_fullsized":"2023-12-14T18:26:48.563425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:54.437297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557742+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:48.557742+0000","last_peered":"2023-12-14T18:26:48.557742+0000","last_clean":"2023-12-14T18:26:48.557742+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:48.557742+0000","last_undegraded":"2023-12-14T18:26:48.557742+0000","last_fullsized":"2023-12-14T18:26:48.557742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:35.196672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564064+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:48.564064+0000","last_peered":"2023-12-14T18:26:48.564064+0000","last_clean":"2023-12-14T18:26:48.564064+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:48.564064+0000","last_undegraded":"2023-12-14T18:26:48.564064+0000","last_fullsized":"2023-12-14T18:26:48.564064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:27.906838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562103+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:48.562103+0000","last_peered":"2023-12-14T18:26:48.562103+0000","last_clean":"2023-12-14T18:26:48.562103+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:48.562103+0000","last_undegraded":"2023-12-14T18:26:48.562103+0000","last_fullsized":"2023-12-14T18:26:48.562103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:44:51.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566964+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:26:48.566964+0000","last_peered":"2023-12-14T18:26:48.566964+0000","last_clean":"2023-12-14T18:26:48.566964+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:26:48.566964+0000","last_undegraded":"2023-12-14T18:26:48.566964+0000","last_fullsized":"2023-12-14T18:26:48.566964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:07:32.112350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559871+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:48.559871+0000","last_peered":"2023-12-14T18:26:48.559871+0000","last_clean":"2023-12-14T18:26:48.559871+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:48.559871+0000","last_undegraded":"2023-12-14T18:26:48.559871+0000","last_fullsized":"2023-12-14T18:26:48.559871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:36:21.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566829+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:48.566829+0000","last_peered":"2023-12-14T18:26:48.566829+0000","last_clean":"2023-12-14T18:26:48.566829+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:48.566829+0000","last_undegraded":"2023-12-14T18:26:48.566829+0000","last_fullsized":"2023-12-14T18:26:48.566829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:17.885326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559062+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:48.559062+0000","last_peered":"2023-12-14T18:26:48.559062+0000","last_clean":"2023-12-14T18:26:48.559062+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:48.559062+0000","last_undegraded":"2023-12-14T18:26:48.559062+0000","last_fullsized":"2023-12-14T18:26:48.559062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:28:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560244+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:48.560244+0000","last_peered":"2023-12-14T18:26:48.560244+0000","last_clean":"2023-12-14T18:26:48.560244+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:48.560244+0000","last_undegraded":"2023-12-14T18:26:48.560244+0000","last_fullsized":"2023-12-14T18:26:48.560244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:54.715861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562903+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:48.562903+0000","last_peered":"2023-12-14T18:26:48.562903+0000","last_clean":"2023-12-14T18:26:48.562903+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:48.562903+0000","last_undegraded":"2023-12-14T18:26:48.562903+0000","last_fullsized":"2023-12-14T18:26:48.562903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:54:48.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562082+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:48.562082+0000","last_peered":"2023-12-14T18:26:48.562082+0000","last_clean":"2023-12-14T18:26:48.562082+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:48.562082+0000","last_undegraded":"2023-12-14T18:26:48.562082+0000","last_fullsized":"2023-12-14T18:26:48.562082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:40:01.866310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63711,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":228,"num_read_kb":184,"num_write":157,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739028,"statfs":{"total":95995035648,"available":95988764672,"internally_reserved":0,"allocated":954368,"data_stored":484012,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.318},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.44}]},{"osd":5,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.47599999999999998}]}]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6580,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738572,"statfs":{"total":95995035648,"available":95988297728,"internally_reserved":0,"allocated":1093632,"data_stored":1130758,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.4960000000000004,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.54300000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80500000000000005}]}]},{"osd":9,"up_from":63,"seq":270582939677,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":974848,"data_stored":488344,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38,"15min":0.38},"max":{"1min":1.01,"5min":1.028,"15min":1.028},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.68300000000000005}]},{"osd":3,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.081,"15min":1.081},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.036,"15min":1.036},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1200000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.627}]},{"osd":8,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0189999999999999,"5min":1.123,"15min":1.123},"last":0.67000000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.754}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]}]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6460,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738692,"statfs":{"total":95995035648,"available":95988420608,"internally_reserved":0,"allocated":970752,"data_stored":484168,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.371,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44}]},{"osd":2,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.377},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.5}]},{"osd":4,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88500000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86199999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.55100000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.86899999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.628,"5min":0.502,"15min":0.502},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78400000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.73299999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":1079376,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.41099999999999998,"15min":0.39700000000000002},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.40799999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.752,"5min":1.8,"15min":1.8},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.36199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.59599999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84899999999999998,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.55000000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61699999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.751},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69299999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81000000000000005}]}]},{"osd":6,"up_from":43,"seq":184683593769,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6440,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738712,"statfs":{"total":95995035648,"available":95988441088,"internally_reserved":0,"allocated":950272,"data_stored":479424,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.42299999999999999,"15min":0.41499999999999998},"min":{"1min":0.30299999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.32100000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.77000000000000002}]},{"osd":1,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.58099999999999996,"15min":0.56899999999999995},"min":{"1min":0.42899999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59899999999999998,"15min":0.58699999999999997},"min":{"1min":0.52800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":1.155}]},{"osd":2,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.49199999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.42499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":1.016}]},{"osd":3,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.50600000000000001,"15min":0.503},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.83499999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.206},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56599999999999995,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81799999999999995,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.99099999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60299999999999998,"15min":0.58999999999999997},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.63900000000000001,"15min":0.628},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.68899999999999995,"15min":0.67900000000000005},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":1.117},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70999999999999996},"min":{"1min":0.622,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.95599999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.97499999999999998,"5min":1.127,"15min":1.127},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":1.0669999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.165},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.57899999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.127,"5min":1.133,"15min":1.133},"last":1.1870000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":1.036},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":1.1399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":539450,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.56000000000000005,"15min":0.51000000000000001},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.0800000000000001,"5min":5.0800000000000001,"15min":5.0800000000000001},"last":0.376}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.53000000000000003,"15min":0.47999999999999998},"min":{"1min":0.40200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53200000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.61099999999999999,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.3150000000000004,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.61299999999999999,"15min":0.54700000000000004},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.312,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70899999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.621},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61599999999999999,"15min":0.58199999999999996},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.496}]},{"osd":6,"last update":"Thu Dec 14 18:25:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.59999999999999998,"15min":0.57699999999999996},"min":{"1min":0.44900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.48199999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.437},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.32500000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:26:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.91400000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.59299999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.56399999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64100000000000001}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":480812,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55200000000000005,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55000000000000004,"15min":0.50600000000000001},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.376}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.81799999999999995}]},{"osd":3,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.747}]},{"osd":4,"last update":"Thu Dec 14 18:26:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":1.091,"15min":1.091},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.503}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98699999999999999,"5min":1.079,"15min":1.079},"last":0.72799999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.82499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73799999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411384,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7540,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737612,"statfs":{"total":95995035648,"available":95987314688,"internally_reserved":0,"allocated":1028096,"data_stored":539287,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.484,"15min":1.484},"last":0.624}]},{"osd":1,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.52300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71699999999999997,"5min":0.75,"15min":0.75},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Dec 14 18:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81000000000000005,"5min":1.514,"15min":1.514},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63}]},{"osd":8,"last update":"Thu Dec 14 18:26:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61399999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215155,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":958464,"data_stored":479345,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38200000000000001},"min":{"1min":0.36499999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.40899999999999997},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.36699999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.68200000000000005,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.44}]},{"osd":7,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.80000000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":6.7249999999999996,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":0.502}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.86299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986224,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":484990,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.29099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.442},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71799999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":1.121,"15min":1.121},"last":0.39800000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58899999999999997,"15min":0.57999999999999996},"min":{"1min":0.42899999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.879,"15min":0.879},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.63100000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":1.161,"15min":1.161},"last":0.59299999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.501}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.55200000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":1024000,"data_stored":1075285,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63800000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.434},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66600000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.314,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":4.7290000000000001,"5min":4.7290000000000001,"15min":4.7290000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.621,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.391,"5min":6.391,"15min":6.391},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46899999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.443,"5min":0.309,"15min":0.309},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.754},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.66500000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:27:01.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:27:01.684 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:27:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:01 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:01.993 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:27:01.994 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:01.994 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:27:01.994 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:27:01.994 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:01 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:27:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: from='mgr.14152 172.21.15.31:0/2523642806' entity='mgr.x' 2023-12-14T18:27:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:27:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:27:02.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:01 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.749+0000 7fb9377ee700 1 -- 172.21.15.31:0/3124822525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930101e50 msgr2=0x7fb93010a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.749+0000 7fb9377ee700 1 --2- 172.21.15.31:0/3124822525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930101e50 0x7fb93010a350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb920009990 tx=0x7fb920009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 -- 172.21.15.31:0/3124822525 shutdown_connections 2023-12-14T18:27:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 --2- 172.21.15.31:0/3124822525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb93010a890 0x7fb93010ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 --2- 172.21.15.31:0/3124822525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb930101540 0x7fb930101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 --2- 172.21.15.31:0/3124822525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb930101e50 0x7fb93010a350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 -- 172.21.15.31:0/3124822525 >> 172.21.15.31:0/3124822525 conn(0x7fb9300770a0 msgr2=0x7fb9300774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 -- 172.21.15.31:0/3124822525 shutdown_connections 2023-12-14T18:27:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.750+0000 7fb9377ee700 1 -- 172.21.15.31:0/3124822525 wait complete. 2023-12-14T18:27:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 Processor -- start 2023-12-14T18:27:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 -- start start 2023-12-14T18:27:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb930101540 0x7fb9300ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb930101e50 0x7fb9300ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 0x7fb9300ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301a4850 con 0x7fb93010a890 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301a49c0 con 0x7fb930101e50 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb9377ee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301a4b80 con 0x7fb930101540 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 0x7fb9300ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 0x7fb9300ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49124/0 (socket says 172.21.15.31:49124) 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 -- 172.21.15.31:0/630740922 learned_addr learned my addr 172.21.15.31:0/630740922 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb93558a700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb930101540 0x7fb9300ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb934d89700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb930101e50 0x7fb9300ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 -- 172.21.15.31:0/630740922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb930101540 msgr2=0x7fb9300ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb930101540 0x7fb9300ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 -- 172.21.15.31:0/630740922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb930101e50 msgr2=0x7fb9300ff6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb930101e50 0x7fb9300ff6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.751+0000 7fb935d8b700 1 -- 172.21.15.31:0/630740922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9200095f0 con 0x7fb93010a890 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.752+0000 7fb935d8b700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 0x7fb9300ffc60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb928007ee0 tx=0x7fb92800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.752+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928017070 con 0x7fb93010a890 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.752+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb930110310 con 0x7fb93010a890 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.752+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92800f040 con 0x7fb93010a890 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.752+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928005580 con 0x7fb93010a890 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.752+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb930110180 con 0x7fb93010a890 2023-12-14T18:27:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.753+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9301104a0 con 0x7fb93010a890 2023-12-14T18:27:02.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.755+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb92800db20 con 0x7fb93010a890 2023-12-14T18:27:02.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.756+0000 7fb9267fc700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb91c0a11e0 0x7fb91c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:02.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.756+0000 7fb93558a700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb91c0a11e0 0x7fb91c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:02.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.756+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb9280ccaf0 con 0x7fb93010a890 2023-12-14T18:27:02.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.756+0000 7fb93558a700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb91c0a11e0 0x7fb91c0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005950 tx=0x7fb92c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:02.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.759+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb928095d70 con 0x7fb93010a890 2023-12-14T18:27:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.966+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9300fc420 con 0x7fb91c0a11e0 2023-12-14T18:27:02.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.979+0000 7fb9267fc700 1 -- 172.21.15.31:0/630740922 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+425778 (secure 0 0 0) 0x7fb9300fc420 con 0x7fb91c0a11e0 2023-12-14T18:27:02.985 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb91c0a11e0 msgr2=0x7fb91c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb91c0a11e0 0x7fb91c0a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb92c005950 tx=0x7fb92c00b410 comp rx=0 tx=0).stop 2023-12-14T18:27:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 msgr2=0x7fb9300ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 0x7fb9300ffc60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb928007ee0 tx=0x7fb92800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 shutdown_connections 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb93010a890 0x7fb9300ffc60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb91c0a11e0 0x7fb91c0a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb930101540 0x7fb9300ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 --2- 172.21.15.31:0/630740922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb930101e50 0x7fb9300ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 >> 172.21.15.31:0/630740922 conn(0x7fb9300770a0 msgr2=0x7fb9300fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 shutdown_connections 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:02.983+0000 7fb9377ee700 1 -- 172.21.15.31:0/630740922 wait complete. 2023-12-14T18:27:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:27:03.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:03 smithi204 ceph-mon[43866]: pgmap v264: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 128 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:27:03.369 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:03 smithi031 ceph-mon[42085]: pgmap v264: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 128 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:27:03.414 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2023-12-14T18:27:01.488247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654325,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":374,"num_read_kb":308,"num_write":340,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1182282,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80800,"kb_used_data":11648,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124861024,"statfs":{"total":1151940427776,"available":1151857688576,"internally_reserved":0,"allocated":11927552,"data_stored":7745251,"data_compressed":319020,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":34,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003383"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562794+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:48.562794+0000","last_peered":"2023-12-14T18:26:48.562794+0000","last_clean":"2023-12-14T18:26:48.562794+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:48.562794+0000","last_undegraded":"2023-12-14T18:26:48.562794+0000","last_fullsized":"2023-12-14T18:26:48.562794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:33:46.076524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557880+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:48.557880+0000","last_peered":"2023-12-14T18:26:48.557880+0000","last_clean":"2023-12-14T18:26:48.557880+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:48.557880+0000","last_undegraded":"2023-12-14T18:26:48.557880+0000","last_fullsized":"2023-12-14T18:26:48.557880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:10.369082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557355+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:48.557355+0000","last_peered":"2023-12-14T18:26:48.557355+0000","last_clean":"2023-12-14T18:26:48.557355+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:48.557355+0000","last_undegraded":"2023-12-14T18:26:48.557355+0000","last_fullsized":"2023-12-14T18:26:48.557355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:23.331965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563177+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:48.563177+0000","last_peered":"2023-12-14T18:26:48.563177+0000","last_clean":"2023-12-14T18:26:48.563177+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:48.563177+0000","last_undegraded":"2023-12-14T18:26:48.563177+0000","last_fullsized":"2023-12-14T18:26:48.563177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:01.529418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567124+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:48.567124+0000","last_peered":"2023-12-14T18:26:48.567124+0000","last_clean":"2023-12-14T18:26:48.567124+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:48.567124+0000","last_undegraded":"2023-12-14T18:26:48.567124+0000","last_fullsized":"2023-12-14T18:26:48.567124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:53:32.988966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647501+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:26:48.647501+0000","last_peered":"2023-12-14T18:26:48.647501+0000","last_clean":"2023-12-14T18:26:48.647501+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:26:48.647501+0000","last_undegraded":"2023-12-14T18:26:48.647501+0000","last_fullsized":"2023-12-14T18:26:48.647501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:00:59.230921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561700+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:48.561700+0000","last_peered":"2023-12-14T18:26:48.561700+0000","last_clean":"2023-12-14T18:26:48.561700+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:48.561700+0000","last_undegraded":"2023-12-14T18:26:48.561700+0000","last_fullsized":"2023-12-14T18:26:48.561700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:47.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559287+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:48.559287+0000","last_peered":"2023-12-14T18:26:48.559287+0000","last_clean":"2023-12-14T18:26:48.559287+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:48.559287+0000","last_undegraded":"2023-12-14T18:26:48.559287+0000","last_fullsized":"2023-12-14T18:26:48.559287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:13:56.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560906+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:48.560906+0000","last_peered":"2023-12-14T18:26:48.560906+0000","last_clean":"2023-12-14T18:26:48.560906+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:48.560906+0000","last_undegraded":"2023-12-14T18:26:48.560906+0000","last_fullsized":"2023-12-14T18:26:48.560906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:38:42.914156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647960+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:26:48.647960+0000","last_peered":"2023-12-14T18:26:48.647960+0000","last_clean":"2023-12-14T18:26:48.647960+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:26:48.647960+0000","last_undegraded":"2023-12-14T18:26:48.647960+0000","last_fullsized":"2023-12-14T18:26:48.647960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:09:41.346067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562267+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:48.562267+0000","last_peered":"2023-12-14T18:26:48.562267+0000","last_clean":"2023-12-14T18:26:48.562267+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:48.562267+0000","last_undegraded":"2023-12-14T18:26:48.562267+0000","last_fullsized":"2023-12-14T18:26:48.562267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:26.374795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563022+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:48.563022+0000","last_peered":"2023-12-14T18:26:48.563022+0000","last_clean":"2023-12-14T18:26:48.563022+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:48.563022+0000","last_undegraded":"2023-12-14T18:26:48.563022+0000","last_fullsized":"2023-12-14T18:26:48.563022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:38.171386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563106+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:48.563106+0000","last_peered":"2023-12-14T18:26:48.563106+0000","last_clean":"2023-12-14T18:26:48.563106+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:48.563106+0000","last_undegraded":"2023-12-14T18:26:48.563106+0000","last_fullsized":"2023-12-14T18:26:48.563106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:18:44.617868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557419+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:48.557419+0000","last_peered":"2023-12-14T18:26:48.557419+0000","last_clean":"2023-12-14T18:26:48.557419+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:48.557419+0000","last_undegraded":"2023-12-14T18:26:48.557419+0000","last_fullsized":"2023-12-14T18:26:48.557419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:04:10.141803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558264+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:48.558264+0000","last_peered":"2023-12-14T18:26:48.558264+0000","last_clean":"2023-12-14T18:26:48.558264+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:48.558264+0000","last_undegraded":"2023-12-14T18:26:48.558264+0000","last_fullsized":"2023-12-14T18:26:48.558264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:53.022658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559788+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:48.559788+0000","last_peered":"2023-12-14T18:26:48.559788+0000","last_clean":"2023-12-14T18:26:48.559788+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:48.559788+0000","last_undegraded":"2023-12-14T18:26:48.559788+0000","last_fullsized":"2023-12-14T18:26:48.559788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:18.996023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562659+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:48.562659+0000","last_peered":"2023-12-14T18:26:48.562659+0000","last_clean":"2023-12-14T18:26:48.562659+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:48.562659+0000","last_undegraded":"2023-12-14T18:26:48.562659+0000","last_fullsized":"2023-12-14T18:26:48.562659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:00:27.549342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562608+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:26:48.562608+0000","last_peered":"2023-12-14T18:26:48.562608+0000","last_clean":"2023-12-14T18:26:48.562608+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:26:48.562608+0000","last_undegraded":"2023-12-14T18:26:48.562608+0000","last_fullsized":"2023-12-14T18:26:48.562608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:06:20.688646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566623+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:26:48.566623+0000","last_peered":"2023-12-14T18:26:48.566623+0000","last_clean":"2023-12-14T18:26:48.566623+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:26:48.566623+0000","last_undegraded":"2023-12-14T18:26:48.566623+0000","last_fullsized":"2023-12-14T18:26:48.566623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:58.296365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564244+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:26:48.564244+0000","last_peered":"2023-12-14T18:26:48.564244+0000","last_clean":"2023-12-14T18:26:48.564244+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:26:48.564244+0000","last_undegraded":"2023-12-14T18:26:48.564244+0000","last_fullsized":"2023-12-14T18:26:48.564244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:23.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563082+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:26:48.563082+0000","last_peered":"2023-12-14T18:26:48.563082+0000","last_clean":"2023-12-14T18:26:48.563082+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:26:48.563082+0000","last_undegraded":"2023-12-14T18:26:48.563082+0000","last_fullsized":"2023-12-14T18:26:48.563082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:35:18.652626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559422+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:48.559422+0000","last_peered":"2023-12-14T18:26:48.559422+0000","last_clean":"2023-12-14T18:26:48.559422+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:48.559422+0000","last_undegraded":"2023-12-14T18:26:48.559422+0000","last_fullsized":"2023-12-14T18:26:48.559422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:31.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562609+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:48.562609+0000","last_peered":"2023-12-14T18:26:48.562609+0000","last_clean":"2023-12-14T18:26:48.562609+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:48.562609+0000","last_undegraded":"2023-12-14T18:26:48.562609+0000","last_fullsized":"2023-12-14T18:26:48.562609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:43.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559749+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:48.559749+0000","last_peered":"2023-12-14T18:26:48.559749+0000","last_clean":"2023-12-14T18:26:48.559749+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:48.559749+0000","last_undegraded":"2023-12-14T18:26:48.559749+0000","last_fullsized":"2023-12-14T18:26:48.559749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:43.650498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558888+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:48.558888+0000","last_peered":"2023-12-14T18:26:48.558888+0000","last_clean":"2023-12-14T18:26:48.558888+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:48.558888+0000","last_undegraded":"2023-12-14T18:26:48.558888+0000","last_fullsized":"2023-12-14T18:26:48.558888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:50.284685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564072+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:48.564072+0000","last_peered":"2023-12-14T18:26:48.564072+0000","last_clean":"2023-12-14T18:26:48.564072+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:48.564072+0000","last_undegraded":"2023-12-14T18:26:48.564072+0000","last_fullsized":"2023-12-14T18:26:48.564072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:29:21.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568778+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:48.568778+0000","last_peered":"2023-12-14T18:26:48.568778+0000","last_clean":"2023-12-14T18:26:48.568778+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:48.568778+0000","last_undegraded":"2023-12-14T18:26:48.568778+0000","last_fullsized":"2023-12-14T18:26:48.568778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:38.658671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560627+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:48.560627+0000","last_peered":"2023-12-14T18:26:48.560627+0000","last_clean":"2023-12-14T18:26:48.560627+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:48.560627+0000","last_undegraded":"2023-12-14T18:26:48.560627+0000","last_fullsized":"2023-12-14T18:26:48.560627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:54:13.436594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562615+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:48.562615+0000","last_peered":"2023-12-14T18:26:48.562615+0000","last_clean":"2023-12-14T18:26:48.562615+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:48.562615+0000","last_undegraded":"2023-12-14T18:26:48.562615+0000","last_fullsized":"2023-12-14T18:26:48.562615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:22:13.915160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568436+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:48.568436+0000","last_peered":"2023-12-14T18:26:48.568436+0000","last_clean":"2023-12-14T18:26:48.568436+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:48.568436+0000","last_undegraded":"2023-12-14T18:26:48.568436+0000","last_fullsized":"2023-12-14T18:26:48.568436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:51:57.429285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563999+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:48.563999+0000","last_peered":"2023-12-14T18:26:48.563999+0000","last_clean":"2023-12-14T18:26:48.563999+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:48.563999+0000","last_undegraded":"2023-12-14T18:26:48.563999+0000","last_fullsized":"2023-12-14T18:26:48.563999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:21:53.454334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564084+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:26:48.564084+0000","last_peered":"2023-12-14T18:26:48.564084+0000","last_clean":"2023-12-14T18:26:48.564084+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:26:48.564084+0000","last_undegraded":"2023-12-14T18:26:48.564084+0000","last_fullsized":"2023-12-14T18:26:48.564084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:33.154296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:55.580449+0000","last_change":"2023-12-14T18:26:48.574808+0000","last_active":"2023-12-14T18:26:55.580449+0000","last_peered":"2023-12-14T18:26:55.580449+0000","last_clean":"2023-12-14T18:26:55.580449+0000","last_became_active":"2023-12-14T18:26:48.568640+0000","last_became_peered":"2023-12-14T18:26:48.568640+0000","last_unstale":"2023-12-14T18:26:55.580449+0000","last_undegraded":"2023-12-14T18:26:55.580449+0000","last_fullsized":"2023-12-14T18:26:55.580449+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:16.685666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566784+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:48.566784+0000","last_peered":"2023-12-14T18:26:48.566784+0000","last_clean":"2023-12-14T18:26:48.566784+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:48.566784+0000","last_undegraded":"2023-12-14T18:26:48.566784+0000","last_fullsized":"2023-12-14T18:26:48.566784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.326220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.573798+0000","last_change":"2023-12-14T18:26:48.573798+0000","last_active":"2023-12-14T18:26:48.573798+0000","last_peered":"2023-12-14T18:26:48.573798+0000","last_clean":"2023-12-14T18:26:48.573798+0000","last_became_active":"2023-12-14T18:26:48.573308+0000","last_became_peered":"2023-12-14T18:26:48.573308+0000","last_unstale":"2023-12-14T18:26:48.573798+0000","last_undegraded":"2023-12-14T18:26:48.573798+0000","last_fullsized":"2023-12-14T18:26:48.573798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:28:46.273622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647410+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:48.647410+0000","last_peered":"2023-12-14T18:26:48.647410+0000","last_clean":"2023-12-14T18:26:48.647410+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:48.647410+0000","last_undegraded":"2023-12-14T18:26:48.647410+0000","last_fullsized":"2023-12-14T18:26:48.647410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:05:15.020350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564341+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:48.564341+0000","last_peered":"2023-12-14T18:26:48.564341+0000","last_clean":"2023-12-14T18:26:48.564341+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:48.564341+0000","last_undegraded":"2023-12-14T18:26:48.564341+0000","last_fullsized":"2023-12-14T18:26:48.564341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:43.990428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570306+0000","last_change":"2023-12-14T18:26:48.570306+0000","last_active":"2023-12-14T18:26:48.570306+0000","last_peered":"2023-12-14T18:26:48.570306+0000","last_clean":"2023-12-14T18:26:48.570306+0000","last_became_active":"2023-12-14T18:26:48.569803+0000","last_became_peered":"2023-12-14T18:26:48.569803+0000","last_unstale":"2023-12-14T18:26:48.570306+0000","last_undegraded":"2023-12-14T18:26:48.570306+0000","last_fullsized":"2023-12-14T18:26:48.570306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:33.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562436+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:48.562436+0000","last_peered":"2023-12-14T18:26:48.562436+0000","last_clean":"2023-12-14T18:26:48.562436+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:48.562436+0000","last_undegraded":"2023-12-14T18:26:48.562436+0000","last_fullsized":"2023-12-14T18:26:48.562436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:53.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559154+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:48.559154+0000","last_peered":"2023-12-14T18:26:48.559154+0000","last_clean":"2023-12-14T18:26:48.559154+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:48.559154+0000","last_undegraded":"2023-12-14T18:26:48.559154+0000","last_fullsized":"2023-12-14T18:26:48.559154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:49.740844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557724+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:48.557724+0000","last_peered":"2023-12-14T18:26:48.557724+0000","last_clean":"2023-12-14T18:26:48.557724+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:48.557724+0000","last_undegraded":"2023-12-14T18:26:48.557724+0000","last_fullsized":"2023-12-14T18:26:48.557724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:44:17.333118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567219+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:48.567219+0000","last_peered":"2023-12-14T18:26:48.567219+0000","last_clean":"2023-12-14T18:26:48.567219+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:48.567219+0000","last_undegraded":"2023-12-14T18:26:48.567219+0000","last_fullsized":"2023-12-14T18:26:48.567219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:45:09.792477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561943+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:26:48.561943+0000","last_peered":"2023-12-14T18:26:48.561943+0000","last_clean":"2023-12-14T18:26:48.561943+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:26:48.561943+0000","last_undegraded":"2023-12-14T18:26:48.561943+0000","last_fullsized":"2023-12-14T18:26:48.561943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:21:43.141339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559411+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:48.559411+0000","last_peered":"2023-12-14T18:26:48.559411+0000","last_clean":"2023-12-14T18:26:48.559411+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:48.559411+0000","last_undegraded":"2023-12-14T18:26:48.559411+0000","last_fullsized":"2023-12-14T18:26:48.559411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:36.307493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:57.582575+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:26:57.582575+0000","last_peered":"2023-12-14T18:26:57.582575+0000","last_clean":"2023-12-14T18:26:57.582575+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:26:57.582575+0000","last_undegraded":"2023-12-14T18:26:57.582575+0000","last_fullsized":"2023-12-14T18:26:57.582575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:20:23.673760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560703+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:48.560703+0000","last_peered":"2023-12-14T18:26:48.560703+0000","last_clean":"2023-12-14T18:26:48.560703+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:48.560703+0000","last_undegraded":"2023-12-14T18:26:48.560703+0000","last_fullsized":"2023-12-14T18:26:48.560703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:33:22.836099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561628+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:26:48.561628+0000","last_peered":"2023-12-14T18:26:48.561628+0000","last_clean":"2023-12-14T18:26:48.561628+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:26:48.561628+0000","last_undegraded":"2023-12-14T18:26:48.561628+0000","last_fullsized":"2023-12-14T18:26:48.561628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:47.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561906+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:48.561906+0000","last_peered":"2023-12-14T18:26:48.561906+0000","last_clean":"2023-12-14T18:26:48.561906+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:48.561906+0000","last_undegraded":"2023-12-14T18:26:48.561906+0000","last_fullsized":"2023-12-14T18:26:48.561906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:00.359853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558750+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:48.558750+0000","last_peered":"2023-12-14T18:26:48.558750+0000","last_clean":"2023-12-14T18:26:48.558750+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:48.558750+0000","last_undegraded":"2023-12-14T18:26:48.558750+0000","last_fullsized":"2023-12-14T18:26:48.558750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:49.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568565+0000","last_change":"2023-12-14T18:26:48.568565+0000","last_active":"2023-12-14T18:26:48.568565+0000","last_peered":"2023-12-14T18:26:48.568565+0000","last_clean":"2023-12-14T18:26:48.568565+0000","last_became_active":"2023-12-14T18:26:48.568090+0000","last_became_peered":"2023-12-14T18:26:48.568090+0000","last_unstale":"2023-12-14T18:26:48.568565+0000","last_undegraded":"2023-12-14T18:26:48.568565+0000","last_fullsized":"2023-12-14T18:26:48.568565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:14:38.160855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563856+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:48.563856+0000","last_peered":"2023-12-14T18:26:48.563856+0000","last_clean":"2023-12-14T18:26:48.563856+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:48.563856+0000","last_undegraded":"2023-12-14T18:26:48.563856+0000","last_fullsized":"2023-12-14T18:26:48.563856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:12.189176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560853+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:48.560853+0000","last_peered":"2023-12-14T18:26:48.560853+0000","last_clean":"2023-12-14T18:26:48.560853+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:48.560853+0000","last_undegraded":"2023-12-14T18:26:48.560853+0000","last_fullsized":"2023-12-14T18:26:48.560853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:55:18.680896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562308+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:26:48.562308+0000","last_peered":"2023-12-14T18:26:48.562308+0000","last_clean":"2023-12-14T18:26:48.562308+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:26:48.562308+0000","last_undegraded":"2023-12-14T18:26:48.562308+0000","last_fullsized":"2023-12-14T18:26:48.562308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:05:51.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":139,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:58.583889+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:26:58.583889+0000","last_peered":"2023-12-14T18:26:58.583889+0000","last_clean":"2023-12-14T18:26:58.583889+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:26:58.583889+0000","last_undegraded":"2023-12-14T18:26:58.583889+0000","last_fullsized":"2023-12-14T18:26:58.583889+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:33.081169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59807,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":98,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568542+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:48.568542+0000","last_peered":"2023-12-14T18:26:48.568542+0000","last_clean":"2023-12-14T18:26:48.568542+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:48.568542+0000","last_undegraded":"2023-12-14T18:26:48.568542+0000","last_fullsized":"2023-12-14T18:26:48.568542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:30.839862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564457+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:48.564457+0000","last_peered":"2023-12-14T18:26:48.564457+0000","last_clean":"2023-12-14T18:26:48.564457+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:48.564457+0000","last_undegraded":"2023-12-14T18:26:48.564457+0000","last_fullsized":"2023-12-14T18:26:48.564457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:27.839169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565422+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:48.565422+0000","last_peered":"2023-12-14T18:26:48.565422+0000","last_clean":"2023-12-14T18:26:48.565422+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:48.565422+0000","last_undegraded":"2023-12-14T18:26:48.565422+0000","last_fullsized":"2023-12-14T18:26:48.565422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:34.931404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562476+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:48.562476+0000","last_peered":"2023-12-14T18:26:48.562476+0000","last_clean":"2023-12-14T18:26:48.562476+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:48.562476+0000","last_undegraded":"2023-12-14T18:26:48.562476+0000","last_fullsized":"2023-12-14T18:26:48.562476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:45.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561959+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:48.561959+0000","last_peered":"2023-12-14T18:26:48.561959+0000","last_clean":"2023-12-14T18:26:48.561959+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:48.561959+0000","last_undegraded":"2023-12-14T18:26:48.561959+0000","last_fullsized":"2023-12-14T18:26:48.561959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:54.542913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565425+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:48.565425+0000","last_peered":"2023-12-14T18:26:48.565425+0000","last_clean":"2023-12-14T18:26:48.565425+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:48.565425+0000","last_undegraded":"2023-12-14T18:26:48.565425+0000","last_fullsized":"2023-12-14T18:26:48.565425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:46.938078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566091+0000","last_change":"2023-12-14T18:26:48.566091+0000","last_active":"2023-12-14T18:26:48.566091+0000","last_peered":"2023-12-14T18:26:48.566091+0000","last_clean":"2023-12-14T18:26:48.566091+0000","last_became_active":"2023-12-14T18:26:48.565714+0000","last_became_peered":"2023-12-14T18:26:48.565714+0000","last_unstale":"2023-12-14T18:26:48.566091+0000","last_undegraded":"2023-12-14T18:26:48.566091+0000","last_fullsized":"2023-12-14T18:26:48.566091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:48.478859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562658+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:26:48.562658+0000","last_peered":"2023-12-14T18:26:48.562658+0000","last_clean":"2023-12-14T18:26:48.562658+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:26:48.562658+0000","last_undegraded":"2023-12-14T18:26:48.562658+0000","last_fullsized":"2023-12-14T18:26:48.562658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:22.379528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:57.665148+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:26:57.665148+0000","last_peered":"2023-12-14T18:26:57.665148+0000","last_clean":"2023-12-14T18:26:57.665148+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:26:57.665148+0000","last_undegraded":"2023-12-14T18:26:57.665148+0000","last_fullsized":"2023-12-14T18:26:57.665148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:26:13.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561690+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:48.561690+0000","last_peered":"2023-12-14T18:26:48.561690+0000","last_clean":"2023-12-14T18:26:48.561690+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:48.561690+0000","last_undegraded":"2023-12-14T18:26:48.561690+0000","last_fullsized":"2023-12-14T18:26:48.561690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:10:17.702084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.649376+0000","last_change":"2023-12-14T18:26:48.649376+0000","last_active":"2023-12-14T18:26:48.649376+0000","last_peered":"2023-12-14T18:26:48.649376+0000","last_clean":"2023-12-14T18:26:48.649376+0000","last_became_active":"2023-12-14T18:26:48.648882+0000","last_became_peered":"2023-12-14T18:26:48.648882+0000","last_unstale":"2023-12-14T18:26:48.649376+0000","last_undegraded":"2023-12-14T18:26:48.649376+0000","last_fullsized":"2023-12-14T18:26:48.649376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:12.045140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566931+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:48.566931+0000","last_peered":"2023-12-14T18:26:48.566931+0000","last_clean":"2023-12-14T18:26:48.566931+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:48.566931+0000","last_undegraded":"2023-12-14T18:26:48.566931+0000","last_fullsized":"2023-12-14T18:26:48.566931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:47.382142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.572300+0000","last_change":"2023-12-14T18:26:48.572300+0000","last_active":"2023-12-14T18:26:48.572300+0000","last_peered":"2023-12-14T18:26:48.572300+0000","last_clean":"2023-12-14T18:26:48.572300+0000","last_became_active":"2023-12-14T18:26:48.571895+0000","last_became_peered":"2023-12-14T18:26:48.571895+0000","last_unstale":"2023-12-14T18:26:48.572300+0000","last_undegraded":"2023-12-14T18:26:48.572300+0000","last_fullsized":"2023-12-14T18:26:48.572300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:40:54.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566446+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:48.566446+0000","last_peered":"2023-12-14T18:26:48.566446+0000","last_clean":"2023-12-14T18:26:48.566446+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:48.566446+0000","last_undegraded":"2023-12-14T18:26:48.566446+0000","last_fullsized":"2023-12-14T18:26:48.566446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:44.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570871+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:48.570871+0000","last_peered":"2023-12-14T18:26:48.570871+0000","last_clean":"2023-12-14T18:26:48.570871+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:48.570871+0000","last_undegraded":"2023-12-14T18:26:48.570871+0000","last_fullsized":"2023-12-14T18:26:48.570871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:28.018194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561975+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:48.561975+0000","last_peered":"2023-12-14T18:26:48.561975+0000","last_clean":"2023-12-14T18:26:48.561975+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:48.561975+0000","last_undegraded":"2023-12-14T18:26:48.561975+0000","last_fullsized":"2023-12-14T18:26:48.561975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:37.576788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566649+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:48.566649+0000","last_peered":"2023-12-14T18:26:48.566649+0000","last_clean":"2023-12-14T18:26:48.566649+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:48.566649+0000","last_undegraded":"2023-12-14T18:26:48.566649+0000","last_fullsized":"2023-12-14T18:26:48.566649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:37.853928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562990+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:26:48.562990+0000","last_peered":"2023-12-14T18:26:48.562990+0000","last_clean":"2023-12-14T18:26:48.562990+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:26:48.562990+0000","last_undegraded":"2023-12-14T18:26:48.562990+0000","last_fullsized":"2023-12-14T18:26:48.562990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:32.215966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647772+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:26:48.647772+0000","last_peered":"2023-12-14T18:26:48.647772+0000","last_clean":"2023-12-14T18:26:48.647772+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:26:48.647772+0000","last_undegraded":"2023-12-14T18:26:48.647772+0000","last_fullsized":"2023-12-14T18:26:48.647772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:51:12.578633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563700+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:48.563700+0000","last_peered":"2023-12-14T18:26:48.563700+0000","last_clean":"2023-12-14T18:26:48.563700+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:48.563700+0000","last_undegraded":"2023-12-14T18:26:48.563700+0000","last_fullsized":"2023-12-14T18:26:48.563700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:58.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561937+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:48.561937+0000","last_peered":"2023-12-14T18:26:48.561937+0000","last_clean":"2023-12-14T18:26:48.561937+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:48.561937+0000","last_undegraded":"2023-12-14T18:26:48.561937+0000","last_fullsized":"2023-12-14T18:26:48.561937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:21.783482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647707+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:26:48.647707+0000","last_peered":"2023-12-14T18:26:48.647707+0000","last_clean":"2023-12-14T18:26:48.647707+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:26:48.647707+0000","last_undegraded":"2023-12-14T18:26:48.647707+0000","last_fullsized":"2023-12-14T18:26:48.647707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:26:41.629317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563228+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:48.563228+0000","last_peered":"2023-12-14T18:26:48.563228+0000","last_clean":"2023-12-14T18:26:48.563228+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:48.563228+0000","last_undegraded":"2023-12-14T18:26:48.563228+0000","last_fullsized":"2023-12-14T18:26:48.563228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:41:45.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566971+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:48.566971+0000","last_peered":"2023-12-14T18:26:48.566971+0000","last_clean":"2023-12-14T18:26:48.566971+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:48.566971+0000","last_undegraded":"2023-12-14T18:26:48.566971+0000","last_fullsized":"2023-12-14T18:26:48.566971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:54.638247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:59.667273+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:26:59.667273+0000","last_peered":"2023-12-14T18:26:59.667273+0000","last_clean":"2023-12-14T18:26:59.667273+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:26:59.667273+0000","last_undegraded":"2023-12-14T18:26:59.667273+0000","last_fullsized":"2023-12-14T18:26:59.667273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:07:36.295981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562086+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:48.562086+0000","last_peered":"2023-12-14T18:26:48.562086+0000","last_clean":"2023-12-14T18:26:48.562086+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:48.562086+0000","last_undegraded":"2023-12-14T18:26:48.562086+0000","last_fullsized":"2023-12-14T18:26:48.562086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:00.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559419+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:48.559419+0000","last_peered":"2023-12-14T18:26:48.559419+0000","last_clean":"2023-12-14T18:26:48.559419+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:48.559419+0000","last_undegraded":"2023-12-14T18:26:48.559419+0000","last_fullsized":"2023-12-14T18:26:48.559419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:01.825894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564209+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:48.564209+0000","last_peered":"2023-12-14T18:26:48.564209+0000","last_clean":"2023-12-14T18:26:48.564209+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:48.564209+0000","last_undegraded":"2023-12-14T18:26:48.564209+0000","last_fullsized":"2023-12-14T18:26:48.564209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:27:58.179129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561521+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:48.561521+0000","last_peered":"2023-12-14T18:26:48.561521+0000","last_clean":"2023-12-14T18:26:48.561521+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:48.561521+0000","last_undegraded":"2023-12-14T18:26:48.561521+0000","last_fullsized":"2023-12-14T18:26:48.561521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:22.740418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566535+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:48.566535+0000","last_peered":"2023-12-14T18:26:48.566535+0000","last_clean":"2023-12-14T18:26:48.566535+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:48.566535+0000","last_undegraded":"2023-12-14T18:26:48.566535+0000","last_fullsized":"2023-12-14T18:26:48.566535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:54:48.048416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562125+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:48.562125+0000","last_peered":"2023-12-14T18:26:48.562125+0000","last_clean":"2023-12-14T18:26:48.562125+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:48.562125+0000","last_undegraded":"2023-12-14T18:26:48.562125+0000","last_fullsized":"2023-12-14T18:26:48.562125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:36:49.964293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561575+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:48.561575+0000","last_peered":"2023-12-14T18:26:48.561575+0000","last_clean":"2023-12-14T18:26:48.561575+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:48.561575+0000","last_undegraded":"2023-12-14T18:26:48.561575+0000","last_fullsized":"2023-12-14T18:26:48.561575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:20:52.709127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562632+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:48.562632+0000","last_peered":"2023-12-14T18:26:48.562632+0000","last_clean":"2023-12-14T18:26:48.562632+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:48.562632+0000","last_undegraded":"2023-12-14T18:26:48.562632+0000","last_fullsized":"2023-12-14T18:26:48.562632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:26.536039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568593+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:48.568593+0000","last_peered":"2023-12-14T18:26:48.568593+0000","last_clean":"2023-12-14T18:26:48.568593+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:48.568593+0000","last_undegraded":"2023-12-14T18:26:48.568593+0000","last_fullsized":"2023-12-14T18:26:48.568593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:59:51.600710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564002+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:48.564002+0000","last_peered":"2023-12-14T18:26:48.564002+0000","last_clean":"2023-12-14T18:26:48.564002+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:48.564002+0000","last_undegraded":"2023-12-14T18:26:48.564002+0000","last_fullsized":"2023-12-14T18:26:48.564002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:06.768114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559014+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:48.559014+0000","last_peered":"2023-12-14T18:26:48.559014+0000","last_clean":"2023-12-14T18:26:48.559014+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:48.559014+0000","last_undegraded":"2023-12-14T18:26:48.559014+0000","last_fullsized":"2023-12-14T18:26:48.559014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:32.285691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.577550+0000","last_change":"2023-12-14T18:26:48.577550+0000","last_active":"2023-12-14T18:26:48.577550+0000","last_peered":"2023-12-14T18:26:48.577550+0000","last_clean":"2023-12-14T18:26:48.577550+0000","last_became_active":"2023-12-14T18:26:48.569607+0000","last_became_peered":"2023-12-14T18:26:48.569607+0000","last_unstale":"2023-12-14T18:26:48.577550+0000","last_undegraded":"2023-12-14T18:26:48.577550+0000","last_fullsized":"2023-12-14T18:26:48.577550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:23.766724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559798+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:48.559798+0000","last_peered":"2023-12-14T18:26:48.559798+0000","last_clean":"2023-12-14T18:26:48.559798+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:48.559798+0000","last_undegraded":"2023-12-14T18:26:48.559798+0000","last_fullsized":"2023-12-14T18:26:48.559798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:29.491959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563921+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:26:48.563921+0000","last_peered":"2023-12-14T18:26:48.563921+0000","last_clean":"2023-12-14T18:26:48.563921+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:26:48.563921+0000","last_undegraded":"2023-12-14T18:26:48.563921+0000","last_fullsized":"2023-12-14T18:26:48.563921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:12.984642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563541+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:48.563541+0000","last_peered":"2023-12-14T18:26:48.563541+0000","last_clean":"2023-12-14T18:26:48.563541+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:48.563541+0000","last_undegraded":"2023-12-14T18:26:48.563541+0000","last_fullsized":"2023-12-14T18:26:48.563541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:56.278549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557904+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:48.557904+0000","last_peered":"2023-12-14T18:26:48.557904+0000","last_clean":"2023-12-14T18:26:48.557904+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:48.557904+0000","last_undegraded":"2023-12-14T18:26:48.557904+0000","last_fullsized":"2023-12-14T18:26:48.557904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:36:53.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563087+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:48.563087+0000","last_peered":"2023-12-14T18:26:48.563087+0000","last_clean":"2023-12-14T18:26:48.563087+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:48.563087+0000","last_undegraded":"2023-12-14T18:26:48.563087+0000","last_fullsized":"2023-12-14T18:26:48.563087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:19.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559959+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:48.559959+0000","last_peered":"2023-12-14T18:26:48.559959+0000","last_clean":"2023-12-14T18:26:48.559959+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:48.559959+0000","last_undegraded":"2023-12-14T18:26:48.559959+0000","last_fullsized":"2023-12-14T18:26:48.559959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:05.103613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566175+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:48.566175+0000","last_peered":"2023-12-14T18:26:48.566175+0000","last_clean":"2023-12-14T18:26:48.566175+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:48.566175+0000","last_undegraded":"2023-12-14T18:26:48.566175+0000","last_fullsized":"2023-12-14T18:26:48.566175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:21.311660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562945+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:26:48.562945+0000","last_peered":"2023-12-14T18:26:48.562945+0000","last_clean":"2023-12-14T18:26:48.562945+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:26:48.562945+0000","last_undegraded":"2023-12-14T18:26:48.562945+0000","last_fullsized":"2023-12-14T18:26:48.562945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:04:53.461295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558908+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:48.558908+0000","last_peered":"2023-12-14T18:26:48.558908+0000","last_clean":"2023-12-14T18:26:48.558908+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:48.558908+0000","last_undegraded":"2023-12-14T18:26:48.558908+0000","last_fullsized":"2023-12-14T18:26:48.558908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:04:46.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561591+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:48.561591+0000","last_peered":"2023-12-14T18:26:48.561591+0000","last_clean":"2023-12-14T18:26:48.561591+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:48.561591+0000","last_undegraded":"2023-12-14T18:26:48.561591+0000","last_fullsized":"2023-12-14T18:26:48.561591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:52:50.350437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562857+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:48.562857+0000","last_peered":"2023-12-14T18:26:48.562857+0000","last_clean":"2023-12-14T18:26:48.562857+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:48.562857+0000","last_undegraded":"2023-12-14T18:26:48.562857+0000","last_fullsized":"2023-12-14T18:26:48.562857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:22.082703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565524+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:26:48.565524+0000","last_peered":"2023-12-14T18:26:48.565524+0000","last_clean":"2023-12-14T18:26:48.565524+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:26:48.565524+0000","last_undegraded":"2023-12-14T18:26:48.565524+0000","last_fullsized":"2023-12-14T18:26:48.565524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:51:55.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559293+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:48.559293+0000","last_peered":"2023-12-14T18:26:48.559293+0000","last_clean":"2023-12-14T18:26:48.559293+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:48.559293+0000","last_undegraded":"2023-12-14T18:26:48.559293+0000","last_fullsized":"2023-12-14T18:26:48.559293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:17:28.674237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563263+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:48.563263+0000","last_peered":"2023-12-14T18:26:48.563263+0000","last_clean":"2023-12-14T18:26:48.563263+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:48.563263+0000","last_undegraded":"2023-12-14T18:26:48.563263+0000","last_fullsized":"2023-12-14T18:26:48.563263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:08.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.578387+0000","last_change":"2023-12-14T18:26:48.577964+0000","last_active":"2023-12-14T18:26:48.578387+0000","last_peered":"2023-12-14T18:26:48.578387+0000","last_clean":"2023-12-14T18:26:48.578387+0000","last_became_active":"2023-12-14T18:26:48.569637+0000","last_became_peered":"2023-12-14T18:26:48.569637+0000","last_unstale":"2023-12-14T18:26:48.578387+0000","last_undegraded":"2023-12-14T18:26:48.578387+0000","last_fullsized":"2023-12-14T18:26:48.578387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:31:31.614815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562048+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:48.562048+0000","last_peered":"2023-12-14T18:26:48.562048+0000","last_clean":"2023-12-14T18:26:48.562048+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:48.562048+0000","last_undegraded":"2023-12-14T18:26:48.562048+0000","last_fullsized":"2023-12-14T18:26:48.562048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:43.573208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559867+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:48.559867+0000","last_peered":"2023-12-14T18:26:48.559867+0000","last_clean":"2023-12-14T18:26:48.559867+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:48.559867+0000","last_undegraded":"2023-12-14T18:26:48.559867+0000","last_fullsized":"2023-12-14T18:26:48.559867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:15:04.165007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.574468+0000","last_change":"2023-12-14T18:26:48.574083+0000","last_active":"2023-12-14T18:26:48.574468+0000","last_peered":"2023-12-14T18:26:48.574468+0000","last_clean":"2023-12-14T18:26:48.574468+0000","last_became_active":"2023-12-14T18:26:48.565205+0000","last_became_peered":"2023-12-14T18:26:48.565205+0000","last_unstale":"2023-12-14T18:26:48.574468+0000","last_undegraded":"2023-12-14T18:26:48.574468+0000","last_fullsized":"2023-12-14T18:26:48.574468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:19.409191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562445+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:48.562445+0000","last_peered":"2023-12-14T18:26:48.562445+0000","last_clean":"2023-12-14T18:26:48.562445+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:48.562445+0000","last_undegraded":"2023-12-14T18:26:48.562445+0000","last_fullsized":"2023-12-14T18:26:48.562445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:37.001669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557585+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:48.557585+0000","last_peered":"2023-12-14T18:26:48.557585+0000","last_clean":"2023-12-14T18:26:48.557585+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:48.557585+0000","last_undegraded":"2023-12-14T18:26:48.557585+0000","last_fullsized":"2023-12-14T18:26:48.557585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:54.017877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564140+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:48.564140+0000","last_peered":"2023-12-14T18:26:48.564140+0000","last_clean":"2023-12-14T18:26:48.564140+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:48.564140+0000","last_undegraded":"2023-12-14T18:26:48.564140+0000","last_fullsized":"2023-12-14T18:26:48.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:46:47.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562523+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:48.562523+0000","last_peered":"2023-12-14T18:26:48.562523+0000","last_clean":"2023-12-14T18:26:48.562523+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:48.562523+0000","last_undegraded":"2023-12-14T18:26:48.562523+0000","last_fullsized":"2023-12-14T18:26:48.562523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:52:04.129230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566378+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:48.566378+0000","last_peered":"2023-12-14T18:26:48.566378+0000","last_clean":"2023-12-14T18:26:48.566378+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:48.566378+0000","last_undegraded":"2023-12-14T18:26:48.566378+0000","last_fullsized":"2023-12-14T18:26:48.566378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:08:14.934634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564930+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:48.564930+0000","last_peered":"2023-12-14T18:26:48.564930+0000","last_clean":"2023-12-14T18:26:48.564930+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:48.564930+0000","last_undegraded":"2023-12-14T18:26:48.564930+0000","last_fullsized":"2023-12-14T18:26:48.564930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:28.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562150+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:48.562150+0000","last_peered":"2023-12-14T18:26:48.562150+0000","last_clean":"2023-12-14T18:26:48.562150+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:48.562150+0000","last_undegraded":"2023-12-14T18:26:48.562150+0000","last_fullsized":"2023-12-14T18:26:48.562150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:43:39.281554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562009+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:48.562009+0000","last_peered":"2023-12-14T18:26:48.562009+0000","last_clean":"2023-12-14T18:26:48.562009+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:48.562009+0000","last_undegraded":"2023-12-14T18:26:48.562009+0000","last_fullsized":"2023-12-14T18:26:48.562009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:49:45.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564454+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:26:48.564454+0000","last_peered":"2023-12-14T18:26:48.564454+0000","last_clean":"2023-12-14T18:26:48.564454+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:26:48.564454+0000","last_undegraded":"2023-12-14T18:26:48.564454+0000","last_fullsized":"2023-12-14T18:26:48.564454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:16.913094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563425+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:26:48.563425+0000","last_peered":"2023-12-14T18:26:48.563425+0000","last_clean":"2023-12-14T18:26:48.563425+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:26:48.563425+0000","last_undegraded":"2023-12-14T18:26:48.563425+0000","last_fullsized":"2023-12-14T18:26:48.563425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:54.437297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557742+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:48.557742+0000","last_peered":"2023-12-14T18:26:48.557742+0000","last_clean":"2023-12-14T18:26:48.557742+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:48.557742+0000","last_undegraded":"2023-12-14T18:26:48.557742+0000","last_fullsized":"2023-12-14T18:26:48.557742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:35.196672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564064+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:48.564064+0000","last_peered":"2023-12-14T18:26:48.564064+0000","last_clean":"2023-12-14T18:26:48.564064+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:48.564064+0000","last_undegraded":"2023-12-14T18:26:48.564064+0000","last_fullsized":"2023-12-14T18:26:48.564064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:27.906838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562103+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:48.562103+0000","last_peered":"2023-12-14T18:26:48.562103+0000","last_clean":"2023-12-14T18:26:48.562103+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:48.562103+0000","last_undegraded":"2023-12-14T18:26:48.562103+0000","last_fullsized":"2023-12-14T18:26:48.562103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:44:51.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566964+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:26:48.566964+0000","last_peered":"2023-12-14T18:26:48.566964+0000","last_clean":"2023-12-14T18:26:48.566964+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:26:48.566964+0000","last_undegraded":"2023-12-14T18:26:48.566964+0000","last_fullsized":"2023-12-14T18:26:48.566964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:07:32.112350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559871+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:48.559871+0000","last_peered":"2023-12-14T18:26:48.559871+0000","last_clean":"2023-12-14T18:26:48.559871+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:48.559871+0000","last_undegraded":"2023-12-14T18:26:48.559871+0000","last_fullsized":"2023-12-14T18:26:48.559871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:36:21.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566829+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:48.566829+0000","last_peered":"2023-12-14T18:26:48.566829+0000","last_clean":"2023-12-14T18:26:48.566829+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:48.566829+0000","last_undegraded":"2023-12-14T18:26:48.566829+0000","last_fullsized":"2023-12-14T18:26:48.566829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:17.885326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559062+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:48.559062+0000","last_peered":"2023-12-14T18:26:48.559062+0000","last_clean":"2023-12-14T18:26:48.559062+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:48.559062+0000","last_undegraded":"2023-12-14T18:26:48.559062+0000","last_fullsized":"2023-12-14T18:26:48.559062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:28:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560244+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:48.560244+0000","last_peered":"2023-12-14T18:26:48.560244+0000","last_clean":"2023-12-14T18:26:48.560244+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:48.560244+0000","last_undegraded":"2023-12-14T18:26:48.560244+0000","last_fullsized":"2023-12-14T18:26:48.560244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:54.715861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562903+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:48.562903+0000","last_peered":"2023-12-14T18:26:48.562903+0000","last_clean":"2023-12-14T18:26:48.562903+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:48.562903+0000","last_undegraded":"2023-12-14T18:26:48.562903+0000","last_fullsized":"2023-12-14T18:26:48.562903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:54:48.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562082+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:48.562082+0000","last_peered":"2023-12-14T18:26:48.562082+0000","last_clean":"2023-12-14T18:26:48.562082+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:48.562082+0000","last_undegraded":"2023-12-14T18:26:48.562082+0000","last_fullsized":"2023-12-14T18:26:48.562082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:40:01.866310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63711,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":238,"num_read_kb":189,"num_write":157,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739028,"statfs":{"total":95995035648,"available":95988764672,"internally_reserved":0,"allocated":954368,"data_stored":484012,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.318},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.44}]},{"osd":5,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.59199999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.66900000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.47599999999999998}]}]},{"osd":10,"up_from":68,"seq":292057776155,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6580,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738572,"statfs":{"total":95995035648,"available":95988297728,"internally_reserved":0,"allocated":1093632,"data_stored":1130758,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":3.4449999999999998,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.44400000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":3.3119999999999998,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":5.4960000000000004,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.54300000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80500000000000005}]}]},{"osd":9,"up_from":63,"seq":270582939678,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":974848,"data_stored":488344,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38,"15min":0.38},"max":{"1min":1.01,"5min":1.028,"15min":1.028},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.45400000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.437},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.746}]},{"osd":5,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.081,"15min":1.081},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.036,"15min":1.036},"last":0.82899999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1200000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.86199999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0189999999999999,"5min":1.123,"15min":1.123},"last":0.84699999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.89800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]}]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6460,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738692,"statfs":{"total":95995035648,"available":95988420608,"internally_reserved":0,"allocated":970752,"data_stored":484168,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60599999999999998},"min":{"1min":0.371,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57199999999999995,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.44}]},{"osd":2,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.312,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.377},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.27200000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59699999999999998,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.5}]},{"osd":4,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88500000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.30099999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69499999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86199999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.55100000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69799999999999995,"15min":0.68999999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.86899999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.628,"5min":0.502,"15min":0.502},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.78400000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.73299999999999998}]}]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":1079376,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39300000000000002},"min":{"1min":0.33100000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.57899999999999996,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.41099999999999998,"15min":0.39700000000000002},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.34699999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54100000000000004},"min":{"1min":0.40799999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.46100000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.752,"5min":1.8,"15min":1.8},"last":0.57399999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47199999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.63300000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.46500000000000002},"min":{"1min":0.36199999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46000000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.46500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:26:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.59599999999999997,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84899999999999998,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.65000000000000002,"15min":0.63700000000000001},"min":{"1min":0.55000000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.61699999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.751},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.69299999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.81000000000000005}]}]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6440,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738712,"statfs":{"total":95995035648,"available":95988441088,"internally_reserved":0,"allocated":950272,"data_stored":479424,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.34599999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64200000000000002,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.627},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.95999999999999996}]},{"osd":5,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.752}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72199999999999998,"15min":0.70999999999999996},"min":{"1min":0.622,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.81399999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.754,"15min":0.75700000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.97499999999999998,"5min":1.127,"15min":1.127},"last":1.038},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.57899999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.127,"5min":1.133,"15min":1.133},"last":0.59499999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.873},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":1.0069999999999999}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.0649999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":539450,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.35799999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.56000000000000005,"15min":0.51000000000000001},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.0800000000000001,"5min":5.0800000000000001,"15min":5.0800000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.53000000000000003,"15min":0.47999999999999998},"min":{"1min":0.40200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.61099999999999999,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.3150000000000004,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.61299999999999999,"15min":0.54700000000000004},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":5.3460000000000001,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.80700000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:26:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.38400000000000001},"min":{"1min":0.34699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.625},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.312,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70899999999999996,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.497}]},{"osd":5,"last update":"Thu Dec 14 18:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61599999999999999,"15min":0.58199999999999996},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82899999999999996,"5min":1.657,"15min":1.657},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.58599999999999997},"min":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.755}]},{"osd":7,"last update":"Thu Dec 14 18:26:08 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.48199999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.87,"5min":0.874,"15min":0.874},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81799999999999995,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67800000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:26:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.51900000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.91400000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72999999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.59299999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.78400000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70399999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":480812,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55200000000000005,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.55000000000000004,"15min":0.50600000000000001},"min":{"1min":0.38600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88700000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.376}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.81799999999999995}]},{"osd":3,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.747}]},{"osd":4,"last update":"Thu Dec 14 18:26:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.60299999999999998,"15min":0.58199999999999996},"min":{"1min":0.40500000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":1.091,"15min":1.091},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88900000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.503}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58099999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:25:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98699999999999999,"5min":1.079,"15min":1.079},"last":0.72799999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.82499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":0.76700000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68899999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.73799999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411384,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7540,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737612,"statfs":{"total":95995035648,"available":95987314688,"internally_reserved":0,"allocated":1028096,"data_stored":539287,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.484,"15min":1.484},"last":0.624}]},{"osd":1,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.52300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82199999999999995,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.56499999999999995},"min":{"1min":0.42499999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.86599999999999999,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.58799999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71699999999999997,"5min":0.75,"15min":0.75},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Dec 14 18:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81000000000000005,"5min":1.514,"15min":1.514},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63}]},{"osd":8,"last update":"Thu Dec 14 18:26:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.56499999999999995,"15min":0.55400000000000005},"min":{"1min":0.34799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36399999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.61399999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:26:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":958464,"data_stored":479345,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.38200000000000001},"min":{"1min":0.36499999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.42899999999999999,"15min":0.40899999999999997},"min":{"1min":0.33100000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.93300000000000005}]},{"osd":2,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.58599999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.65000000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.82799999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.68200000000000005,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.624}]},{"osd":7,"last update":"Thu Dec 14 18:26:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.81499999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":6.7249999999999996,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":0.77200000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":1.0029999999999999}]},{"osd":11,"last update":"Thu Dec 14 18:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.0349999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":484990,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"min":{"1min":0.29099999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.374,"15min":0.35799999999999998},"min":{"1min":0.27000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.56299999999999994},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80300000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.88700000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:26:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.374,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.71799999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78300000000000003,"5min":1.121,"15min":1.121},"last":0.62}]},{"osd":5,"last update":"Thu Dec 14 18:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.752}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85999999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:26:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.37,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.873,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83299999999999996,"5min":0.879,"15min":0.879},"last":0.65300000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.81399999999999995}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":1.161,"15min":1.161},"last":0.73299999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83399999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":1024000,"data_stored":1075285,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63800000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.434},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.54000000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66600000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.314,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":4.7290000000000001,"5min":4.7290000000000001,"15min":4.7290000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.621,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.391,"5min":6.391,"15min":6.391},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63500000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:25:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:26:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.41399999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46899999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.443,"5min":0.309,"15min":0.309},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.624,"15min":0.626},"min":{"1min":0.432,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60899999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.754},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.66500000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:26:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:27:03.422 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-14T18:27:03.422 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-14T18:27:03.422 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-14T18:27:03.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph health --format=json 2023-12-14T18:27:03.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:03 smithi176 ceph-mon[43506]: pgmap v264: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 128 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:27:03.782 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:04 smithi031 ceph-mon[42085]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:04 smithi204 ceph-mon[43866]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:04.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:04 smithi176 ceph-mon[43506]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 -- 172.21.15.31:0/1574037542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ffe00 msgr2=0x7f9f5c103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/1574037542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ffe00 0x7f9f5c103dd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f500099c0 tx=0x7f9f50009cd0 comp rx=0 tx=0).stop 2023-12-14T18:27:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 -- 172.21.15.31:0/1574037542 shutdown_connections 2023-12-14T18:27:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/1574037542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f5c104500 0x7f9f5c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/1574037542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/1574037542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ffe00 0x7f9f5c103dd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 -- 172.21.15.31:0/1574037542 >> 172.21.15.31:0/1574037542 conn(0x7f9f5c0770a0 msgr2=0x7f9f5c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 -- 172.21.15.31:0/1574037542 shutdown_connections 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.812+0000 7f9f60b6e700 1 -- 172.21.15.31:0/1574037542 wait complete. 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 Processor -- start 2023-12-14T18:27:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 -- start start 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f5c0ffe00 0x7f9f5c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f5c104500 0x7f9f5c1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c1a2790 con 0x7f9f5c0ff460 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c1a28d0 con 0x7f9f5c104500 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f60b6e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c1a2a10 con 0x7f9f5c0ffe00 2023-12-14T18:27:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f5a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f5a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51344/0 (socket says 172.21.15.31:51344) 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f5a59c700 1 -- 172.21.15.31:0/2774466278 learned_addr learned my addr 172.21.15.31:0/2774466278 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f59d9b700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f5c0ffe00 0x7f9f5c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.813+0000 7f9f5ad9d700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f5c104500 0x7f9f5c1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f5a59c700 1 -- 172.21.15.31:0/2774466278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f5c0ffe00 msgr2=0x7f9f5c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f5a59c700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f5c0ffe00 0x7f9f5c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f5a59c700 1 -- 172.21.15.31:0/2774466278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f5c104500 msgr2=0x7f9f5c1a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f5a59c700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f5c104500 0x7f9f5c1a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f5a59c700 1 -- 172.21.15.31:0/2774466278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500095f0 con 0x7f9f5c0ff460 2023-12-14T18:27:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f5a59c700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c1971d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d980 tx=0x7f9f4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004220 con 0x7f9f5c0ff460 2023-12-14T18:27:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f4c007640 con 0x7f9f5c0ff460 2023-12-14T18:27:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c10f230 con 0x7f9f5c0ff460 2023-12-14T18:27:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c005690 con 0x7f9f5c0ff460 2023-12-14T18:27:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.814+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c10f0a0 con 0x7f9f5c0ff460 2023-12-14T18:27:04.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.815+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c10f3c0 con 0x7f9f5c0ff460 2023-12-14T18:27:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.816+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9f4c004750 con 0x7f9f5c0ff460 2023-12-14T18:27:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.817+0000 7f9f4b7fe700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f440a11e0 0x7f9f440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.818+0000 7f9f59d9b700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f440a11e0 0x7f9f440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.818+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9f4c0ccb70 con 0x7f9f5c0ff460 2023-12-14T18:27:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.818+0000 7f9f59d9b700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f440a11e0 0x7f9f440a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f50006010 tx=0x7f9f50003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:04.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:04.822+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9f4c095e50 con 0x7f9f5c0ff460 2023-12-14T18:27:05.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.076+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9f5c066ea0 con 0x7f9f5c0ff460 2023-12-14T18:27:05.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.077+0000 7f9f4b7fe700 1 -- 172.21.15.31:0/2774466278 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9f4c016080 con 0x7f9f5c0ff460 2023-12-14T18:27:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:05.078 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-14T18:27:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f440a11e0 msgr2=0x7f9f440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f440a11e0 0x7f9f440a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f50006010 tx=0x7f9f50003680 comp rx=0 tx=0).stop 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 msgr2=0x7f9f5c1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c1971d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00d980 tx=0x7f9f4c00dc90 comp rx=0 tx=0).stop 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 shutdown_connections 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9f5c104500 0x7f9f5c1a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f5c0ff460 0x7f9f5c1971d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9f440a11e0 0x7f9f440a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 --2- 172.21.15.31:0/2774466278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f5c0ffe00 0x7f9f5c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 >> 172.21.15.31:0/2774466278 conn(0x7f9f5c0770a0 msgr2=0x7f9f5c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 shutdown_connections 2023-12-14T18:27:05.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.079+0000 7f9f60b6e700 1 -- 172.21.15.31:0/2774466278 wait complete. 2023-12-14T18:27:05.297 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:05 smithi031 ceph-mon[42085]: pgmap v265: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 80 B/s, 0 objects/s recovering 2023-12-14T18:27:05.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:05 smithi204 ceph-mon[43866]: pgmap v265: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 80 B/s, 0 objects/s recovering 2023-12-14T18:27:05.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:05 smithi176 ceph-mon[43506]: pgmap v265: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 80 B/s, 0 objects/s recovering 2023-12-14T18:27:05.497 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-14T18:27:05.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 -- 172.21.15.31:0/1404722952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14101db0 msgr2=0x7faa14108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 --2- 172.21.15.31:0/1404722952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14101db0 0x7faa14108670 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00b0a0 tx=0x7faa0c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 -- 172.21.15.31:0/1404722952 shutdown_connections 2023-12-14T18:27:05.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 --2- 172.21.15.31:0/1404722952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14101db0 0x7faa14108670 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 --2- 172.21.15.31:0/1404722952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14100230 0x7faa14100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 --2- 172.21.15.31:0/1404722952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa14101420 0x7faa14101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 -- 172.21.15.31:0/1404722952 >> 172.21.15.31:0/1404722952 conn(0x7faa140fb810 msgr2=0x7faa140fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 -- 172.21.15.31:0/1404722952 shutdown_connections 2023-12-14T18:27:05.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.645+0000 7faa19052700 1 -- 172.21.15.31:0/1404722952 wait complete. 2023-12-14T18:27:05.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 Processor -- start 2023-12-14T18:27:05.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 -- start start 2023-12-14T18:27:05.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 0x7faa14196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:05.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101420 0x7faa141965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:05.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa14101db0 0x7faa1419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419dc00 con 0x7faa14100230 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419dd40 con 0x7faa14101db0 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa19052700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419ded0 con 0x7faa14101420 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 0x7faa14196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa1259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101420 0x7faa141965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 0x7faa14196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51358/0 (socket says 172.21.15.31:51358) 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa12d9d700 1 -- 172.21.15.31:0/348415664 learned_addr learned my addr 172.21.15.31:0/348415664 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.646+0000 7faa1359e700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa14101db0 0x7faa1419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa12d9d700 1 -- 172.21.15.31:0/348415664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101420 msgr2=0x7faa141965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa12d9d700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101420 0x7faa141965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa12d9d700 1 -- 172.21.15.31:0/348415664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa14101db0 msgr2=0x7faa1419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa12d9d700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa14101db0 0x7faa1419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa12d9d700 1 -- 172.21.15.31:0/348415664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c009d00 con 0x7faa14100230 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa1259c700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101420 0x7faa141965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa12d9d700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 0x7faa14196080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa0400d980 tx=0x7faa0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04004020 con 0x7faa14100230 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa0400f690 con 0x7faa14100230 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1410bba0 con 0x7faa14100230 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04003a60 con 0x7faa14100230 2023-12-14T18:27:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.647+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1410ba10 con 0x7faa14100230 2023-12-14T18:27:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.648+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa141070c0 con 0x7faa14100230 2023-12-14T18:27:05.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.649+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7faa0400fb00 con 0x7faa14100230 2023-12-14T18:27:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.650+0000 7fa9fbfff700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9fc0a11e0 0x7fa9fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.650+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faa040cca00 con 0x7faa14100230 2023-12-14T18:27:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.650+0000 7faa1259c700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9fc0a11e0 0x7fa9fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:05.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.650+0000 7faa1259c700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9fc0a11e0 0x7fa9fc0a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa08005950 tx=0x7faa0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:05.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.654+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faa04095c80 con 0x7faa14100230 2023-12-14T18:27:05.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.865+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa1402cfe0 con 0x7faa14100230 2023-12-14T18:27:05.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.867+0000 7fa9fbfff700 1 -- 172.21.15.31:0/348415664 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7faa040992a0 con 0x7faa14100230 2023-12-14T18:27:05.868 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:05.869 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:05.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9fc0a11e0 msgr2=0x7fa9fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9fc0a11e0 0x7fa9fc0a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa08005950 tx=0x7faa0800b410 comp rx=0 tx=0).stop 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 msgr2=0x7faa14196080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 0x7faa14196080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7faa0400d980 tx=0x7faa0400dc90 comp rx=0 tx=0).stop 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 shutdown_connections 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa14101db0 0x7faa1419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa14100230 0x7faa14196080 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa9fc0a11e0 0x7fa9fc0a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 --2- 172.21.15.31:0/348415664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101420 0x7faa141965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:05.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 >> 172.21.15.31:0/348415664 conn(0x7faa140fb810 msgr2=0x7faa140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:05.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 shutdown_connections 2023-12-14T18:27:05.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:05.869+0000 7faa19052700 1 -- 172.21.15.31:0/348415664 wait complete. 2023-12-14T18:27:05.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:05.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 -- 172.21.15.31:0/2076139407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c105ee0 msgr2=0x7f139c108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2076139407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c105ee0 0x7f139c108340 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f139400b0a0 tx=0x7f139400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 -- 172.21.15.31:0/2076139407 shutdown_connections 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2076139407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c105ee0 0x7f139c108340 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2076139407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c100ca0 0x7f139c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2076139407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c1035c0 0x7f139c1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 -- 172.21.15.31:0/2076139407 >> 172.21.15.31:0/2076139407 conn(0x7f139c0fa740 msgr2=0x7f139c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 -- 172.21.15.31:0/2076139407 shutdown_connections 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.028+0000 7f13a45fe700 1 -- 172.21.15.31:0/2076139407 wait complete. 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 Processor -- start 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 -- start start 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 0x7f139c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c1035c0 0x7f139c0fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c105ee0 0x7f139c0ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0ff930 con 0x7f139c100ca0 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c0ffaa0 con 0x7f139c105ee0 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.029+0000 7f13a45fe700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c1a00e0 con 0x7f139c1035c0 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 0x7f139c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 0x7f139c100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51366/0 (socket says 172.21.15.31:51366) 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 -- 172.21.15.31:0/2703258795 learned_addr learned my addr 172.21.15.31:0/2703258795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a2b9b700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c105ee0 0x7f139c0ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a1b99700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c1035c0 0x7f139c0fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 -- 172.21.15.31:0/2703258795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c1035c0 msgr2=0x7f139c0fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c1035c0 0x7f139c0fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 -- 172.21.15.31:0/2703258795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c105ee0 msgr2=0x7f139c0ff190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c105ee0 0x7f139c0ff190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 -- 172.21.15.31:0/2703258795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394009d00 con 0x7f139c100ca0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a239a700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 0x7f139c100610 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9a0 tx=0x7f139800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398009790 con 0x7f139c100ca0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1398010430 con 0x7f139c100ca0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c10bb80 con 0x7f139c100ca0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398018680 con 0x7f139c100ca0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.030+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c10b9f0 con 0x7f139c100ca0 2023-12-14T18:27:06.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.032+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139c0fc2b0 con 0x7f139c100ca0 2023-12-14T18:27:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.033+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f13980105a0 con 0x7f139c100ca0 2023-12-14T18:27:06.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.033+0000 7f13937fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13880a1260 0x7f13880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.033+0000 7f13a1b99700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13880a1260 0x7f13880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.033+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1398014070 con 0x7f139c100ca0 2023-12-14T18:27:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.034+0000 7f13a1b99700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13880a1260 0x7f13880a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f138c009c60 tx=0x7f138c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.037+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1398051730 con 0x7f139c100ca0 2023-12-14T18:27:06.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:06 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2774466278' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-14T18:27:06.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:06 smithi031 ceph-mon[42085]: pgmap v266: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-12-14T18:27:06.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:06 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/348415664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:06.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.246+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f139c02cfe0 con 0x7f139c100ca0 2023-12-14T18:27:06.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.246+0000 7f13937fe700 1 -- 172.21.15.31:0/2703258795 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1398096200 con 0x7f139c100ca0 2023-12-14T18:27:06.247 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:06.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13880a1260 msgr2=0x7f13880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13880a1260 0x7f13880a3710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f138c009c60 tx=0x7f138c009260 comp rx=0 tx=0).stop 2023-12-14T18:27:06.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 msgr2=0x7f139c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 0x7f139c100610 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9a0 tx=0x7f139800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 shutdown_connections 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c105ee0 0x7f139c0ff190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c100ca0 0x7f139c100610 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f13880a1260 0x7f13880a3710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 --2- 172.21.15.31:0/2703258795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c1035c0 0x7f139c0fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 >> 172.21.15.31:0/2703258795 conn(0x7f139c0fa740 msgr2=0x7f139c101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 shutdown_connections 2023-12-14T18:27:06.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.249+0000 7f13a45fe700 1 -- 172.21.15.31:0/2703258795 wait complete. 2023-12-14T18:27:06.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:06.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:06 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2774466278' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-14T18:27:06.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:06 smithi204 ceph-mon[43866]: pgmap v266: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-12-14T18:27:06.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:06 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/348415664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/3721330534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 msgr2=0x7f2d68107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/3721330534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d68107590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000b0a0 tx=0x7f2d6000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/3721330534 shutdown_connections 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/3721330534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d68107590 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/3721330534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d680ff150 0x7f2d680ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/3721330534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d68100340 0x7f2d68100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/3721330534 >> 172.21.15.31:0/3721330534 conn(0x7f2d680fa770 msgr2=0x7f2d680fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/3721330534 shutdown_connections 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/3721330534 wait complete. 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 Processor -- start 2023-12-14T18:27:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.403+0000 7f2d6e5a3700 1 -- start start 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6e5a3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d680ff150 0x7f2d680698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6e5a3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d68100340 0x7f2d6819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6e5a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d6819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6e5a3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819dbf0 con 0x7f2d68100cd0 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6e5a3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819dd60 con 0x7f2d680ff150 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6e5a3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6819df20 con 0x7f2d68100340 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d6819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d6819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51378/0 (socket says 172.21.15.31:51378) 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 -- 172.21.15.31:0/1821168781 learned_addr learned my addr 172.21.15.31:0/1821168781 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d67fff700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d680ff150 0x7f2d680698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d677fe700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d68100340 0x7f2d6819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 -- 172.21.15.31:0/1821168781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d68100340 msgr2=0x7f2d6819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d68100340 0x7f2d6819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 -- 172.21.15.31:0/1821168781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d680ff150 msgr2=0x7f2d680698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d680ff150 0x7f2d680698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 -- 172.21.15.31:0/1821168781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60009d00 con 0x7f2d68100cd0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.404+0000 7f2d6cb40700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d6819d450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d010 tx=0x7f2d6000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.405+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000e040 con 0x7f2d68100cd0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.405+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d60003e10 con 0x7f2d68100cd0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.405+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6810ab20 con 0x7f2d68100cd0 2023-12-14T18:27:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.405+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600053b0 con 0x7f2d68100cd0 2023-12-14T18:27:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.405+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6810a990 con 0x7f2d68100cd0 2023-12-14T18:27:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.406+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6819e4c0 con 0x7f2d68100cd0 2023-12-14T18:27:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.407+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2d6001d9d0 con 0x7f2d68100cd0 2023-12-14T18:27:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.407+0000 7f2d657fa700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2d500a11e0 0x7f2d500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.408+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d600ce2e0 con 0x7f2d68100cd0 2023-12-14T18:27:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.408+0000 7f2d67fff700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2d500a11e0 0x7f2d500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.408+0000 7f2d67fff700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2d500a11e0 0x7f2d500a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d681001a0 tx=0x7f2d58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.411+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d60097650 con 0x7f2d68100cd0 2023-12-14T18:27:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:06 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2774466278' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-14T18:27:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:06 smithi176 ceph-mon[43506]: pgmap v266: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2023-12-14T18:27:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:06 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/348415664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.619+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d68105f80 con 0x7f2d68100cd0 2023-12-14T18:27:06.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.619+0000 7f2d657fa700 1 -- 172.21.15.31:0/1821168781 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d6009ac70 con 0x7f2d68100cd0 2023-12-14T18:27:06.620 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2d500a11e0 msgr2=0x7f2d500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2d500a11e0 0x7f2d500a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2d681001a0 tx=0x7f2d58009260 comp rx=0 tx=0).stop 2023-12-14T18:27:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 msgr2=0x7f2d6819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d6819d450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d010 tx=0x7f2d6000bd30 comp rx=0 tx=0).stop 2023-12-14T18:27:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 shutdown_connections 2023-12-14T18:27:06.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d68100cd0 0x7f2d6819d450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d680ff150 0x7f2d680698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2d500a11e0 0x7f2d500a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 --2- 172.21.15.31:0/1821168781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d68100340 0x7f2d6819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.621+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 >> 172.21.15.31:0/1821168781 conn(0x7f2d680fa770 msgr2=0x7f2d680fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.622+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 shutdown_connections 2023-12-14T18:27:06.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.622+0000 7f2d6e5a3700 1 -- 172.21.15.31:0/1821168781 wait complete. 2023-12-14T18:27:06.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 -- 172.21.15.31:0/1485653207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740ff130 msgr2=0x7f04740ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 --2- 172.21.15.31:0/1485653207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740ff130 0x7f04740ff540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0464009990 tx=0x7f0464009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 -- 172.21.15.31:0/1485653207 shutdown_connections 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 --2- 172.21.15.31:0/1485653207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 0x7f0474107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 --2- 172.21.15.31:0/1485653207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04740ff130 0x7f04740ff540 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 --2- 172.21.15.31:0/1485653207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0474100320 0x7f0474100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 -- 172.21.15.31:0/1485653207 >> 172.21.15.31:0/1485653207 conn(0x7f04740fa750 msgr2=0x7f04740fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 -- 172.21.15.31:0/1485653207 shutdown_connections 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.776+0000 7f0479911700 1 -- 172.21.15.31:0/1485653207 wait complete. 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 Processor -- start 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 -- start start 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04740ff130 0x7f0474195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 0x7f0474196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 0x7f047419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047419db60 con 0x7f0474100320 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047419dca0 con 0x7f0474100cb0 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f0479911700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047419de30 con 0x7f04740ff130 2023-12-14T18:27:06.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.777+0000 7f04727fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 0x7f0474196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 0x7f0474196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51394/0 (socket says 172.21.15.31:51394) 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 -- 172.21.15.31:0/1838422398 learned_addr learned my addr 172.21.15.31:0/1838422398 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f0472ffd700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04740ff130 0x7f0474195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04737fe700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 0x7f047419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 -- 172.21.15.31:0/1838422398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04740ff130 msgr2=0x7f0474195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04740ff130 0x7f0474195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 -- 172.21.15.31:0/1838422398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 msgr2=0x7f047419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 0x7f047419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 -- 172.21.15.31:0/1838422398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c009c40 con 0x7f0474100320 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04737fe700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 0x7f047419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f04727fc700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 0x7f0474196520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f046800e9a0 tx=0x7f046800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468009790 con 0x7f0474100320 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f046800cbd0 con 0x7f0474100320 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468018680 con 0x7f0474100320 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04640095f0 con 0x7f0474100320 2023-12-14T18:27:06.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.778+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047410aaa0 con 0x7f0474100320 2023-12-14T18:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.779+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0474067e40 con 0x7f0474100320 2023-12-14T18:27:06.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.781+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f046800cd40 con 0x7f0474100320 2023-12-14T18:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.781+0000 7f047890f700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f045c0a1260 0x7f045c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.781+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0468014070 con 0x7f0474100320 2023-12-14T18:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.781+0000 7f0472ffd700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f045c0a1260 0x7f045c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.782+0000 7f0472ffd700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f045c0a1260 0x7f045c0a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0464006010 tx=0x7f0464005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:06.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.785+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0468096520 con 0x7f0474100320 2023-12-14T18:27:06.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.995+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f047419e8f0 con 0x7f0474100320 2023-12-14T18:27:06.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.995+0000 7f047890f700 1 -- 172.21.15.31:0/1838422398 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04680260a0 con 0x7f0474100320 2023-12-14T18:27:06.997 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f045c0a1260 msgr2=0x7f045c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f045c0a1260 0x7f045c0a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0464006010 tx=0x7f0464005960 comp rx=0 tx=0).stop 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 msgr2=0x7f0474196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 0x7f0474196520 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f046800e9a0 tx=0x7f046800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 shutdown_connections 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0474100cb0 0x7f047419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f045c0a1260 0x7f045c0a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04740ff130 0x7f0474195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 --2- 172.21.15.31:0/1838422398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0474100320 0x7f0474196520 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 >> 172.21.15.31:0/1838422398 conn(0x7f04740fa750 msgr2=0x7f0474068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 shutdown_connections 2023-12-14T18:27:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:06.998+0000 7f0479911700 1 -- 172.21.15.31:0/1838422398 wait complete. 2023-12-14T18:27:07.013 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:07.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 -- 172.21.15.31:0/3585128383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e380ff120 msgr2=0x7f2e380ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 --2- 172.21.15.31:0/3585128383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e380ff120 0x7f2e380ff530 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009990 tx=0x7f2e34009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 -- 172.21.15.31:0/3585128383 shutdown_connections 2023-12-14T18:27:07.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 --2- 172.21.15.31:0/3585128383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 0x7f2e38107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 --2- 172.21.15.31:0/3585128383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e380ff120 0x7f2e380ff530 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 --2- 172.21.15.31:0/3585128383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e38100310 0x7f2e38100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 -- 172.21.15.31:0/3585128383 >> 172.21.15.31:0/3585128383 conn(0x7f2e380fa740 msgr2=0x7f2e380fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 -- 172.21.15.31:0/3585128383 shutdown_connections 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 -- 172.21.15.31:0/3585128383 wait complete. 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 Processor -- start 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.153+0000 7f2e3f355700 1 -- start start 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3f355700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e380ff120 0x7f2e381030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3f355700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 0x7f2e381016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3f355700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 0x7f2e38101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3f355700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e381023d0 con 0x7f2e38100310 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3f355700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38102540 con 0x7f2e380ff120 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3f355700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e381a0050 con 0x7f2e38100ca0 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 0x7f2e381016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 0x7f2e381016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51404/0 (socket says 172.21.15.31:51404) 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 -- 172.21.15.31:0/1835221376 learned_addr learned my addr 172.21.15.31:0/1835221376 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3d0f1700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e380ff120 0x7f2e381030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3d8f2700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 0x7f2e38101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 -- 172.21.15.31:0/1835221376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 msgr2=0x7f2e38101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 0x7f2e38101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 -- 172.21.15.31:0/1835221376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e380ff120 msgr2=0x7f2e381030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e380ff120 0x7f2e381030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 -- 172.21.15.31:0/1835221376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340095f0 con 0x7f2e38100310 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3d8f2700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 0x7f2e38101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3d0f1700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e380ff120 0x7f2e381030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e3c8f0700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 0x7f2e381016f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800e9a0 tx=0x7f2e2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.154+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28009790 con 0x7f2e38100310 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.155+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3810aa90 con 0x7f2e38100310 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.155+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e2800cbd0 con 0x7f2e38100310 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.155+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28018680 con 0x7f2e38100310 2023-12-14T18:27:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.155+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3810a900 con 0x7f2e38100310 2023-12-14T18:27:07.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.156+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e381a0740 con 0x7f2e38100310 2023-12-14T18:27:07.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.157+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2e2800cd40 con 0x7f2e38100310 2023-12-14T18:27:07.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.158+0000 7f2e2e7fc700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2e240a11e0 0x7f2e240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.158+0000 7f2e3d0f1700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2e240a11e0 0x7f2e240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.158+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2e28014070 con 0x7f2e38100310 2023-12-14T18:27:07.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.158+0000 7f2e3d0f1700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2e240a11e0 0x7f2e240a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400b2e0 tx=0x7f2e34005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:07.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.162+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2e28051730 con 0x7f2e38100310 2023-12-14T18:27:07.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2703258795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1821168781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1838422398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.374+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2e381a0de0 con 0x7f2e38100310 2023-12-14T18:27:07.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.375+0000 7f2e2e7fc700 1 -- 172.21.15.31:0/1835221376 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2e28099ea0 con 0x7f2e38100310 2023-12-14T18:27:07.376 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:07.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2e240a11e0 msgr2=0x7f2e240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2e240a11e0 0x7f2e240a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400b2e0 tx=0x7f2e34005fd0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 msgr2=0x7f2e381016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 0x7f2e381016f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800e9a0 tx=0x7f2e2800ed60 comp rx=0 tx=0).stop 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 shutdown_connections 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2e240a11e0 0x7f2e240a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e38100ca0 0x7f2e38101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2e380ff120 0x7f2e381030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 --2- 172.21.15.31:0/1835221376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e38100310 0x7f2e381016f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 >> 172.21.15.31:0/1835221376 conn(0x7f2e380fa740 msgr2=0x7f2e38067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 shutdown_connections 2023-12-14T18:27:07.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.377+0000 7f2e3f355700 1 -- 172.21.15.31:0/1835221376 wait complete. 2023-12-14T18:27:07.391 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2703258795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.391 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1821168781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.391 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1838422398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:07.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2703258795' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1821168781' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1838422398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 -- 172.21.15.31:0/235832892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 msgr2=0x7f0bb0100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 --2- 172.21.15.31:0/235832892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb0100640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009990 tx=0x7f0ba4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 -- 172.21.15.31:0/235832892 shutdown_connections 2023-12-14T18:27:07.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 --2- 172.21.15.31:0/235832892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb0101db0 0x7f0bb0108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 --2- 172.21.15.31:0/235832892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb0100640 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 --2- 172.21.15.31:0/235832892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101420 0x7f0bb0101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 -- 172.21.15.31:0/235832892 >> 172.21.15.31:0/235832892 conn(0x7f0bb00fb810 msgr2=0x7f0bb00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 -- 172.21.15.31:0/235832892 shutdown_connections 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.529+0000 7f0bb7798700 1 -- 172.21.15.31:0/235832892 wait complete. 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 Processor -- start 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 -- start start 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb0101420 0x7f0bb0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101db0 0x7f0bb0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0073bc0 con 0x7f0bb0100230 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0073d30 con 0x7f0bb0101420 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb7798700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb019ffa0 con 0x7f0bb0101db0 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51426/0 (socket says 172.21.15.31:51426) 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 -- 172.21.15.31:0/1691773129 learned_addr learned my addr 172.21.15.31:0/1691773129 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb4d33700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb0101420 0x7f0bb0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5d35700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101db0 0x7f0bb0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 -- 172.21.15.31:0/1691773129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101db0 msgr2=0x7f0bb0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101db0 0x7f0bb0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 -- 172.21.15.31:0/1691773129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb0101420 msgr2=0x7f0bb0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb0101420 0x7f0bb0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5534700 1 -- 172.21.15.31:0/1691773129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40095f0 con 0x7f0bb0100230 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.530+0000 7f0bb5d35700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101db0 0x7f0bb0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.531+0000 7f0bb5534700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb00748a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40059d0 tx=0x7f0ba4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.531+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba401d070 con 0x7f0bb0100230 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.531+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ba4004120 con 0x7f0bb0100230 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.531+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb010bba0 con 0x7f0bb0100230 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.531+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4022de0 con 0x7f0bb0100230 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.531+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb010ba10 con 0x7f0bb0100230 2023-12-14T18:27:07.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.532+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb01a05a0 con 0x7f0bb0100230 2023-12-14T18:27:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.534+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0ba4022470 con 0x7f0bb0100230 2023-12-14T18:27:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.535+0000 7f0ba27fc700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b9c0a1260 0x7f0b9c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.535+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0ba40ce850 con 0x7f0bb0100230 2023-12-14T18:27:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.535+0000 7f0bb4d33700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b9c0a1260 0x7f0b9c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.535+0000 7f0bb4d33700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b9c0a1260 0x7f0b9c0a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0bac005ea0 tx=0x7f0bac005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:07.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.538+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0ba4097bc0 con 0x7f0bb0100230 2023-12-14T18:27:07.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.801+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bb01070b0 con 0x7f0bb0100230 2023-12-14T18:27:07.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.802+0000 7f0ba27fc700 1 -- 172.21.15.31:0/1691773129 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f0ba409b1e0 con 0x7f0bb0100230 2023-12-14T18:27:07.803 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:07.803 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:07.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b9c0a1260 msgr2=0x7f0b9c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b9c0a1260 0x7f0b9c0a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0bac005ea0 tx=0x7f0bac005e30 comp rx=0 tx=0).stop 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 msgr2=0x7f0bb00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb00748a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40059d0 tx=0x7f0ba4005ef0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 shutdown_connections 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0b9c0a1260 0x7f0b9c0a3710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0bb0101db0 0x7f0bb0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bb0100230 0x7f0bb00748a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 --2- 172.21.15.31:0/1691773129 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb0101420 0x7f0bb0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 >> 172.21.15.31:0/1691773129 conn(0x7f0bb00fb810 msgr2=0x7f0bb00fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.804+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 shutdown_connections 2023-12-14T18:27:07.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.805+0000 7f0bb7798700 1 -- 172.21.15.31:0/1691773129 wait complete. 2023-12-14T18:27:07.807 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:07.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3025962870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f001035b0 msgr2=0x7f3f00105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3025962870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f001035b0 0x7f3f00105990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009990 tx=0x7f3ef4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3025962870 shutdown_connections 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3025962870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f00105ed0 0x7f3f00108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3025962870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f00100c90 0x7f3f00103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3025962870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f001035b0 0x7f3f00105990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3025962870 >> 172.21.15.31:0/3025962870 conn(0x7f3f000fa6f0 msgr2=0x7f3f000fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3025962870 shutdown_connections 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.955+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3025962870 wait complete. 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 Processor -- start 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 -- start start 2023-12-14T18:27:07.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 0x7f3f00195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f001035b0 0x7f3f00196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f00105ed0 0x7f3f0019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0019db40 con 0x7f3f00100c90 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0019dc80 con 0x7f3f00105ed0 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.956+0000 7f3f05b5b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0019de10 con 0x7f3f001035b0 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 0x7f3f00195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 0x7f3f00195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51442/0 (socket says 172.21.15.31:51442) 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 -- 172.21.15.31:0/3079548630 learned_addr learned my addr 172.21.15.31:0/3079548630 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3efffff700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f00105ed0 0x7f3f0019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3efeffd700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f001035b0 0x7f3f00196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 -- 172.21.15.31:0/3079548630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f001035b0 msgr2=0x7f3f00196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f001035b0 0x7f3f00196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 -- 172.21.15.31:0/3079548630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f00105ed0 msgr2=0x7f3f0019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f00105ed0 0x7f3f0019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 -- 172.21.15.31:0/3079548630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef40095f0 con 0x7f3f00100c90 2023-12-14T18:27:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3eff7fe700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 0x7f3f00195fc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d950 tx=0x7f3ef000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0009940 con 0x7f3f00100c90 2023-12-14T18:27:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0010bc20 con 0x7f3f00100c90 2023-12-14T18:27:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef0010460 con 0x7f3f00100c90 2023-12-14T18:27:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000f620 con 0x7f3f00100c90 2023-12-14T18:27:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.957+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0010ba90 con 0x7f3f00100c90 2023-12-14T18:27:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.959+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f000fc260 con 0x7f3f00100c90 2023-12-14T18:27:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.960+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ef00108d0 con 0x7f3f00100c90 2023-12-14T18:27:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.960+0000 7f3efcff9700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.961+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ef00ccc40 con 0x7f3f00100c90 2023-12-14T18:27:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.961+0000 7f3efeffd700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.961+0000 7f3efeffd700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b5c0 tx=0x7f3ef4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:07.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:07.964+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ef0095f20 con 0x7f3f00100c90 2023-12-14T18:27:08.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.175+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f000679f0 con 0x7f3f00100c90 2023-12-14T18:27:08.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.176+0000 7f3efcff9700 1 -- 172.21.15.31:0/3079548630 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f3ef0016090 con 0x7f3f00100c90 2023-12-14T18:27:08.178 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:08.178 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 msgr2=0x7f3ee80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400b5c0 tx=0x7f3ef4005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 msgr2=0x7f3f00195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 0x7f3f00195fc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000d950 tx=0x7f3ef000dc60 comp rx=0 tx=0).stop 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 shutdown_connections 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f00105ed0 0x7f3f0019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f00100c90 0x7f3f00195fc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 --2- 172.21.15.31:0/3079548630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f001035b0 0x7f3f00196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 >> 172.21.15.31:0/3079548630 conn(0x7f3f000fa6f0 msgr2=0x7f3f00101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:08.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 shutdown_connections 2023-12-14T18:27:08.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.179+0000 7f3f05b5b700 1 -- 172.21.15.31:0/3079548630 wait complete. 2023-12-14T18:27:08.193 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:08 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1835221376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:08.193 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:08 smithi031 ceph-mon[42085]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:08.193 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:08 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1691773129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:08.194 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:08.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 -- 172.21.15.31:0/524779598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 msgr2=0x7f963c105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 --2- 172.21.15.31:0/524779598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c105a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9630009990 tx=0x7f9630009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 -- 172.21.15.31:0/524779598 shutdown_connections 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 --2- 172.21.15.31:0/524779598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f963c105f40 0x7f963c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 --2- 172.21.15.31:0/524779598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c100d00 0x7f963c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 --2- 172.21.15.31:0/524779598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c105a00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 -- 172.21.15.31:0/524779598 >> 172.21.15.31:0/524779598 conn(0x7f963c0fa740 msgr2=0x7f963c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 -- 172.21.15.31:0/524779598 shutdown_connections 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.334+0000 7f96423bc700 1 -- 172.21.15.31:0/524779598 wait complete. 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.335+0000 7f96423bc700 1 Processor -- start 2023-12-14T18:27:08.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.335+0000 7f96423bc700 1 -- start start 2023-12-14T18:27:08.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f963c100d00 0x7f963c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c0fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c105f40 0x7f963c0ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0ff990 con 0x7f963c103620 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0ffb00 con 0x7f963c105f40 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1a0010 con 0x7f963c100d00 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c0fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c0fecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51464/0 (socket says 172.21.15.31:51464) 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 -- 172.21.15.31:0/3162308607 learned_addr learned my addr 172.21.15.31:0/3162308607 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963bfff700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f963c100d00 0x7f963c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f9640959700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c105f40 0x7f963c0ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 -- 172.21.15.31:0/3162308607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f963c100d00 msgr2=0x7f963c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f963c100d00 0x7f963c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 -- 172.21.15.31:0/3162308607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c105f40 msgr2=0x7f963c0ff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c105f40 0x7f963c0ff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 -- 172.21.15.31:0/3162308607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300095f0 con 0x7f963c103620 2023-12-14T18:27:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f9640959700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c105f40 0x7f963c0ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f963b7fe700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c0fecb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9630005910 tx=0x7f963000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963001d070 con 0x7f963c103620 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c10bbd0 con 0x7f963c103620 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f963000fb80 con 0x7f963c103620 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.336+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630017650 con 0x7f963c103620 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.337+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c10ba40 con 0x7f963c103620 2023-12-14T18:27:08.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.338+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c0fc2b0 con 0x7f963c103620 2023-12-14T18:27:08.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:08 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1835221376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:08.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:08 smithi204 ceph-mon[43866]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:08.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:08 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1691773129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.339+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9630022470 con 0x7f963c103620 2023-12-14T18:27:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.340+0000 7f96397fa700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f96240a11e0 0x7f96240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.340+0000 7f963bfff700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f96240a11e0 0x7f96240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.340+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f96300ce7b0 con 0x7f963c103620 2023-12-14T18:27:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.340+0000 7f963bfff700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f96240a11e0 0x7f96240a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f962c005950 tx=0x7f962c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.344+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9630097b20 con 0x7f963c103620 2023-12-14T18:27:08.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:08 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1835221376' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:08.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:08 smithi176 ceph-mon[43506]: pgmap v267: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:08.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:08 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1691773129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:08.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.554+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f963c061b00 con 0x7f963c103620 2023-12-14T18:27:08.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.555+0000 7f96397fa700 1 -- 172.21.15.31:0/3162308607 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f963009b140 con 0x7f963c103620 2023-12-14T18:27:08.556 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:08.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f96240a11e0 msgr2=0x7f96240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f96240a11e0 0x7f96240a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f962c005950 tx=0x7f962c00b410 comp rx=0 tx=0).stop 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 msgr2=0x7f963c0fecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c0fecb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9630005910 tx=0x7f963000f690 comp rx=0 tx=0).stop 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 shutdown_connections 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f963c105f40 0x7f963c0ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f96240a11e0 0x7f96240a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f963c100d00 0x7f963c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 --2- 172.21.15.31:0/3162308607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c103620 0x7f963c0fecb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 >> 172.21.15.31:0/3162308607 conn(0x7f963c0fa740 msgr2=0x7f963c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 shutdown_connections 2023-12-14T18:27:08.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.557+0000 7f96423bc700 1 -- 172.21.15.31:0/3162308607 wait complete. 2023-12-14T18:27:08.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 -- 172.21.15.31:0/1401843245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db8100340 msgr2=0x7f1db8100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/1401843245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db8100340 0x7f1db8100790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009990 tx=0x7f1da0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 -- 172.21.15.31:0/1401843245 shutdown_connections 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/1401843245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db8100cd0 0x7f1db8107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/1401843245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db80ff150 0x7f1db80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/1401843245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db8100340 0x7f1db8100790 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 -- 172.21.15.31:0/1401843245 >> 172.21.15.31:0/1401843245 conn(0x7f1db80fa770 msgr2=0x7f1db80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.712+0000 7f1dbd42f700 1 -- 172.21.15.31:0/1401843245 shutdown_connections 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.713+0000 7f1dbd42f700 1 -- 172.21.15.31:0/1401843245 wait complete. 2023-12-14T18:27:08.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.713+0000 7f1dbd42f700 1 Processor -- start 2023-12-14T18:27:08.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.713+0000 7f1dbd42f700 1 -- start start 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.713+0000 7f1dbd42f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 0x7f1db8195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db8100340 0x7f1db81964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db8100cd0 0x7f1db819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819db20 con 0x7f1db80ff150 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819dc60 con 0x7f1db8100340 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819ddf0 con 0x7f1db8100cd0 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 0x7f1db8195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 0x7f1db8195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51478/0 (socket says 172.21.15.31:51478) 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 -- 172.21.15.31:0/998192029 learned_addr learned my addr 172.21.15.31:0/998192029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dae7fc700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db8100340 0x7f1db81964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db77fe700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db8100cd0 0x7f1db819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 -- 172.21.15.31:0/998192029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db8100cd0 msgr2=0x7f1db819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db8100cd0 0x7f1db819d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 -- 172.21.15.31:0/998192029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db8100340 msgr2=0x7f1db81964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db8100340 0x7f1db81964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 -- 172.21.15.31:0/998192029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00095f0 con 0x7f1db80ff150 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dae7fc700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db8100340 0x7f1db81964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db6ffd700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 0x7f1db8195fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1da800e970 tx=0x7f1da800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8009790 con 0x7f1db80ff150 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1da80044e0 con 0x7f1db80ff150 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db810aab0 con 0x7f1db80ff150 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8003e30 con 0x7f1db80ff150 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.714+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db810a920 con 0x7f1db80ff150 2023-12-14T18:27:08.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.715+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db8067e40 con 0x7f1db80ff150 2023-12-14T18:27:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.717+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1da8003f90 con 0x7f1db80ff150 2023-12-14T18:27:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.717+0000 7f1db4ff9700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1da40a11e0 0x7f1da40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.717+0000 7f1dae7fc700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1da40a11e0 0x7f1da40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.717+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1da8014070 con 0x7f1db80ff150 2023-12-14T18:27:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.718+0000 7f1dae7fc700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1da40a11e0 0x7f1da40a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b2e0 tx=0x7f1da0005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:08.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.721+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1da8096390 con 0x7f1db80ff150 2023-12-14T18:27:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.934+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1db819ec80 con 0x7f1db80ff150 2023-12-14T18:27:08.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.935+0000 7f1db4ff9700 1 -- 172.21.15.31:0/998192029 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1da80999b0 con 0x7f1db80ff150 2023-12-14T18:27:08.936 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:08.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1da40a11e0 msgr2=0x7f1da40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1da40a11e0 0x7f1da40a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b2e0 tx=0x7f1da0005e80 comp rx=0 tx=0).stop 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 msgr2=0x7f1db8195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 0x7f1db8195fa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1da800e970 tx=0x7f1da800ed30 comp rx=0 tx=0).stop 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 shutdown_connections 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f1da40a11e0 0x7f1da40a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1db8100cd0 0x7f1db819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1db80ff150 0x7f1db8195fa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 --2- 172.21.15.31:0/998192029 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1db8100340 0x7f1db81964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 >> 172.21.15.31:0/998192029 conn(0x7f1db80fa770 msgr2=0x7f1db8069020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 shutdown_connections 2023-12-14T18:27:08.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:08.937+0000 7f1dbd42f700 1 -- 172.21.15.31:0/998192029 wait complete. 2023-12-14T18:27:08.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 -- 172.21.15.31:0/2311350087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ffe10 msgr2=0x7fbacc100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/2311350087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ffe10 0x7fbacc100280 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbac400b0a0 tx=0x7fbac400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 -- 172.21.15.31:0/2311350087 shutdown_connections 2023-12-14T18:27:09.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/2311350087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ffe10 0x7fbacc100280 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/2311350087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbacc0feb10 0x7fbacc0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/2311350087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbacc0ff460 0x7fbacc0ff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 -- 172.21.15.31:0/2311350087 >> 172.21.15.31:0/2311350087 conn(0x7fbacc0fa720 msgr2=0x7fbacc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 -- 172.21.15.31:0/2311350087 shutdown_connections 2023-12-14T18:27:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.092+0000 7fbad1a2b700 1 -- 172.21.15.31:0/2311350087 wait complete. 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 Processor -- start 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 -- start start 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbacc0feb10 0x7fbacc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 0x7fbacc1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbacc0ffe10 0x7fbacc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc19fe20 con 0x7fbacc0ff460 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc19ff60 con 0x7fbacc0feb10 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbad1a2b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbacc1a00f0 con 0x7fbacc0ffe10 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbac27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 0x7fbacc1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbac27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 0x7fbacc1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51506/0 (socket says 172.21.15.31:51506) 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbac27fc700 1 -- 172.21.15.31:0/744669074 learned_addr learned my addr 172.21.15.31:0/744669074 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbacb7fe700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbacc0ffe10 0x7fbacc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.093+0000 7fbacaffd700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbacc0feb10 0x7fbacc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbac27fc700 1 -- 172.21.15.31:0/744669074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbacc0ffe10 msgr2=0x7fbacc19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbac27fc700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbacc0ffe10 0x7fbacc19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbac27fc700 1 -- 172.21.15.31:0/744669074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbacc0feb10 msgr2=0x7fbacc1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbac27fc700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbacc0feb10 0x7fbacc1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbac27fc700 1 -- 172.21.15.31:0/744669074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac4009d00 con 0x7fbacc0ff460 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbac27fc700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 0x7fbacc1987e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbab400e9a0 tx=0x7fbab400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4009790 con 0x7fbacc0ff460 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbab4010430 con 0x7fbacc0ff460 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbacc10ddd0 con 0x7fbacc0ff460 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4018680 con 0x7fbacc0ff460 2023-12-14T18:27:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.094+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbacc10dc40 con 0x7fbacc0ff460 2023-12-14T18:27:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.095+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbacc1098f0 con 0x7fbacc0ff460 2023-12-14T18:27:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.096+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbab40105a0 con 0x7fbacc0ff460 2023-12-14T18:27:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.097+0000 7fbad0a29700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbab80a1260 0x7fbab80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.097+0000 7fbacaffd700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbab80a1260 0x7fbab80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.097+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbab4014070 con 0x7fbacc0ff460 2023-12-14T18:27:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.097+0000 7fbacaffd700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbab80a1260 0x7fbab80a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009c60 tx=0x7fbabc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.101+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbab4051730 con 0x7fbacc0ff460 2023-12-14T18:27:09.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:09 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3079548630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:09.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:09 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3162308607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:09.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:09 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/998192029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:09.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.308+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbacc061980 con 0x7fbacc0ff460 2023-12-14T18:27:09.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.308+0000 7fbad0a29700 1 -- 172.21.15.31:0/744669074 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbab4096200 con 0x7fbacc0ff460 2023-12-14T18:27:09.309 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbab80a1260 msgr2=0x7fbab80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbab80a1260 0x7fbab80a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009c60 tx=0x7fbabc009260 comp rx=0 tx=0).stop 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 msgr2=0x7fbacc1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 0x7fbacc1987e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbab400e9a0 tx=0x7fbab400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 shutdown_connections 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbab80a1260 0x7fbab80a3710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbacc0ffe10 0x7fbacc19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbacc0feb10 0x7fbacc1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 --2- 172.21.15.31:0/744669074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbacc0ff460 0x7fbacc1987e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 >> 172.21.15.31:0/744669074 conn(0x7fbacc0fa720 msgr2=0x7fbacc0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 shutdown_connections 2023-12-14T18:27:09.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.311+0000 7fbad1a2b700 1 -- 172.21.15.31:0/744669074 wait complete. 2023-12-14T18:27:09.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:09.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:09 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3079548630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:09.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:09 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3162308607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:09.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:09 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/998192029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 -- 172.21.15.31:0/3065021001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 msgr2=0x7fb2ec105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 --2- 172.21.15.31:0/3065021001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec105a40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009990 tx=0x7fb2e8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 -- 172.21.15.31:0/3065021001 shutdown_connections 2023-12-14T18:27:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 --2- 172.21.15.31:0/3065021001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2ec105f80 0x7fb2ec108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 --2- 172.21.15.31:0/3065021001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 0x7fb2ec103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 --2- 172.21.15.31:0/3065021001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec105a40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 -- 172.21.15.31:0/3065021001 >> 172.21.15.31:0/3065021001 conn(0x7fb2ec0fa740 msgr2=0x7fb2ec0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 -- 172.21.15.31:0/3065021001 shutdown_connections 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.465+0000 7fb2f35db700 1 -- 172.21.15.31:0/3065021001 wait complete. 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 Processor -- start 2023-12-14T18:27:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 -- start start 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 0x7fb2ec1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec0fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2ec105f80 0x7fb2ec0ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0ff9d0 con 0x7fb2ec103660 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0ffb40 con 0x7fb2ec100d40 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f35db700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec19fff0 con 0x7fb2ec105f80 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec0fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec0fecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51524/0 (socket says 172.21.15.31:51524) 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 -- 172.21.15.31:0/4236398921 learned_addr learned my addr 172.21.15.31:0/4236398921 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f1377700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 0x7fb2ec1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f1b78700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2ec105f80 0x7fb2ec0ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 -- 172.21.15.31:0/4236398921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2ec105f80 msgr2=0x7fb2ec0ff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2ec105f80 0x7fb2ec0ff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 -- 172.21.15.31:0/4236398921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 msgr2=0x7fb2ec1006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 0x7fb2ec1006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f0b76700 1 -- 172.21.15.31:0/4236398921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e80095f0 con 0x7fb2ec103660 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.466+0000 7fb2f1377700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 0x7fb2ec1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.467+0000 7fb2f0b76700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec0fecf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.467+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e801d070 con 0x7fb2ec103660 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.467+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e800fcb0 con 0x7fb2ec103660 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.467+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec10bbe0 con 0x7fb2ec103660 2023-12-14T18:27:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.467+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8017650 con 0x7fb2ec103660 2023-12-14T18:27:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.467+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec10ba50 con 0x7fb2ec103660 2023-12-14T18:27:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.468+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec0fc2b0 con 0x7fb2ec103660 2023-12-14T18:27:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.470+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb2e8021410 con 0x7fb2ec103660 2023-12-14T18:27:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.470+0000 7fb2e27fc700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2d80a11e0 0x7fb2d80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.470+0000 7fb2f1377700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2d80a11e0 0x7fb2d80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.470+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb2e80cea10 con 0x7fb2ec103660 2023-12-14T18:27:09.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.471+0000 7fb2f1377700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2d80a11e0 0x7fb2d80a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc011490 tx=0x7fb2dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.474+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb2e8097d80 con 0x7fb2ec103660 2023-12-14T18:27:09.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:09 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3079548630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:09.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:09 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3162308607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:09.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:09 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/998192029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.685+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2ec1a01b0 con 0x7fb2ec103660 2023-12-14T18:27:09.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.685+0000 7fb2e27fc700 1 -- 172.21.15.31:0/4236398921 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb2e809b3a0 con 0x7fb2ec103660 2023-12-14T18:27:09.686 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2d80a11e0 msgr2=0x7fb2d80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2d80a11e0 0x7fb2d80a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc011490 tx=0x7fb2dc009260 comp rx=0 tx=0).stop 2023-12-14T18:27:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 msgr2=0x7fb2ec0fecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec0fecf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e800f460 comp rx=0 tx=0).stop 2023-12-14T18:27:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 shutdown_connections 2023-12-14T18:27:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb2d80a11e0 0x7fb2d80a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2ec105f80 0x7fb2ec0ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2ec100d40 0x7fb2ec1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 --2- 172.21.15.31:0/4236398921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec103660 0x7fb2ec0fecf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 >> 172.21.15.31:0/4236398921 conn(0x7fb2ec0fa740 msgr2=0x7fb2ec101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 shutdown_connections 2023-12-14T18:27:09.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.688+0000 7fb2f35db700 1 -- 172.21.15.31:0/4236398921 wait complete. 2023-12-14T18:27:09.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 -- 172.21.15.31:0/1506882174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 msgr2=0x7fab3c108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 --2- 172.21.15.31:0/1506882174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c108360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fab34009990 tx=0x7fab34009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 -- 172.21.15.31:0/1506882174 shutdown_connections 2023-12-14T18:27:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 --2- 172.21.15.31:0/1506882174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c108360 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 --2- 172.21.15.31:0/1506882174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c100cc0 0x7fab3c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 --2- 172.21.15.31:0/1506882174 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 0x7fab3c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 -- 172.21.15.31:0/1506882174 >> 172.21.15.31:0/1506882174 conn(0x7fab3c0fa780 msgr2=0x7fab3c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 -- 172.21.15.31:0/1506882174 shutdown_connections 2023-12-14T18:27:09.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 -- 172.21.15.31:0/1506882174 wait complete. 2023-12-14T18:27:09.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.840+0000 7fab41480700 1 Processor -- start 2023-12-14T18:27:09.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 -- start start 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c100cc0 0x7fab3c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 0x7fab3c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19da30 con 0x7fab3c105f00 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19db70 con 0x7fab3c1035e0 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab41480700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c19dd00 con 0x7fab3c100cc0 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51544/0 (socket says 172.21.15.31:51544) 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab3b7fe700 1 -- 172.21.15.31:0/2231598232 learned_addr learned my addr 172.21.15.31:0/2231598232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab3affd700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c100cc0 0x7fab3c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.841+0000 7fab3a7fc700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 0x7fab3c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3b7fe700 1 -- 172.21.15.31:0/2231598232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c100cc0 msgr2=0x7fab3c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3b7fe700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c100cc0 0x7fab3c195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3b7fe700 1 -- 172.21.15.31:0/2231598232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 msgr2=0x7fab3c1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3b7fe700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 0x7fab3c1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3b7fe700 1 -- 172.21.15.31:0/2231598232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340095f0 con 0x7fab3c105f00 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3a7fc700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 0x7fab3c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab3b7fe700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c19d460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fab34005910 tx=0x7fab3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3401d070 con 0x7fab3c105f00 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab340047e0 con 0x7fab3c105f00 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c10bc00 con 0x7fab3c105f00 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34005400 con 0x7fab3c105f00 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.842+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c10ba70 con 0x7fab3c105f00 2023-12-14T18:27:09.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.843+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c0fc2f0 con 0x7fab3c105f00 2023-12-14T18:27:09.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.845+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fab34004950 con 0x7fab3c105f00 2023-12-14T18:27:09.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.846+0000 7fab23fff700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fab240a11e0 0x7fab240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.846+0000 7fab3affd700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fab240a11e0 0x7fab240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.846+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fab340ce200 con 0x7fab3c105f00 2023-12-14T18:27:09.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.846+0000 7fab3affd700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fab240a11e0 0x7fab240a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009550 tx=0x7fab2c0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:09.849+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fab34097510 con 0x7fab3c105f00 2023-12-14T18:27:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.108+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab3c02cfe0 con 0x7fab3c105f00 2023-12-14T18:27:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.109+0000 7fab23fff700 1 -- 172.21.15.31:0/2231598232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7fab3409ab30 con 0x7fab3c105f00 2023-12-14T18:27:10.110 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:10.110 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fab240a11e0 msgr2=0x7fab240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fab240a11e0 0x7fab240a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009550 tx=0x7fab2c0091b0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 msgr2=0x7fab3c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c19d460 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fab34005910 tx=0x7fab3400f690 comp rx=0 tx=0).stop 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 shutdown_connections 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab3c105f00 0x7fab3c19d460 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fab240a11e0 0x7fab240a3690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab3c100cc0 0x7fab3c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 --2- 172.21.15.31:0/2231598232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab3c1035e0 0x7fab3c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.111+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 >> 172.21.15.31:0/2231598232 conn(0x7fab3c0fa780 msgr2=0x7fab3c101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.112+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 shutdown_connections 2023-12-14T18:27:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.112+0000 7fab41480700 1 -- 172.21.15.31:0/2231598232 wait complete. 2023-12-14T18:27:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:10.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:10.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1585694308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec1006f0 msgr2=0x7fc4ec100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1585694308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec1006f0 0x7fc4ec100b00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009990 tx=0x7fc4e8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1585694308 shutdown_connections 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1585694308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec1075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1585694308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec1006f0 0x7fc4ec100b00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1585694308 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4ec0ff340 0x7fc4ec0ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.263+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1585694308 >> 172.21.15.31:0/1585694308 conn(0x7fc4ec0f9e80 msgr2=0x7fc4ec0fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1585694308 shutdown_connections 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1585694308 wait complete. 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 Processor -- start 2023-12-14T18:27:10.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 -- start start 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4ec0ff340 0x7fc4ec196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4ec1006f0 0x7fc4ec19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec19dba0 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec19dce0 con 0x7fc4ec1006f0 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.264+0000 7fc4f3c84700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec19de70 con 0x7fc4ec0ff340 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51552/0 (socket says 172.21.15.31:51552) 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 -- 172.21.15.31:0/1410376990 learned_addr learned my addr 172.21.15.31:0/1410376990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:10.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f1a20700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4ec0ff340 0x7fc4ec196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f2221700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4ec1006f0 0x7fc4ec19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 -- 172.21.15.31:0/1410376990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4ec0ff340 msgr2=0x7fc4ec196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4ec0ff340 0x7fc4ec196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 -- 172.21.15.31:0/1410376990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4ec1006f0 msgr2=0x7fc4ec19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4ec1006f0 0x7fc4ec19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 -- 172.21.15.31:0/1410376990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80095f0 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f121f700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec196560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e970 tx=0x7fc4e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0009790 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e000c7e0 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec10aae0 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0003c70 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.265+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec10a950 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.266+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec19e610 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.268+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc4e000cc50 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.269+0000 7fc4deffd700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc4d80a1260 0x7fc4d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.269+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc4e0014070 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.269+0000 7fc4f1a20700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc4d80a1260 0x7fc4d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.270+0000 7fc4f1a20700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc4d80a1260 0x7fc4d80a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b2e0 tx=0x7fc4e8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:10.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.272+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc4e00962f0 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/744669074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:10 smithi204 ceph-mon[43866]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:10 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4236398921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/744669074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:10 smithi031 ceph-mon[42085]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:10 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4236398921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:10.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.481+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4ec19ebc0 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.482+0000 7fc4deffd700 1 -- 172.21.15.31:0/1410376990 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7fc4e0099910 con 0x7fc4ec0ffcf0 2023-12-14T18:27:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:10.484 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:10.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc4d80a1260 msgr2=0x7fc4d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc4d80a1260 0x7fc4d80a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800b2e0 tx=0x7fc4e8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 msgr2=0x7fc4ec196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec196560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000e970 tx=0x7fc4e000ec80 comp rx=0 tx=0).stop 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 shutdown_connections 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc4ec1006f0 0x7fc4ec19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc4d80a1260 0x7fc4d80a3710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc4ec0ff340 0x7fc4ec196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 --2- 172.21.15.31:0/1410376990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4ec0ffcf0 0x7fc4ec196560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 >> 172.21.15.31:0/1410376990 conn(0x7fc4ec0f9e80 msgr2=0x7fc4ec0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 shutdown_connections 2023-12-14T18:27:10.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.485+0000 7fc4f3c84700 1 -- 172.21.15.31:0/1410376990 wait complete. 2023-12-14T18:27:10.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/744669074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:10.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:10 smithi176 ceph-mon[43506]: pgmap v268: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:10.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:10 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4236398921' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:10.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 -- 172.21.15.31:0/2730970403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0103600 msgr2=0x7f3ca01059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/2730970403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0103600 0x7f3ca01059e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009990 tx=0x7f3c9c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 -- 172.21.15.31:0/2730970403 shutdown_connections 2023-12-14T18:27:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/2730970403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ca0105f20 0x7f3ca0108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/2730970403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ca0100ce0 0x7f3ca01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/2730970403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0103600 0x7f3ca01059e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 -- 172.21.15.31:0/2730970403 >> 172.21.15.31:0/2730970403 conn(0x7f3ca00fa740 msgr2=0x7f3ca00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 -- 172.21.15.31:0/2730970403 shutdown_connections 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.638+0000 7f3ca85cf700 1 -- 172.21.15.31:0/2730970403 wait complete. 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 Processor -- start 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 -- start start 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 0x7f3ca0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ca0103600 0x7f3ca0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ca0105f20 0x7f3ca019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019da90 con 0x7f3ca0100ce0 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019dbd0 con 0x7f3ca0105f20 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca85cf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca019dd60 con 0x7f3ca0103600 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 0x7f3ca0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 0x7f3ca0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51562/0 (socket says 172.21.15.31:51562) 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 -- 172.21.15.31:0/1996805253 learned_addr learned my addr 172.21.15.31:0/1996805253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca6b6c700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ca0105f20 0x7f3ca019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca5b6a700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ca0103600 0x7f3ca0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 -- 172.21.15.31:0/1996805253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ca0103600 msgr2=0x7f3ca0196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ca0103600 0x7f3ca0196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 -- 172.21.15.31:0/1996805253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ca0105f20 msgr2=0x7f3ca019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.639+0000 7f3ca636b700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ca0105f20 0x7f3ca019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3ca636b700 1 -- 172.21.15.31:0/1996805253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0095f0 con 0x7f3ca0100ce0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3ca636b700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 0x7f3ca0195f10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000e9a0 tx=0x7f3c9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90009790 con 0x7f3ca0100ce0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c90010430 con 0x7f3ca0100ce0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca010bbe0 con 0x7f3ca0100ce0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90018680 con 0x7f3ca0100ce0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.640+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca010ba50 con 0x7f3ca0100ce0 2023-12-14T18:27:10.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.641+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca019e330 con 0x7f3ca0100ce0 2023-12-14T18:27:10.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.642+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3c900105a0 con 0x7f3ca0100ce0 2023-12-14T18:27:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.643+0000 7f3c977fe700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3c8c0a11e0 0x7f3c8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.643+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c90014070 con 0x7f3ca0100ce0 2023-12-14T18:27:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.643+0000 7f3ca5b6a700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3c8c0a11e0 0x7f3c8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:10.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.644+0000 7f3ca5b6a700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3c8c0a11e0 0x7f3c8c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b2e0 tx=0x7f3c9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:10.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.647+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3c900963f0 con 0x7f3ca0100ce0 2023-12-14T18:27:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.902+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ca019e9d0 con 0x7f3ca0100ce0 2023-12-14T18:27:10.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.902+0000 7f3c977fe700 1 -- 172.21.15.31:0/1996805253 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f3c90026090 con 0x7f3ca0100ce0 2023-12-14T18:27:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:10.904 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3c8c0a11e0 msgr2=0x7f3c8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3c8c0a11e0 0x7f3c8c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00b2e0 tx=0x7f3c9c005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 msgr2=0x7f3ca0195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:10.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 0x7f3ca0195f10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000e9a0 tx=0x7f3c9000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 shutdown_connections 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3ca0105f20 0x7f3ca019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ca0100ce0 0x7f3ca0195f10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3c8c0a11e0 0x7f3c8c0a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 --2- 172.21.15.31:0/1996805253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ca0103600 0x7f3ca0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 >> 172.21.15.31:0/1996805253 conn(0x7f3ca00fa740 msgr2=0x7f3ca0102140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 shutdown_connections 2023-12-14T18:27:10.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:10.905+0000 7f3ca85cf700 1 -- 172.21.15.31:0/1996805253 wait complete. 2023-12-14T18:27:10.908 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:10.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-14T18:25:55.286836+0000', 'modified': '2023-12-14T18:26:22.578285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24538, 'mds_1': 24536, 'mds_2': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24490': {'gid': 24490, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6837/3783760943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6836', 'nonce': 3783760943}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 3783760943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6837/2569700606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 2569700606}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 2569700606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/3595337412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3595337412}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3595337412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6837/1057123586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 1057123586}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 1057123586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.176:6839/2954758257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 2954758257}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 2954758257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.204:6839/1902825357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1902825357}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1902825357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-14T18:27:10.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-14T18:27:10.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 -- 172.21.15.31:0/383877937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101560 msgr2=0x7f08e41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/383877937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101560 0x7f08e41019b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009990 tx=0x7f08cc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 -- 172.21.15.31:0/383877937 shutdown_connections 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/383877937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4101ef0 0x7f08e41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/383877937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 0x7f08e40fe9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/383877937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101560 0x7f08e41019b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 -- 172.21.15.31:0/383877937 >> 172.21.15.31:0/383877937 conn(0x7f08e40f9fb0 msgr2=0x7f08e40fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 -- 172.21.15.31:0/383877937 shutdown_connections 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.059+0000 7f08e9a0d700 1 -- 172.21.15.31:0/383877937 wait complete. 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 Processor -- start 2023-12-14T18:27:11.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 -- start start 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 0x7f08e41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4101560 0x7f08e41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 0x7f08e419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e419fd30 con 0x7f08e4101ef0 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e419fe70 con 0x7f08e40fe590 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.060+0000 7f08e9a0d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e41a0000 con 0x7f08e4101560 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 0x7f08e419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 0x7f08e419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51586/0 (socket says 172.21.15.31:51586) 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 -- 172.21.15.31:0/156763418 learned_addr learned my addr 172.21.15.31:0/156763418 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08da7fc700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4101560 0x7f08e41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 0x7f08e41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 -- 172.21.15.31:0/156763418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4101560 msgr2=0x7f08e41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4101560 0x7f08e41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 -- 172.21.15.31:0/156763418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 msgr2=0x7f08e41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 0x7f08e41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 -- 172.21.15.31:0/156763418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0095f0 con 0x7f08e4101ef0 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 0x7f08e41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e37fe700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 0x7f08e419f760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08dc007ee0 tx=0x7f08dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e8a0b700 1 -- 172.21.15.31:0/156763418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc017070 con 0x7f08e4101ef0 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e8a0b700 1 -- 172.21.15.31:0/156763418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc00f040 con 0x7f08e4101ef0 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e410bc40 con 0x7f08e4101ef0 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e8a0b700 1 -- 172.21.15.31:0/156763418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc005740 con 0x7f08e4101ef0 2023-12-14T18:27:11.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.061+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e410bab0 con 0x7f08e4101ef0 2023-12-14T18:27:11.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.064+0000 7f08e8a0b700 1 -- 172.21.15.31:0/156763418 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f08dc00dc60 con 0x7f08e4101ef0 2023-12-14T18:27:11.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.064+0000 7f08e8a0b700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f08d00a11e0 0x7f08d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.064+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f08d00a11e0 0x7f08d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.064+0000 7f08e8a0b700 1 -- 172.21.15.31:0/156763418 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f08dc0cc5f0 con 0x7f08e4101ef0 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f08d00a11e0 0x7f08d00a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08e4068de0 tx=0x7f08d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 Processor -- start 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 -- start start 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcad9e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 0x563dcad9f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcada0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcac87590 con 0x563dcad9ee00 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcacf6820 con 0x563dcad9e4b0 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcacefcb0 con 0x563dcad9fcb0 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 0x563dcad9f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08da7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 0x563dcad9f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51588/0 (socket says 172.21.15.31:51588) 2023-12-14T18:27:11.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.065+0000 7f08da7fc700 1 -- 172.21.15.31:0/3782914712 learned_addr learned my addr 172.21.15.31:0/3782914712 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcad9e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08e37fe700 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcada0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08da7fc700 1 -- 172.21.15.31:0/3782914712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 msgr2=0x563dcada0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08da7fc700 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcada0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08da7fc700 1 -- 172.21.15.31:0/3782914712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 msgr2=0x563dcad9e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08da7fc700 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcad9e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08da7fc700 1 -- 172.21.15.31:0/3782914712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dcab71360 con 0x563dcad9ee00 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08e37fe700 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcada0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08da7fc700 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 0x563dcad9f720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f08cc015000 tx=0x7f08cc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2bf8c279043ffb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08d9ffb700 1 -- 172.21.15.31:0/3782914712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc01d070 con 0x563dcad9ee00 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08d9ffb700 1 -- 172.21.15.31:0/3782914712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08cc01e040 con 0x563dcad9ee00 2023-12-14T18:27:11.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.066+0000 7f08d9ffb700 1 -- 172.21.15.31:0/3782914712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004a20 con 0x563dcad9ee00 2023-12-14T18:27:11.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3782914712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 msgr2=0x563dcad9f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 0x563dcad9f720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f08cc015000 tx=0x7f08cc005bc0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3782914712 shutdown_connections 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcada0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcad9e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3782914712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9ee00 0x563dcad9f720 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3782914712 >> 172.21.15.31:0/3782914712 conn(0x563dcad93c50 msgr2=0x563dcad993e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3782914712 shutdown_connections 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3782914712 wait complete. 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.067+0000 7f08f5b9bb80 1 -- start start 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 Processor -- start 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 0x563dcada8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcadaf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcac8bf40 con 0x563dcad9e4b0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcac8bbe0 con 0x563dcad9ee00 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcab73650 con 0x563dcad9fcb0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08f5b9bb80 10 client.0 _ll_register_callbacks cb 0x563dcacaf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 0x563dcada8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 0x563dcada8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51594/0 (socket says 172.21.15.31:51594) 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08e2ffd700 1 -- 172.21.15.31:0/3141793269 learned_addr learned my addr 172.21.15.31:0/3141793269 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08da7fc700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.068+0000 7f08e37fe700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcadaf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08e2ffd700 1 -- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 msgr2=0x563dcadaf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcadaf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08e2ffd700 1 -- 172.21.15.31:0/3141793269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 msgr2=0x563dcada8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08e2ffd700 1 -- 172.21.15.31:0/3141793269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e4109020 con 0x563dcad9e4b0 2023-12-14T18:27:11.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 0x563dcada8050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08d4012c10 tx=0x7f08d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a2134fbebcde6f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08c2ffd700 1 -- 172.21.15.31:0/3141793269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4020840 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08c2ffd700 1 -- 172.21.15.31:0/3141793269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f08d4018070 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dcad91260 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08c2ffd700 1 -- 172.21.15.31:0/3141793269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d40209a0 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08f5b9bb80 10 client.15180 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563dcacb6120 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.069+0000 7f08c2ffd700 1 -- 172.21.15.31:0/3141793269 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563dcad91260 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08c2ffd700 1 -- 172.21.15.31:0/3141793269 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08d4029ba0 con 0x563dcad9e4b0 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08f5b9bb80 10 client.15180 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08f5b9bb80 10 client.15180 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08f5b9bb80 20 client.15180 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcadb33e0 0x563dcadb2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08f5b9bb80 4 client.15180 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:11.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563dcace8c90 con 0x563dcadb33e0 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08da7fc700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcadb33e0 0x563dcadb2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08c2ffd700 10 client.15180 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.070+0000 7f08da7fc700 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcadb33e0 0x563dcadb2780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3fd50d51796752d5 server_cookie=b6fb57648262796f in_seq=0 out_seq=0 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.071+0000 7f08c2ffd700 1 -- 172.21.15.31:0/3141793269 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563dcace8c90 con 0x563dcadb33e0 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.071+0000 7f08c2ffd700 10 client.15180 handle_command_reply: tid=0 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.071+0000 7f08f5b9bb80 1 client.15180 shutdown 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.071+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 msgr2=0x563dcada8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.071+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 0x563dcada8050 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08d4012c10 tx=0x7f08d40047c0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 shutdown_connections 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcadb33e0 0x563dcadb2780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9fcb0 0x563dcadaf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9e4b0 0x563dcada8050 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08c2ffd700 0 client.15180 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:11.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/3141793269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 >> 172.21.15.31:0/3141793269 conn(0x563dcad93c50 msgr2=0x563dcad97ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 shutdown_connections 2023-12-14T18:27:11.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/3141793269 wait complete. 2023-12-14T18:27:11.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.072+0000 7f08f5b9bb80 20 client.15180 trim_cache size 0 max 16384 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.090+0000 7f08f5b9bb80 1 Processor -- start 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.090+0000 7f08f5b9bb80 1 -- start start 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcada6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9ee00 0x563dcadaf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcad98980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcacbea50 con 0x563dcad9fcb0 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcac99580 con 0x563dcad9e4b0 2023-12-14T18:27:11.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcacae290 con 0x563dcad9ee00 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcad98980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcad98980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51608/0 (socket says 172.21.15.31:51608) 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 -- 172.21.15.31:0/7933765 learned_addr learned my addr 172.21.15.31:0/7933765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08da7fc700 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9ee00 0x563dcadaf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcada6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 -- 172.21.15.31:0/7933765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9ee00 msgr2=0x563dcadaf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9ee00 0x563dcadaf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 -- 172.21.15.31:0/7933765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 msgr2=0x563dcada6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcada6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 -- 172.21.15.31:0/7933765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dcada0350 con 0x563dcad9fcb0 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcada6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08e37fe700 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcad98980 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08dc095310 tx=0x7f08dc084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc089ac981b54cef server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08daffd700 1 -- 172.21.15.31:0/7933765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0b9070 con 0x563dcad9fcb0 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.091+0000 7f08daffd700 1 -- 172.21.15.31:0/7933765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc099590 con 0x563dcad9fcb0 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08daffd700 1 -- 172.21.15.31:0/7933765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0cc440 con 0x563dcad9fcb0 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/7933765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 msgr2=0x563dcad98980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcad98980 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08dc095310 tx=0x7f08dc084040 comp rx=0 tx=0).stop 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/7933765 shutdown_connections 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcad98980 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9e4b0 0x563dcada6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/7933765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9ee00 0x563dcadaf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/7933765 >> 172.21.15.31:0/7933765 conn(0x563dcad93c50 msgr2=0x563dcad99290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/7933765 shutdown_connections 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.092+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/7933765 wait complete. 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 -- start start 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 Processor -- start 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9e4b0 0x563dcada4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcadadb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcadae050 con 0x563dcad9fcb0 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcadae1c0 con 0x563dcad9ee00 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dcadae380 con 0x563dcad9e4b0 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08f5b9bb80 10 client.0 _ll_register_callbacks cb 0x563dcacaf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcadadb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcadadb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51614/0 (socket says 172.21.15.31:51614) 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08e37fe700 1 -- 172.21.15.31:0/555332354 learned_addr learned my addr 172.21.15.31:0/555332354 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9e4b0 0x563dcada4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.093+0000 7f08da7fc700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e37fe700 1 -- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9e4b0 msgr2=0x563dcada4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e37fe700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9e4b0 0x563dcada4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e37fe700 1 -- 172.21.15.31:0/555332354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 msgr2=0x563dcada4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e37fe700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e37fe700 1 -- 172.21.15.31:0/555332354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e410bab0 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9e4b0 0x563dcada4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08e37fe700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcadadb10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f08dc099520 tx=0x7f08dc097b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619174ac22c16a0b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08c2ffd700 1 -- 172.21.15.31:0/555332354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0c7070 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08cc01e040 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08c2ffd700 1 -- 172.21.15.31:0/555332354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f08dc0b7030 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08c2ffd700 1 -- 172.21.15.31:0/555332354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc0a19e0 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08f5b9bb80 10 client.15186 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563dcab1db10 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.094+0000 7f08c2ffd700 1 -- 172.21.15.31:0/555332354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08dc065020 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08c2ffd700 1 -- 172.21.15.31:0/555332354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08dc022250 con 0x563dcad9fcb0 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08f5b9bb80 10 client.15186 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08f5b9bb80 10 client.15186 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08f5b9bb80 20 client.15186 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcae0c430 0x563dcae09290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08f5b9bb80 4 client.15186 mds_command: new command op to 24514 tid=0[{"prefix": "status"}] 2023-12-14T18:27:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563dcace89b0 con 0x563dcae0c430 2023-12-14T18:27:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcae0c430 0x563dcae09290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08c2ffd700 10 client.15186 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.095+0000 7f08e2ffd700 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcae0c430 0x563dcae09290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bf17731c3cc7fc6 server_cookie=afbde18d77515cdc in_seq=0 out_seq=0 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08c2ffd700 1 -- 172.21.15.31:0/555332354 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563dcace89b0 con 0x563dcae0c430 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08c2ffd700 10 client.15186 handle_command_reply: tid=0 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 client.15186 shutdown 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 msgr2=0x563dcadadb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcadadb10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f08dc099520 tx=0x7f08dc097b40 comp rx=0 tx=0).stop 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 shutdown_connections 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563dcae0c430 0x563dcae09290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563dcad9fcb0 0x563dcadadb10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563dcad9e4b0 0x563dcada4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08c2ffd700 0 client.15186 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 --2- 172.21.15.31:0/555332354 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563dcad9ee00 0x563dcada4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 >> 172.21.15.31:0/555332354 conn(0x563dcad93c50 msgr2=0x563dcada5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 shutdown_connections 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 1 -- 172.21.15.31:0/555332354 wait complete. 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.096+0000 7f08f5b9bb80 20 client.15186 trim_cache size 0 max 16384 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24514, 2023-12-14T18:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 49.525525975000001, 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 24, 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 85, 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 90.931513598999999 2023-12-14T18:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f08d00a11e0 msgr2=0x7f08d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f08d00a11e0 0x7f08d00a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08e4068de0 tx=0x7f08d4009260 comp rx=0 tx=0).stop 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 msgr2=0x7f08e419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 0x7f08e419f760 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08dc007ee0 tx=0x7f08dc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 shutdown_connections 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08e4101ef0 0x7f08e419f760 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f08e40fe590 0x7f08e41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f08d00a11e0 0x7f08d00a3690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 --2- 172.21.15.31:0/156763418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4101560 0x7f08e41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 >> 172.21.15.31:0/156763418 conn(0x7f08e40f9fb0 msgr2=0x7f08e40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 shutdown_connections 2023-12-14T18:27:11.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.097+0000 7f08e9a0d700 1 -- 172.21.15.31:0/156763418 wait complete. 2023-12-14T18:27:11.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-14T18:27:11.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:11 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2231598232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:11 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1410376990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:11 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1996805253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 -- 172.21.15.31:0/2637807394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c240ff150 msgr2=0x7f8c240ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/2637807394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c240ff150 0x7f8c240ff560 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009990 tx=0x7f8c20009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 -- 172.21.15.31:0/2637807394 shutdown_connections 2023-12-14T18:27:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/2637807394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c24100cd0 0x7f8c24107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/2637807394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c240ff150 0x7f8c240ff560 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/2637807394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c24100340 0x7f8c24100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 -- 172.21.15.31:0/2637807394 >> 172.21.15.31:0/2637807394 conn(0x7f8c240fa770 msgr2=0x7f8c240fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 -- 172.21.15.31:0/2637807394 shutdown_connections 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.248+0000 7f8c2b14d700 1 -- 172.21.15.31:0/2637807394 wait complete. 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 Processor -- start 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 -- start start 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c240ff150 0x7f8c241030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 0x7f8c24101720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c24100cd0 0x7f8c24101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c24102400 con 0x7f8c24100340 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c24102570 con 0x7f8c240ff150 2023-12-14T18:27:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.249+0000 7f8c2b14d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c241a0080 con 0x7f8c24100cd0 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 0x7f8c24101720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 0x7f8c24101720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51636/0 (socket says 172.21.15.31:51636) 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1068869459 learned_addr learned my addr 172.21.15.31:0/1068869459 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c240ff150 0x7f8c241030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c296ea700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c24100cd0 0x7f8c24101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1068869459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c24100cd0 msgr2=0x7f8c24101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c24100cd0 0x7f8c24101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1068869459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c240ff150 msgr2=0x7f8c241030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c240ff150 0x7f8c241030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1068869459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14009520 con 0x7f8c24100340 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c240ff150 0x7f8c241030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 0x7f8c24101720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400eb30 tx=0x7f8c1400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c19ffb700 1 -- 172.21.15.31:0/1068869459 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14009880 con 0x7f8c24100340 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c19ffb700 1 -- 172.21.15.31:0/1068869459 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c140044e0 con 0x7f8c24100340 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c009c40 con 0x7f8c24100340 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.250+0000 7f8c19ffb700 1 -- 172.21.15.31:0/1068869459 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14010430 con 0x7f8c24100340 2023-12-14T18:27:11.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.251+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c200095f0 con 0x7f8c24100340 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.253+0000 7f8c19ffb700 1 -- 172.21.15.31:0/1068869459 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8c14003ee0 con 0x7f8c24100340 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.253+0000 7f8c19ffb700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8c0c0a11e0 0x7f8c0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.254+0000 7f8c19ffb700 1 -- 172.21.15.31:0/1068869459 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c14014070 con 0x7f8c24100340 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.254+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8c0c0a11e0 0x7f8c0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.254+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8c0c0a11e0 0x7f8c0c0a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000c00 tx=0x7f8c20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.254+0000 7f8c372dbb80 1 Processor -- start 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 -- start start 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f041968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04196e00 0x563f04197720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 0x563f04198120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0407f590 con 0x563f04197cb0 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f040ee820 con 0x563f04196e00 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f040e7cb0 con 0x563f041964b0 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 0x563f04198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 0x563f04198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51652/0 (socket says 172.21.15.31:51652) 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 -- 172.21.15.31:0/386604734 learned_addr learned my addr 172.21.15.31:0/386604734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04196e00 0x563f04197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f041968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 -- 172.21.15.31:0/386604734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 msgr2=0x563f041968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f041968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 -- 172.21.15.31:0/386604734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04196e00 msgr2=0x563f04197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04196e00 0x563f04197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.255+0000 7f8c296ea700 1 -- 172.21.15.31:0/386604734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f03f69360 con 0x563f04197cb0 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.256+0000 7f8c296ea700 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 0x563f04198120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009550 tx=0x7f8c1c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbfc58f8788de47 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.256+0000 7f8c0affd700 1 -- 172.21.15.31:0/386604734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00e070 con 0x563f04197cb0 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.256+0000 7f8c0affd700 1 -- 172.21.15.31:0/386604734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c1c00f040 con 0x563f04197cb0 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.256+0000 7f8c0affd700 1 -- 172.21.15.31:0/386604734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c0149d0 con 0x563f04197cb0 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.256+0000 7f8c372dbb80 1 -- 172.21.15.31:0/386604734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 msgr2=0x563f04198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.256+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 0x563f04198120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009550 tx=0x7f8c1c009920 comp rx=0 tx=0).stop 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 -- 172.21.15.31:0/386604734 shutdown_connections 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04197cb0 0x563f04198120 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f041968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/386604734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04196e00 0x563f04197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 -- 172.21.15.31:0/386604734 >> 172.21.15.31:0/386604734 conn(0x563f0418bc50 msgr2=0x563f041913e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 -- 172.21.15.31:0/386604734 shutdown_connections 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 -- 172.21.15.31:0/386604734 wait complete. 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 -- start start 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 Processor -- start 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f041964b0 0x563f041a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f041a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.257+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04197cb0 0x563f041a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f04083f40 con 0x563f04196e00 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f04083be0 con 0x563f041964b0 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f03f6b650 con 0x563f04197cb0 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c372dbb80 10 client.0 _ll_register_callbacks cb 0x563f040a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:11.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f041a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f041a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51662/0 (socket says 172.21.15.31:51662) 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 -- 172.21.15.31:0/182502779 learned_addr learned my addr 172.21.15.31:0/182502779 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c296ea700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04197cb0 0x563f041a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f041964b0 0x563f041a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 -- 172.21.15.31:0/182502779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04197cb0 msgr2=0x563f041a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04197cb0 0x563f041a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 -- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f041964b0 msgr2=0x563f041a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f041964b0 0x563f041a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 -- 172.21.15.31:0/182502779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c009280 con 0x563f04196e00 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f041964b0 0x563f041a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c296ea700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04197cb0 0x563f041a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f041a05f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c140039c0 tx=0x7f8c14016460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b3120691fdd87f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c03fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c03fff700 1 -- 172.21.15.31:0/182502779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1409a6f0 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c03fff700 1 -- 172.21.15.31:0/182502779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c1409e2c0 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f04189260 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.258+0000 7f8c03fff700 1 -- 172.21.15.31:0/182502779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c140cd150 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 10 client.15198 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563f040ae120 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c03fff700 1 -- 172.21.15.31:0/182502779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f04189260 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c03fff700 1 -- 172.21.15.31:0/182502779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c140a2250 con 0x563f04196e00 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 20 client.15198 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f041ab2a0 0x563f041aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 4 client.15198 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f040e0c90 con 0x563f041ab2a0 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.259+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f041ab2a0 0x563f041aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.260+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f041ab2a0 0x563f041aa6e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8dd31f62085662a server_cookie=893a381c758d260b in_seq=0 out_seq=0 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.260+0000 7f8c03fff700 10 client.15198 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c03fff700 1 -- 172.21.15.31:0/182502779 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f040e0c90 con 0x563f041ab2a0 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c03fff700 10 client.15198 handle_command_reply: tid=0 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 client.15198 shutdown 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 msgr2=0x563f041a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f041a05f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8c140039c0 tx=0x7f8c14016460 comp rx=0 tx=0).stop 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 shutdown_connections 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f041ab2a0 0x563f041aa6e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04197cb0 0x563f041a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f041964b0 0x563f041a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/182502779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f041a05f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c03fff700 0 client.15198 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 >> 172.21.15.31:0/182502779 conn(0x563f0418bc50 msgr2=0x563f0418fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 shutdown_connections 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 1 -- 172.21.15.31:0/182502779 wait complete. 2023-12-14T18:27:11.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.261+0000 7f8c372dbb80 20 client.15198 trim_cache size 0 max 16384 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 Processor -- start 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 -- start start 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f0419e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f0419e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f04091580 con 0x563f04196e00 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f040a6290 con 0x563f04197cb0 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f03ff3c50 con 0x563f041964b0 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f0419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f0419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51670/0 (socket says 172.21.15.31:51670) 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1995296486 learned_addr learned my addr 172.21.15.31:0/1995296486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f0419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c296ea700 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1995296486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 msgr2=0x563f0419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f0419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1995296486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 msgr2=0x563f041a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c1bfff700 1 -- 172.21.15.31:0/1995296486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f040ade70 con 0x563f04196e00 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.280+0000 7f8c296ea700 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f0419e540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400c660 tx=0x7f8c14096700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ef48d10b6ea68c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/1995296486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140cd150 con 0x563f04196e00 2023-12-14T18:27:11.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/1995296486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c1409e2c0 con 0x563f04196e00 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c0b7fe700 1 -- 172.21.15.31:0/1995296486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1409a4f0 con 0x563f04196e00 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 -- 172.21.15.31:0/1995296486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 msgr2=0x563f0419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f0419e540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400c660 tx=0x7f8c14096700 comp rx=0 tx=0).stop 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 -- 172.21.15.31:0/1995296486 shutdown_connections 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f041964b0 0x563f0419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/1995296486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f04196e00 0x563f0419e540 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.281+0000 7f8c372dbb80 1 -- 172.21.15.31:0/1995296486 >> 172.21.15.31:0/1995296486 conn(0x563f0418bc50 msgr2=0x563f041aa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 -- 172.21.15.31:0/1995296486 shutdown_connections 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 -- 172.21.15.31:0/1995296486 wait complete. 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 -- start start 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 Processor -- start 2023-12-14T18:27:11.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 0x563f041a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04196e00 0x563f041a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f041a2880 con 0x563f041964b0 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f041a29f0 con 0x563f04197cb0 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f041ff140 con 0x563f04196e00 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c372dbb80 10 client.0 _ll_register_callbacks cb 0x7f8c14099dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c28ee9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 0x563f041a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c28ee9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 0x563f041a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51672/0 (socket says 172.21.15.31:51672) 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.282+0000 7f8c28ee9700 1 -- 172.21.15.31:0/4057477839 learned_addr learned my addr 172.21.15.31:0/4057477839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04196e00 0x563f041a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c28ee9700 1 -- 172.21.15.31:0/4057477839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04196e00 msgr2=0x563f041a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04196e00 0x563f041a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c296ea700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c28ee9700 1 -- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 msgr2=0x563f041a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c28ee9700 1 -- 172.21.15.31:0/4057477839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f040ae120 con 0x563f041964b0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c296ea700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c28ee9700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 0x563f041a1e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8c241001a0 tx=0x7f8c2001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc55b89c4e4d480 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c03fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c03fff700 1 -- 172.21.15.31:0/4057477839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20035500 con 0x563f041964b0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c03fff700 1 -- 172.21.15.31:0/4057477839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c200104f0 con 0x563f041964b0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c1c00f040 con 0x563f041964b0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c03fff700 1 -- 172.21.15.31:0/4057477839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c200106f0 con 0x563f041964b0 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c372dbb80 10 client.15204 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:11.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.283+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563f04094950 con 0x563f041964b0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c03fff700 1 -- 172.21.15.31:0/4057477839 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c1c00f040 con 0x563f041964b0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c03fff700 1 -- 172.21.15.31:0/4057477839 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c20024720 con 0x563f041964b0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c372dbb80 10 client.15204 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c372dbb80 10 client.15204 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c372dbb80 20 client.15204 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f04203ec0 0x563f042007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c372dbb80 4 client.15204 mds_command: new command op to 24536 tid=0[{"prefix": "status"}] 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563f040e09b0 con 0x563f04203ec0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.284+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f04203ec0 0x563f042007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c03fff700 10 client.15204 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c1bfff700 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f04203ec0 0x563f042007f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4385eee90df99636 server_cookie=b69fa7be7cffc115 in_seq=0 out_seq=0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c03fff700 1 -- 172.21.15.31:0/4057477839 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563f040e09b0 con 0x563f04203ec0 2023-12-14T18:27:11.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c03fff700 10 client.15204 handle_command_reply: tid=0 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 client.15204 shutdown 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 msgr2=0x563f041a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 0x563f041a1e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8c241001a0 tx=0x7f8c2001a050 comp rx=0 tx=0).stop 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 shutdown_connections 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563f04197cb0 0x563f041a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563f041964b0 0x563f041a1e00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563f04203ec0 0x563f042007f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 --2- 172.21.15.31:0/4057477839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563f04196e00 0x563f041a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c03fff700 0 client.15204 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 >> 172.21.15.31:0/4057477839 conn(0x563f0418bc50 msgr2=0x563f0419e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 shutdown_connections 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 1 -- 172.21.15.31:0/4057477839 wait complete. 2023-12-14T18:27:11.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.285+0000 7f8c372dbb80 20 client.15204 trim_cache size 0 max 16384 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24536, 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 50.728602944999999, 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 24, 2023-12-14T18:27:11.294 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-12-14T18:27:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 85, 2023-12-14T18:27:11.295 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 81.960841673000004 2023-12-14T18:27:11.295 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:11.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8c0c0a11e0 msgr2=0x7f8c0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8c0c0a11e0 0x7f8c0c0a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000c00 tx=0x7f8c20005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 msgr2=0x7f8c24101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 0x7f8c24101720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c1400eb30 tx=0x7f8c1400eef0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 shutdown_connections 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8c0c0a11e0 0x7f8c0c0a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c24100cd0 0x7f8c24101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8c240ff150 0x7f8c241030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 --2- 172.21.15.31:0/1068869459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c24100340 0x7f8c24101720 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 >> 172.21.15.31:0/1068869459 conn(0x7f8c240fa770 msgr2=0x7f8c24068dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 shutdown_connections 2023-12-14T18:27:11.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.286+0000 7f8c2b14d700 1 -- 172.21.15.31:0/1068869459 wait complete. 2023-12-14T18:27:11.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-14T18:27:11.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:11 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2231598232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:11 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1410376990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:11 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1996805253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 -- 172.21.15.31:0/3516080247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 msgr2=0x7f8b48108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3516080247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48108670 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b0a0 tx=0x7f8b4000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 -- 172.21.15.31:0/3516080247 shutdown_connections 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3516080247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48108670 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3516080247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b48100230 0x7f8b48100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3516080247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48101420 0x7f8b48101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 -- 172.21.15.31:0/3516080247 >> 172.21.15.31:0/3516080247 conn(0x7f8b480fb850 msgr2=0x7f8b480fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 -- 172.21.15.31:0/3516080247 shutdown_connections 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.440+0000 7f8b502ef700 1 -- 172.21.15.31:0/3516080247 wait complete. 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 Processor -- start 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 -- start start 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48100230 0x7f8b480748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b48101420 0x7f8b48072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48073bc0 con 0x7f8b48101db0 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b48073d30 con 0x7f8b48101420 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b502ef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481a0120 con 0x7f8b48100230 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b4e88c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b4e08b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48100230 0x7f8b480748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b4e88c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51694/0 (socket says 172.21.15.31:51694) 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b4e88c700 1 -- 172.21.15.31:0/3931533928 learned_addr learned my addr 172.21.15.31:0/3931533928 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.441+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b48101420 0x7f8b48072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e88c700 1 -- 172.21.15.31:0/3931533928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48100230 msgr2=0x7f8b480748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48100230 0x7f8b480748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e88c700 1 -- 172.21.15.31:0/3931533928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b48101420 msgr2=0x7f8b48072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b48101420 0x7f8b48072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e88c700 1 -- 172.21.15.31:0/3931533928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40009d00 con 0x7f8b48101db0 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48100230 0x7f8b480748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48073420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b780 tx=0x7f8b4000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b3f7fe700 1 -- 172.21.15.31:0/3931533928 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000e040 con 0x7f8b48101db0 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4810bba0 con 0x7f8b48101db0 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b3f7fe700 1 -- 172.21.15.31:0/3931533928 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b40004e60 con 0x7f8b48101db0 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b3f7fe700 1 -- 172.21.15.31:0/3931533928 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001b6b0 con 0x7f8b48101db0 2023-12-14T18:27:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.442+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4810ba10 con 0x7f8b48101db0 2023-12-14T18:27:11.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.445+0000 7f8b3f7fe700 1 -- 172.21.15.31:0/3931533928 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8b4001b810 con 0x7f8b48101db0 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.445+0000 7f8b3f7fe700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b340a11e0 0x7f8b340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.446+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b340a11e0 0x7f8b340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.446+0000 7f8b3f7fe700 1 -- 172.21.15.31:0/3931533928 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8b400ce580 con 0x7f8b48101db0 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.446+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b340a11e0 0x7f8b340a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b48101280 tx=0x7f8b38007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.446+0000 7f8b5c47db80 1 Processor -- start 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 -- start start 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 0x557559d048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d04e00 0x557559d05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559bed590 con 0x557559d044b0 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559c5c820 con 0x557559d04e00 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559c55cb0 con 0x557559d05cb0 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 0x557559d048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 0x557559d048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51706/0 (socket says 172.21.15.31:51706) 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 -- 172.21.15.31:0/381062311 learned_addr learned my addr 172.21.15.31:0/381062311 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d04e00 0x557559d05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 -- 172.21.15.31:0/381062311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 msgr2=0x557559d06120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d06120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 -- 172.21.15.31:0/381062311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d04e00 msgr2=0x557559d05720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d04e00 0x557559d05720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e08b700 1 -- 172.21.15.31:0/381062311 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557559ad7360 con 0x557559d044b0 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.447+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 0x557559d048c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8b380115d0 tx=0x7f8b38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e7109b284b6b14 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b3cdac700 1 -- 172.21.15.31:0/381062311 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3801e620 con 0x557559d044b0 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b3cdac700 1 -- 172.21.15.31:0/381062311 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b38015070 con 0x557559d044b0 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b3cdac700 1 -- 172.21.15.31:0/381062311 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3801c9d0 con 0x557559d044b0 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b5c47db80 1 -- 172.21.15.31:0/381062311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 msgr2=0x557559d048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 0x557559d048c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8b380115d0 tx=0x7f8b38013000 comp rx=0 tx=0).stop 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b5c47db80 1 -- 172.21.15.31:0/381062311 shutdown_connections 2023-12-14T18:27:11.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d06120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.448+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d044b0 0x557559d048c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/381062311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d04e00 0x557559d05720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- 172.21.15.31:0/381062311 >> 172.21.15.31:0/381062311 conn(0x557559cf9c50 msgr2=0x557559cff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- 172.21.15.31:0/381062311 shutdown_connections 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- 172.21.15.31:0/381062311 wait complete. 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- start start 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 Processor -- start 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d044b0 0x557559d0e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d0e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d05cb0 0x557559d15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559bf1f40 con 0x557559d04e00 2023-12-14T18:27:11.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559bf1be0 con 0x557559d05cb0 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559ad9650 con 0x557559d044b0 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b5c47db80 10 client.0 _ll_register_callbacks cb 0x557559c158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b4d88a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d0e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b4d88a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d0e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51710/0 (socket says 172.21.15.31:51710) 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.449+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1125138917 learned_addr learned my addr 172.21.15.31:0/1125138917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d05cb0 0x557559d15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d044b0 0x557559d0e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d044b0 msgr2=0x557559d0e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d044b0 0x557559d0e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1125138917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d05cb0 msgr2=0x557559d15670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d05cb0 0x557559d15670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1125138917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b38010eb0 con 0x557559d04e00 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d044b0 0x557559d0e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d05cb0 0x557559d15670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d0e630 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b44012770 tx=0x7f8b44012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15595550e27b73f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1125138917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44017070 con 0x557559d04e00 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1125138917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8b4400f5c0 con 0x557559d04e00 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557559cf7260 con 0x557559d04e00 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1125138917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b440122e0 con 0x557559d04e00 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b5c47db80 10 client.15216 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.450+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557559c1c120 con 0x557559d04e00 2023-12-14T18:27:11.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1125138917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557559cf7260 con 0x557559d04e00 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1125138917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b440109c0 con 0x557559d04e00 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b5c47db80 10 client.15216 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b5c47db80 10 client.15216 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b5c47db80 20 client.15216 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d193e0 0x557559d18790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b5c47db80 4 client.15216 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557559c4ec90 con 0x557559d193e0 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.451+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d193e0 0x557559d18790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.452+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d193e0 0x557559d18790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f92ec72af916a6 server_cookie=6a96f2b20dd6bea in_seq=0 out_seq=0 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.452+0000 7f8b31ffb700 10 client.15216 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.452+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1125138917 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557559c4ec90 con 0x557559d193e0 2023-12-14T18:27:11.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.452+0000 7f8b31ffb700 10 client.15216 handle_command_reply: tid=0 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.452+0000 7f8b5c47db80 1 client.15216 shutdown 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 msgr2=0x557559d0e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d0e630 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b44012770 tx=0x7f8b44012b30 comp rx=0 tx=0).stop 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 shutdown_connections 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d193e0 0x557559d18790 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d05cb0 0x557559d15670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d044b0 0x557559d0e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b31ffb700 0 client.15216 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1125138917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d0e630 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 >> 172.21.15.31:0/1125138917 conn(0x557559cf9c50 msgr2=0x557559cfdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 shutdown_connections 2023-12-14T18:27:11.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1125138917 wait complete. 2023-12-14T18:27:11.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.453+0000 7f8b5c47db80 20 client.15216 trim_cache size 0 max 16384 2023-12-14T18:27:11.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.471+0000 7f8b5c47db80 1 Processor -- start 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.471+0000 7f8b5c47db80 1 -- start start 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559cff230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d15630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559cfe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559d006b0 con 0x557559d04e00 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559c24a50 con 0x557559d044b0 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559c14290 con 0x557559d05cb0 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d15630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d15630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51722/0 (socket says 172.21.15.31:51722) 2023-12-14T18:27:11.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 -- 172.21.15.31:0/2873123811 learned_addr learned my addr 172.21.15.31:0/2873123811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559cff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559cfe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 -- 172.21.15.31:0/2873123811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 msgr2=0x557559cfe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559cfe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 -- 172.21.15.31:0/2873123811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 msgr2=0x557559cff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559cff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 -- 172.21.15.31:0/2873123811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557559d0f040 con 0x557559d04e00 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559cff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d15630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b440122e0 tx=0x7f8b44012310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94083d2881afca98 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.472+0000 7f8b3d5ad700 1 -- 172.21.15.31:0/2873123811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44017040 con 0x557559d04e00 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b3d5ad700 1 -- 172.21.15.31:0/2873123811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b44005590 con 0x557559d04e00 2023-12-14T18:27:11.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b3d5ad700 1 -- 172.21.15.31:0/2873123811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44003bf0 con 0x557559d04e00 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 -- 172.21.15.31:0/2873123811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 msgr2=0x557559d15630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d15630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b440122e0 tx=0x7f8b44012310 comp rx=0 tx=0).stop 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 -- 172.21.15.31:0/2873123811 shutdown_connections 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559cfe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559cff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/2873123811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d15630 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 -- 172.21.15.31:0/2873123811 >> 172.21.15.31:0/2873123811 conn(0x557559cf9c50 msgr2=0x557559d0ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 -- 172.21.15.31:0/2873123811 shutdown_connections 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 -- 172.21.15.31:0/2873123811 wait complete. 2023-12-14T18:27:11.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.473+0000 7f8b5c47db80 1 -- start start 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 Processor -- start 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559d10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d10990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559d11070 con 0x557559d04e00 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559d6d7e0 con 0x557559d044b0 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557559d6d920 con 0x557559d05cb0 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b5c47db80 10 client.0 _ll_register_callbacks cb 0x7f8b44009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d10990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d10990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51730/0 (socket says 172.21.15.31:51730) 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1251546861 learned_addr learned my addr 172.21.15.31:0/1251546861 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4e88c700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559d10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 msgr2=0x557559d6d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d6d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1251546861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 msgr2=0x557559d10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559d10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.474+0000 7f8b4d88a700 1 -- 172.21.15.31:0/1251546861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557559c1c120 con 0x557559d04e00 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b4d88a700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d10990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b44010dc0 tx=0x7f8b440096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8fd29cb85b660 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:11.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1251546861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44017070 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1251546861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8b44000e40 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b38015070 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1251546861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b44010b30 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 10 client.15222 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557559a83b10 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1251546861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b38015070 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1251546861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8b4400fc40 con 0x557559d04e00 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 10 client.15222 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 20 client.15222 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d721f0 0x557559d6eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:11.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 4 client.15222 mds_command: new command op to 24538 tid=0[{"prefix": "status"}] 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.475+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557559c4e9b0 con 0x557559d721f0 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d721f0 0x557559d6eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b4e08b700 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d721f0 0x557559d6eba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=460f8ab234787bea server_cookie=497198795bdd918e in_seq=0 out_seq=0 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b31ffb700 10 client.15222 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b31ffb700 1 -- 172.21.15.31:0/1251546861 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x557559c4e9b0 con 0x557559d721f0 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b31ffb700 10 client.15222 handle_command_reply: tid=0 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b5c47db80 1 client.15222 shutdown 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 msgr2=0x557559d10990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.476+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d10990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b44010dc0 tx=0x7f8b440096d0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 shutdown_connections 2023-12-14T18:27:11.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557559d721f0 0x557559d6eba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557559d05cb0 0x557559d6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557559d044b0 0x557559d10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b31ffb700 0 client.15222 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 --2- 172.21.15.31:0/1251546861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557559d04e00 0x557559d10990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 >> 172.21.15.31:0/1251546861 conn(0x557559cf9c50 msgr2=0x557559d0c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 shutdown_connections 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 1 -- 172.21.15.31:0/1251546861 wait complete. 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b5c47db80 20 client.15222 trim_cache size 0 max 16384 2023-12-14T18:27:11.486 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24538, 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 76.186924055999995, 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 24, 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 85, 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 87.769988885000004 2023-12-14T18:27:11.487 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:11.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b340a11e0 msgr2=0x7f8b340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b340a11e0 0x7f8b340a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8b48101280 tx=0x7f8b38007400 comp rx=0 tx=0).stop 2023-12-14T18:27:11.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 msgr2=0x7f8b48073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48073420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000b780 tx=0x7f8b4000bd00 comp rx=0 tx=0).stop 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 shutdown_connections 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b48101db0 0x7f8b48073420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8b340a11e0 0x7f8b340a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b48100230 0x7f8b480748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 --2- 172.21.15.31:0/3931533928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b48101420 0x7f8b48072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 >> 172.21.15.31:0/3931533928 conn(0x7f8b480fb850 msgr2=0x7f8b481048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 shutdown_connections 2023-12-14T18:27:11.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:11.477+0000 7f8b502ef700 1 -- 172.21.15.31:0/3931533928 wait complete. 2023-12-14T18:27:11.492 INFO:tasks.cephadm:Setup complete, yielding 2023-12-14T18:27:11.492 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-14T18:27:11.502 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:27:11.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch status' 2023-12-14T18:27:11.503 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:11 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2231598232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.503 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:11 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1410376990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.503 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:11 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1996805253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:11.854 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:12.338 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:12 smithi031 ceph-mon[42085]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:12.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:12 smithi204 ceph-mon[43866]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:12.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:12 smithi176 ceph-mon[43506]: pgmap v269: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 -- 172.21.15.31:0/203314427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 msgr2=0x7fc578108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/203314427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc578108430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc57000b0a0 tx=0x7fc57000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 -- 172.21.15.31:0/203314427 shutdown_connections 2023-12-14T18:27:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/203314427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc578108430 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:12.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/203314427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578100e10 0x7fc5781031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/203314427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578103730 0x7fc578105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 -- 172.21.15.31:0/203314427 >> 172.21.15.31:0/203314427 conn(0x7fc5780fa7f0 msgr2=0x7fc5780fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 -- 172.21.15.31:0/203314427 shutdown_connections 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.915+0000 7fc57d3b7700 1 -- 172.21.15.31:0/203314427 wait complete. 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 Processor -- start 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 -- start start 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 0x7fc578100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578103730 0x7fc5780fedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc5780ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ffa80 con 0x7fc578106050 2023-12-14T18:27:12.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5780ffbf0 con 0x7fc578100e10 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc57d3b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5781a0140 con 0x7fc578103730 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.916+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc5780ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 0x7fc578100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc5780ff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51744/0 (socket says 172.21.15.31:51744) 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc576ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 0x7fc578100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:51446/0 (socket says 172.21.15.31:51446) 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 -- 172.21.15.31:0/2882367483 learned_addr learned my addr 172.21.15.31:0/2882367483 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5767fc700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578103730 0x7fc5780fedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 -- 172.21.15.31:0/2882367483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578103730 msgr2=0x7fc5780fedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578103730 0x7fc5780fedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:12.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 -- 172.21.15.31:0/2882367483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 msgr2=0x7fc578100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 0x7fc578100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 -- 172.21.15.31:0/2882367483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570009d00 con 0x7fc578106050 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5767fc700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578103730 0x7fc5780fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc576ffd700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 0x7fc578100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc5777fe700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc5780ff300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5700062a0 tx=0x7fc57000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000e040 con 0x7fc578106050 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57810bcb0 con 0x7fc578106050 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5700094d0 con 0x7fc578106050 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57001b6c0 con 0x7fc578106050 2023-12-14T18:27:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.917+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57810bb20 con 0x7fc578106050 2023-12-14T18:27:12.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.918+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5780fc360 con 0x7fc578106050 2023-12-14T18:27:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.920+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc57001d470 con 0x7fc578106050 2023-12-14T18:27:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.920+0000 7fc56ffff700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5640a11e0 0x7fc5640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.920+0000 7fc576ffd700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5640a11e0 0x7fc5640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.920+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5700cdf20 con 0x7fc578106050 2023-12-14T18:27:12.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.921+0000 7fc576ffd700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5640a11e0 0x7fc5640a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc568011490 tx=0x7fc568009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:12.925+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc570097290 con 0x7fc578106050 2023-12-14T18:27:13.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.151+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc578061a30 con 0x7fc5640a11e0 2023-12-14T18:27:13.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.153+0000 7fc56ffff700 1 -- 172.21.15.31:0/2882367483 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc578061a30 con 0x7fc5640a11e0 2023-12-14T18:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2023-12-14T18:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2023-12-14T18:27:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5640a11e0 msgr2=0x7fc5640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5640a11e0 0x7fc5640a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc568011490 tx=0x7fc568009260 comp rx=0 tx=0).stop 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 msgr2=0x7fc5780ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc5780ff300 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc5700062a0 tx=0x7fc57000bd60 comp rx=0 tx=0).stop 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 shutdown_connections 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc578106050 0x7fc5780ff300 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc578100e10 0x7fc578100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc5640a11e0 0x7fc5640a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 --2- 172.21.15.31:0/2882367483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc578103730 0x7fc5780fedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 >> 172.21.15.31:0/2882367483 conn(0x7fc5780fa7f0 msgr2=0x7fc578101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 shutdown_connections 2023-12-14T18:27:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:13.155+0000 7fc57d3b7700 1 -- 172.21.15.31:0/2882367483 wait complete. 2023-12-14T18:27:13.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-14T18:27:13.913 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:14.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:14 smithi204 ceph-mon[43866]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:14.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:14 smithi204 ceph-mon[43866]: pgmap v270: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:14 smithi031 ceph-mon[42085]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:14 smithi031 ceph-mon[42085]: pgmap v270: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:14.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:14 smithi176 ceph-mon[43506]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:14.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:14 smithi176 ceph-mon[43506]: pgmap v270: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 -- 172.21.15.31:0/1918728810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc1038e0 msgr2=0x7f3efc105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 --2- 172.21.15.31:0/1918728810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc1038e0 0x7f3efc105cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009990 tx=0x7f3ef0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 -- 172.21.15.31:0/1918728810 shutdown_connections 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 --2- 172.21.15.31:0/1918728810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3efc106200 0x7f3efc108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 --2- 172.21.15.31:0/1918728810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc100fc0 0x7f3efc1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 --2- 172.21.15.31:0/1918728810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc1038e0 0x7f3efc105cc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 -- 172.21.15.31:0/1918728810 >> 172.21.15.31:0/1918728810 conn(0x7f3efc0faaa0 msgr2=0x7f3efc0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 -- 172.21.15.31:0/1918728810 shutdown_connections 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 -- 172.21.15.31:0/1918728810 wait complete. 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.967+0000 7f3f04560700 1 Processor -- start 2023-12-14T18:27:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 -- start start 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 0x7f3efc196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3efc1038e0 0x7f3efc196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc106200 0x7f3efc19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc19ddb0 con 0x7f3efc100fc0 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc19def0 con 0x7f3efc1038e0 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f04560700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc19e080 con 0x7f3efc106200 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 0x7f3efc196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 0x7f3efc196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40246/0 (socket says 172.21.15.31:40246) 2023-12-14T18:27:14.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 -- 172.21.15.31:0/2472421037 learned_addr learned my addr 172.21.15.31:0/2472421037 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f01afb700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3efc1038e0 0x7f3efc196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f02afd700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc106200 0x7f3efc19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 -- 172.21.15.31:0/2472421037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc106200 msgr2=0x7f3efc19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc106200 0x7f3efc19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 -- 172.21.15.31:0/2472421037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3efc1038e0 msgr2=0x7f3efc196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3efc1038e0 0x7f3efc196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.968+0000 7f3f022fc700 1 -- 172.21.15.31:0/2472421037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00095f0 con 0x7f3efc100fc0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.969+0000 7f3f022fc700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 0x7f3efc196220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800d9b0 tx=0x7f3ef800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.969+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8004220 con 0x7f3efc100fc0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.969+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc10bec0 con 0x7f3efc100fc0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.969+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef8007640 con 0x7f3efc100fc0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.969+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8005650 con 0x7f3efc100fc0 2023-12-14T18:27:14.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.969+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc10bd30 con 0x7f3efc100fc0 2023-12-14T18:27:14.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.970+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc0fc610 con 0x7f3efc100fc0 2023-12-14T18:27:14.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.971+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ef8004750 con 0x7f3efc100fc0 2023-12-14T18:27:14.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.972+0000 7f3eef7fe700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:14.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.972+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ef80cd210 con 0x7f3efc100fc0 2023-12-14T18:27:14.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.972+0000 7f3f01afb700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:14.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.972+0000 7f3f01afb700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000b680 tx=0x7f3ef0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:14.976+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ef8096490 con 0x7f3efc100fc0 2023-12-14T18:27:15.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.195+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3efc060f30 con 0x7f3ee80a11e0 2023-12-14T18:27:15.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.209+0000 7f3eef7fe700 1 -- 172.21.15.31:0/2472421037 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f3efc060f30 con 0x7f3ee80a11e0 2023-12-14T18:27:15.210 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-14T18:27:15.210 INFO:teuthology.orchestra.run.smithi031.stdout:mds.a smithi031 running (87s) 14s ago 87s 14.2M - 18.0.0-3905-g1faa3533 0caf899250bd a4aeec6b69d1 2023-12-14T18:27:15.210 INFO:teuthology.orchestra.run.smithi031.stdout:mds.b smithi176 running (86s) 20s ago 85s 21.1M - 18.0.0-3905-g1faa3533 0caf899250bd d46cb1843272 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.c smithi204 running (95s) 20s ago 94s 18.3M - 18.0.0-3905-g1faa3533 0caf899250bd 314c41ce30d1 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.d smithi031 running (100s) 14s ago 100s 17.3M - 18.0.0-3905-g1faa3533 0caf899250bd 3823d38038dc 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.e smithi176 running (98s) 20s ago 98s 14.9M - 18.0.0-3905-g1faa3533 0caf899250bd bba9e0b45c36 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.f smithi204 running (97s) 20s ago 96s 20.8M - 18.0.0-3905-g1faa3533 0caf899250bd 59206bc6f397 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.g smithi031 running (89s) 14s ago 89s 15.1M - 18.0.0-3905-g1faa3533 0caf899250bd 1f7b22ca15db 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.h smithi176 running (93s) 20s ago 93s 18.6M - 18.0.0-3905-g1faa3533 0caf899250bd 7836a5e6387e 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.i smithi204 running (91s) 20s ago 91s 30.6M - 18.0.0-3905-g1faa3533 0caf899250bd 14669656c1c1 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.j smithi031 running (102s) 14s ago 102s 17.3M - 18.0.0-3905-g1faa3533 0caf899250bd 444fb1c55d97 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.k smithi176 running (104s) 20s ago 103s 14.0M - 18.0.0-3905-g1faa3533 0caf899250bd 352dcf74b7d7 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mds.l smithi204 running (105s) 20s ago 105s 17.5M - 18.0.0-3905-g1faa3533 0caf899250bd 545ab783cb56 2023-12-14T18:27:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.x smithi031 *:9283,8765 running (7m) 14s ago 7m 515M - 18.0.0-3905-g1faa3533 0caf899250bd cddee8c58820 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.y smithi176 *:8443,8765 running (5m) 20s ago 5m 444M - 18.0.0-3905-g1faa3533 0caf899250bd 39a0076bb178 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.z smithi204 *:8443,8765 running (5m) 20s ago 5m 446M - 18.0.0-3905-g1faa3533 0caf899250bd 98094b690650 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a smithi031 running (7m) 14s ago 7m 97.9M 2048M 18.0.0-3905-g1faa3533 0caf899250bd df226c72c46f 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b smithi176 running (5m) 20s ago 5m 64.2M 2048M 18.0.0-3905-g1faa3533 0caf899250bd a6c8807c56e1 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c smithi204 running (5m) 20s ago 5m 64.5M 2048M 18.0.0-3905-g1faa3533 0caf899250bd 47946612c9cb 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 14s ago 5m 70.1M 4279M 18.0.0-3905-g1faa3533 0caf899250bd 40f28444ce6e 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi176 running (5m) 20s ago 5m 78.1M 4278M 18.0.0-3905-g1faa3533 0caf899250bd d13459c2bf88 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi204 running (4m) 20s ago 4m 73.1M 4278M 18.0.0-3905-g1faa3533 0caf899250bd b55cfbdedf50 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi031 running (4m) 14s ago 4m 67.1M 4279M 18.0.0-3905-g1faa3533 0caf899250bd 2c2c0b685855 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi176 running (4m) 20s ago 4m 61.9M 4278M 18.0.0-3905-g1faa3533 0caf899250bd c61d4c92a3b1 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi204 running (3m) 20s ago 3m 67.6M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 97e4b585e504 2023-12-14T18:27:15.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 smithi031 running (3m) 14s ago 3m 60.0M 4279M 18.0.0-3905-g1faa3533 0caf899250bd a4d22740432d 2023-12-14T18:27:15.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 smithi176 running (3m) 20s ago 3m 64.4M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 9f4496708432 2023-12-14T18:27:15.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 smithi204 running (2m) 20s ago 2m 60.2M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 495f6107726d 2023-12-14T18:27:15.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 smithi031 running (2m) 14s ago 2m 58.3M 4279M 18.0.0-3905-g1faa3533 0caf899250bd 9cd596e16f57 2023-12-14T18:27:15.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 smithi176 running (2m) 20s ago 2m 63.3M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 3d9a867f2d0c 2023-12-14T18:27:15.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 smithi204 running (2m) 20s ago 2m 57.4M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 2150253145ce 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.211+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 msgr2=0x7f3ee80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.211+0000 7f3f04560700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000b680 tx=0x7f3ef0005fd0 comp rx=0 tx=0).stop 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.211+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 msgr2=0x7f3efc196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.211+0000 7f3f04560700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 0x7f3efc196220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3ef800d9b0 tx=0x7f3ef800dcc0 comp rx=0 tx=0).stop 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 shutdown_connections 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f3ee80a11e0 0x7f3ee80a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3efc106200 0x7f3efc19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3efc100fc0 0x7f3efc196220 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 --2- 172.21.15.31:0/2472421037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3efc1038e0 0x7f3efc196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:15.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 >> 172.21.15.31:0/2472421037 conn(0x7f3efc0faaa0 msgr2=0x7f3efc101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:15.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 shutdown_connections 2023-12-14T18:27:15.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:15.212+0000 7f3f04560700 1 -- 172.21.15.31:0/2472421037 wait complete. 2023-12-14T18:27:15.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-14T18:27:15.911 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:16.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:16 smithi031 ceph-mon[42085]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:16.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:16 smithi031 ceph-mon[42085]: pgmap v271: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:16 smithi204 ceph-mon[43866]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:16 smithi204 ceph-mon[43866]: pgmap v271: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:16 smithi176 ceph-mon[43506]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:16 smithi176 ceph-mon[43506]: pgmap v271: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 -- 172.21.15.31:0/331068979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c741036f0 msgr2=0x7f7c74105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 --2- 172.21.15.31:0/331068979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c741036f0 0x7f7c74105ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009990 tx=0x7f7c64009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 -- 172.21.15.31:0/331068979 shutdown_connections 2023-12-14T18:27:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 --2- 172.21.15.31:0/331068979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c74106010 0x7f7c74108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 --2- 172.21.15.31:0/331068979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c74100dd0 0x7f7c741031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 --2- 172.21.15.31:0/331068979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c741036f0 0x7f7c74105ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 -- 172.21.15.31:0/331068979 >> 172.21.15.31:0/331068979 conn(0x7f7c740fa7f0 msgr2=0x7f7c740fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:17.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 -- 172.21.15.31:0/331068979 shutdown_connections 2023-12-14T18:27:17.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 -- 172.21.15.31:0/331068979 wait complete. 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.016+0000 7f7c7c747700 1 Processor -- start 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 -- start start 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 0x7f7c74196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c741036f0 0x7f7c741965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c74106010 0x7f7c7419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419dbf0 con 0x7f7c74100dd0 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419dd30 con 0x7f7c74106010 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7c747700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7419dec0 con 0x7f7c741036f0 2023-12-14T18:27:17.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 0x7f7c74196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 0x7f7c74196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40266/0 (socket says 172.21.15.31:40266) 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 -- 172.21.15.31:0/4225391362 learned_addr learned my addr 172.21.15.31:0/4225391362 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c79ce2700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c741036f0 0x7f7c741965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7ace4700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c74106010 0x7f7c7419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 -- 172.21.15.31:0/4225391362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c741036f0 msgr2=0x7f7c741965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c741036f0 0x7f7c741965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 -- 172.21.15.31:0/4225391362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c74106010 msgr2=0x7f7c7419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c74106010 0x7f7c7419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 -- 172.21.15.31:0/4225391362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640095f0 con 0x7f7c74100dd0 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c79ce2700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c741036f0 0x7f7c741965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.017+0000 7f7c7a4e3700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 0x7f7c74196060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000d9b0 tx=0x7f7c7000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.018+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000bb60 con 0x7f7c74100dd0 2023-12-14T18:27:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.018+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7410bc80 con 0x7f7c74100dd0 2023-12-14T18:27:17.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.018+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c70007e80 con 0x7f7c74100dd0 2023-12-14T18:27:17.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.018+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004dd0 con 0x7f7c74100dd0 2023-12-14T18:27:17.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.018+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7410baf0 con 0x7f7c74100dd0 2023-12-14T18:27:17.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.019+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c740fc360 con 0x7f7c74100dd0 2023-12-14T18:27:17.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.020+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7c70004020 con 0x7f7c74100dd0 2023-12-14T18:27:17.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.021+0000 7f7c6b7fe700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7c600a11e0 0x7f7c600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:17.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.021+0000 7f7c79ce2700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7c600a11e0 0x7f7c600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:17.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.021+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c700cd290 con 0x7f7c74100dd0 2023-12-14T18:27:17.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.021+0000 7f7c79ce2700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7c600a11e0 0x7f7c600a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400b2e0 tx=0x7f7c6400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:17.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.024+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7c70096570 con 0x7f7c74100dd0 2023-12-14T18:27:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.242+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c74061aa0 con 0x7f7c600a11e0 2023-12-14T18:27:17.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.248+0000 7f7c6b7fe700 1 -- 172.21.15.31:0/4225391362 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f7c74061aa0 con 0x7f7c600a11e0 2023-12-14T18:27:17.249 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-14T18:27:17.249 INFO:teuthology.orchestra.run.smithi031.stdout:mds.all 12/12 22s ago 109s smithi031=a;smithi031=d;smithi031=g;smithi031=j;smithi176=b;smithi176=e;smithi176=h;smithi176=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2023-12-14T18:27:17.249 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 3/3 22s ago 5m smithi031=x;smithi176=y;smithi204=z;count:3 2023-12-14T18:27:17.249 INFO:teuthology.orchestra.run.smithi031.stdout:mon 3/3 22s ago 6m smithi031:172.21.15.31=a;smithi176:172.21.15.176=b;smithi204:172.21.15.204=c;count:3 2023-12-14T18:27:17.250 INFO:teuthology.orchestra.run.smithi031.stdout:osd 12 22s ago - 2023-12-14T18:27:17.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7c600a11e0 msgr2=0x7f7c600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7c600a11e0 0x7f7c600a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400b2e0 tx=0x7f7c6400b610 comp rx=0 tx=0).stop 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 msgr2=0x7f7c74196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 0x7f7c74196060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000d9b0 tx=0x7f7c7000dd70 comp rx=0 tx=0).stop 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 shutdown_connections 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7c74106010 0x7f7c7419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c74100dd0 0x7f7c74196060 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7c600a11e0 0x7f7c600a3690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 --2- 172.21.15.31:0/4225391362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c741036f0 0x7f7c741965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 >> 172.21.15.31:0/4225391362 conn(0x7f7c740fa7f0 msgr2=0x7f7c74101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 shutdown_connections 2023-12-14T18:27:17.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:17.250+0000 7f7c7c747700 1 -- 172.21.15.31:0/4225391362 wait complete. 2023-12-14T18:27:17.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-14T18:27:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:18.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:18 smithi031 ceph-mon[42085]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:18.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:18 smithi031 ceph-mon[42085]: pgmap v272: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:18 smithi204 ceph-mon[43866]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:18.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:18 smithi204 ceph-mon[43866]: pgmap v272: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:18 smithi176 ceph-mon[43506]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:18 smithi176 ceph-mon[43506]: pgmap v272: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 -- 172.21.15.31:0/1564026593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe24100ba0 msgr2=0x7fbe24107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 --2- 172.21.15.31:0/1564026593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe24100ba0 0x7fbe24107460 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b0a0 tx=0x7fbe1c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 -- 172.21.15.31:0/1564026593 shutdown_connections 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 --2- 172.21.15.31:0/1564026593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe24100ba0 0x7fbe24107460 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 --2- 172.21.15.31:0/1564026593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe240ff010 0x7fbe240ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 --2- 172.21.15.31:0/1564026593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe24100210 0x7fbe24100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 -- 172.21.15.31:0/1564026593 >> 172.21.15.31:0/1564026593 conn(0x7fbe240fa620 msgr2=0x7fbe240fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 -- 172.21.15.31:0/1564026593 shutdown_connections 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.042+0000 7fbe29da2700 1 -- 172.21.15.31:0/1564026593 wait complete. 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 Processor -- start 2023-12-14T18:27:19.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 -- start start 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 0x7fbe24195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe24100210 0x7fbe24196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe24100ba0 0x7fbe2419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2419db20 con 0x7fbe240ff010 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2419dc60 con 0x7fbe24100210 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe29da2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2419ddf0 con 0x7fbe24100ba0 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe28da0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 0x7fbe24195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe28da0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 0x7fbe24195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40298/0 (socket says 172.21.15.31:40298) 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe28da0700 1 -- 172.21.15.31:0/2559227491 learned_addr learned my addr 172.21.15.31:0/2559227491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe295a1700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe24100ba0 0x7fbe2419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.043+0000 7fbe23fff700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe24100210 0x7fbe24196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:19.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe28da0700 1 -- 172.21.15.31:0/2559227491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe24100ba0 msgr2=0x7fbe2419d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe28da0700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe24100ba0 0x7fbe2419d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe28da0700 1 -- 172.21.15.31:0/2559227491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe24100210 msgr2=0x7fbe24196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe28da0700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe24100210 0x7fbe24196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe28da0700 1 -- 172.21.15.31:0/2559227491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c009d00 con 0x7fbe240ff010 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe28da0700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 0x7fbe24195fe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d9b0 tx=0x7fbe1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800b950 con 0x7fbe240ff010 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2410a960 con 0x7fbe240ff010 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe18007620 con 0x7fbe240ff010 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800f3c0 con 0x7fbe240ff010 2023-12-14T18:27:19.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.044+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2410a7d0 con 0x7fbe240ff010 2023-12-14T18:27:19.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.045+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2404e8c0 con 0x7fbe240ff010 2023-12-14T18:27:19.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.046+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbe18010640 con 0x7fbe240ff010 2023-12-14T18:27:19.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.047+0000 7fbe21ffb700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe140a11e0 0x7fbe140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:19.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.047+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbe180ccf70 con 0x7fbe240ff010 2023-12-14T18:27:19.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.047+0000 7fbe23fff700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe140a11e0 0x7fbe140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:19.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.048+0000 7fbe23fff700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe140a11e0 0x7fbe140a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005950 tx=0x7fbe100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:19.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.052+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbe18096250 con 0x7fbe240ff010 2023-12-14T18:27:19.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.268+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe240695d0 con 0x7fbe140a11e0 2023-12-14T18:27:19.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.270+0000 7fbe21ffb700 1 -- 172.21.15.31:0/2559227491 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fbe240695d0 con 0x7fbe140a11e0 2023-12-14T18:27:19.271 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2023-12-14T18:27:19.271 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2023-12-14T18:27:19.271 INFO:teuthology.orchestra.run.smithi031.stdout:smithi176 172.21.15.176 2023-12-14T18:27:19.271 INFO:teuthology.orchestra.run.smithi031.stdout:smithi204 172.21.15.204 2023-12-14T18:27:19.272 INFO:teuthology.orchestra.run.smithi031.stdout:3 hosts in cluster 2023-12-14T18:27:19.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe140a11e0 msgr2=0x7fbe140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:19.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe140a11e0 0x7fbe140a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe10005950 tx=0x7fbe100058e0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 msgr2=0x7fbe24195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 0x7fbe24195fe0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800d9b0 tx=0x7fbe1800dcc0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 shutdown_connections 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fbe140a11e0 0x7fbe140a3690 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe24100ba0 0x7fbe2419d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe240ff010 0x7fbe24195fe0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 --2- 172.21.15.31:0/2559227491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe24100210 0x7fbe24196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 >> 172.21.15.31:0/2559227491 conn(0x7fbe240fa620 msgr2=0x7fbe24068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 shutdown_connections 2023-12-14T18:27:19.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:19.272+0000 7fbe29da2700 1 -- 172.21.15.31:0/2559227491 wait complete. 2023-12-14T18:27:19.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-14T18:27:20.018 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:20.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:20 smithi204 ceph-mon[43866]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:20.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:20 smithi204 ceph-mon[43866]: pgmap v273: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:20.864 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:20 smithi031 ceph-mon[42085]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:20.864 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:20 smithi031 ceph-mon[42085]: pgmap v273: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:20 smithi176 ceph-mon[43506]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:20 smithi176 ceph-mon[43506]: pgmap v273: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.109+0000 7ff829254700 1 -- 172.21.15.31:0/1455122371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 msgr2=0x7ff824100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.109+0000 7ff829254700 1 --2- 172.21.15.31:0/1455122371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff824100830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff818009990 tx=0x7ff818009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 -- 172.21.15.31:0/1455122371 shutdown_connections 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 --2- 172.21.15.31:0/1455122371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff824100d70 0x7ff824107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 --2- 172.21.15.31:0/1455122371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8240ff1e0 0x7ff8240ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 --2- 172.21.15.31:0/1455122371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff824100830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 -- 172.21.15.31:0/1455122371 >> 172.21.15.31:0/1455122371 conn(0x7ff8240fa7f0 msgr2=0x7ff8240fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 -- 172.21.15.31:0/1455122371 shutdown_connections 2023-12-14T18:27:21.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 -- 172.21.15.31:0/1455122371 wait complete. 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.110+0000 7ff829254700 1 Processor -- start 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 -- start start 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8240ff1e0 0x7ff82419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff8241a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff824100d70 0x7ff8241a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a41b0 con 0x7ff8241003e0 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a4320 con 0x7ff8240ff1e0 2023-12-14T18:27:21.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff829254700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a44e0 con 0x7ff824100d70 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff8241a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff8241a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40322/0 (socket says 172.21.15.31:40322) 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 -- 172.21.15.31:0/4233371715 learned_addr learned my addr 172.21.15.31:0/4233371715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff822d9d700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8240ff1e0 0x7ff82419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82359e700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff824100d70 0x7ff8241a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 -- 172.21.15.31:0/4233371715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff824100d70 msgr2=0x7ff8241a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff824100d70 0x7ff8241a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 -- 172.21.15.31:0/4233371715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8240ff1e0 msgr2=0x7ff82419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8240ff1e0 0x7ff82419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.111+0000 7ff82259c700 1 -- 172.21.15.31:0/4233371715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8180095f0 con 0x7ff8241003e0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.112+0000 7ff82259c700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff8241a1450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff818000c00 tx=0x7ff818009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.112+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81801d070 con 0x7ff8241003e0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.112+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82410ab40 con 0x7ff8241003e0 2023-12-14T18:27:21.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.112+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff818022470 con 0x7ff8241003e0 2023-12-14T18:27:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.112+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff818017650 con 0x7ff8241003e0 2023-12-14T18:27:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.112+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82410a9b0 con 0x7ff8241003e0 2023-12-14T18:27:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.113+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8241a4a80 con 0x7ff8241003e0 2023-12-14T18:27:21.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.114+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff818022850 con 0x7ff8241003e0 2023-12-14T18:27:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.115+0000 7ff80bfff700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff80c0a11e0 0x7ff80c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.115+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff8180ce100 con 0x7ff8241003e0 2023-12-14T18:27:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.115+0000 7ff822d9d700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff80c0a11e0 0x7ff80c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:21.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.115+0000 7ff822d9d700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff80c0a11e0 0x7ff80c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff8241079e0 tx=0x7ff814009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:21.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.119+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff818097580 con 0x7ff8241003e0 2023-12-14T18:27:21.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.340+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff824068d40 con 0x7ff80c0a11e0 2023-12-14T18:27:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.342+0000 7ff80bfff700 1 -- 172.21.15.31:0/4233371715 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7ff824068d40 con 0x7ff80c0a11e0 2023-12-14T18:27:21.343 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-14T18:27:21.343 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 2m ago LVM detected, locked 2023-12-14T18:27:21.343 INFO:teuthology.orchestra.run.smithi031.stdout:smithi176 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330099400BGN 372G No 2m ago LVM detected, locked 2023-12-14T18:27:21.343 INFO:teuthology.orchestra.run.smithi031.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago LVM detected, locked 2023-12-14T18:27:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff80c0a11e0 msgr2=0x7ff80c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff80c0a11e0 0x7ff80c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff8241079e0 tx=0x7ff814009260 comp rx=0 tx=0).stop 2023-12-14T18:27:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 msgr2=0x7ff8241a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff8241a1450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff818000c00 tx=0x7ff818009580 comp rx=0 tx=0).stop 2023-12-14T18:27:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 shutdown_connections 2023-12-14T18:27:21.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff80c0a11e0 0x7ff80c0a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff824100d70 0x7ff8241a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff8240ff1e0 0x7ff82419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 --2- 172.21.15.31:0/4233371715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8241003e0 0x7ff8241a1450 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 >> 172.21.15.31:0/4233371715 conn(0x7ff8240fa7f0 msgr2=0x7ff824068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 shutdown_connections 2023-12-14T18:27:21.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:21.344+0000 7ff829254700 1 -- 172.21.15.31:0/4233371715 wait complete. 2023-12-14T18:27:21.732 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-14T18:27:21.740 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:27:21.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-14T18:27:22.101 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:22 smithi204 ceph-mon[43866]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:22.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:22 smithi204 ceph-mon[43866]: pgmap v274: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:22.939 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:22 smithi031 ceph-mon[42085]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:22.940 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:22 smithi031 ceph-mon[42085]: pgmap v274: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:22 smithi176 ceph-mon[43506]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:22.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:22 smithi176 ceph-mon[43506]: pgmap v274: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 -- 172.21.15.31:0/3097543549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c1003e0 msgr2=0x7fb45c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 --2- 172.21.15.31:0/3097543549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c1003e0 0x7fb45c100830 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009990 tx=0x7fb44c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 -- 172.21.15.31:0/3097543549 shutdown_connections 2023-12-14T18:27:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 --2- 172.21.15.31:0/3097543549 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb45c100d70 0x7fb45c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 --2- 172.21.15.31:0/3097543549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb45c0ff1e0 0x7fb45c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 --2- 172.21.15.31:0/3097543549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c1003e0 0x7fb45c100830 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 -- 172.21.15.31:0/3097543549 >> 172.21.15.31:0/3097543549 conn(0x7fb45c0fa7f0 msgr2=0x7fb45c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 -- 172.21.15.31:0/3097543549 shutdown_connections 2023-12-14T18:27:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.128+0000 7fb46300a700 1 -- 172.21.15.31:0/3097543549 wait complete. 2023-12-14T18:27:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 Processor -- start 2023-12-14T18:27:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 -- start start 2023-12-14T18:27:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb45c0ff1e0 0x7fb45c195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb45c1003e0 0x7fb45c196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 0x7fb45c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19db60 con 0x7fb45c100d70 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19dca0 con 0x7fb45c0ff1e0 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb46300a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c19de30 con 0x7fb45c1003e0 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb4615a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 0x7fb45c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb453fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb45c1003e0 0x7fb45c196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb4615a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 0x7fb45c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40338/0 (socket says 172.21.15.31:40338) 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb4615a7700 1 -- 172.21.15.31:0/45552136 learned_addr learned my addr 172.21.15.31:0/45552136 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.129+0000 7fb460da6700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb45c0ff1e0 0x7fb45c195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb4615a7700 1 -- 172.21.15.31:0/45552136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb45c1003e0 msgr2=0x7fb45c196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:23.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb4615a7700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb45c1003e0 0x7fb45c196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb4615a7700 1 -- 172.21.15.31:0/45552136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb45c0ff1e0 msgr2=0x7fb45c195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb4615a7700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb45c0ff1e0 0x7fb45c195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb4615a7700 1 -- 172.21.15.31:0/45552136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c0095f0 con 0x7fb45c100d70 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb4615a7700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 0x7fb45c19d590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb454007f10 tx=0x7fb45400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454017070 con 0x7fb45c100d70 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb45400f040 con 0x7fb45c100d70 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c10ab40 con 0x7fb45c100d70 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454005610 con 0x7fb45c100d70 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.130+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c10a9b0 con 0x7fb45c100d70 2023-12-14T18:27:23.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.131+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c19e510 con 0x7fb45c100d70 2023-12-14T18:27:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.132+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb454004ad0 con 0x7fb45c100d70 2023-12-14T18:27:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.133+0000 7fb451ffb700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb4440a1260 0x7fb4440a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.133+0000 7fb460da6700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb4440a1260 0x7fb4440a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:23.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.133+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb4540cc510 con 0x7fb45c100d70 2023-12-14T18:27:23.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.134+0000 7fb460da6700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb4440a1260 0x7fb4440a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb45c100240 tx=0x7fb45800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:23.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.137+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb454095790 con 0x7fb45c100d70 2023-12-14T18:27:23.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.397+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb45c19eac0 con 0x7fb45c100d70 2023-12-14T18:27:23.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.398+0000 7fb451ffb700 1 -- 172.21.15.31:0/45552136 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2904 (secure 0 0 0) 0x7fb454098db0 con 0x7fb45c100d70 2023-12-14T18:27:23.399 INFO:teuthology.orchestra.run.smithi031.stdout:e24 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 24 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-12-14T18:25:55.286836+0000 2023-12-14T18:27:23.400 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2023-12-14T18:26:22.578285+0000 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 300 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2023-12-14T18:27:23.401 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 3 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1,2 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24538,1=24536,2=24514} 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3,4] 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2023-12-14T18:27:23.402 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.i{0:24538} state up:active seq 4 addr [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f{0:24502} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/3595337412,v1:172.21.15.204:6835/3595337412] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b{1:24536} state up:active seq 9 addr [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.h{1:24497} state up:standby-replay seq 1 addr [v2:172.21.15.176:6836/2569700606,v1:172.21.15.176:6837/2569700606] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c{2:24514} state up:active seq 12 addr [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d{2:24490} state up:standby-replay seq 1 addr [v2:172.21.15.31:6836/3783760943,v1:172.21.15.31:6837/3783760943] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2023-12-14T18:27:23.403 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.31:6834/28036797,v1:172.21.15.31:6835/28036797] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.g{-1:14589} state up:standby seq 1 addr [v2:172.21.15.31:6838/2398776190,v1:172.21.15.31:6839/2398776190] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a{-1:14601} state up:standby seq 1 addr [v2:172.21.15.31:6840/3456971189,v1:172.21.15.31:6841/3456971189] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.176:6832/2002803706,v1:172.21.15.176:6833/2002803706] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.l{-1:24460} state up:standby seq 1 addr [v2:172.21.15.204:6832/1411039043,v1:172.21.15.204:6833/1411039043] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.404 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e{-1:24476} state up:standby seq 1 addr [v2:172.21.15.176:6834/1697669779,v1:172.21.15.176:6835/1697669779] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:27:23.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb4440a1260 msgr2=0x7fb4440a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:23.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb4440a1260 0x7fb4440a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb45c100240 tx=0x7fb45800b410 comp rx=0 tx=0).stop 2023-12-14T18:27:23.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 msgr2=0x7fb45c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:23.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 0x7fb45c19d590 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb454007f10 tx=0x7fb45400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 shutdown_connections 2023-12-14T18:27:23.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb45c100d70 0x7fb45c19d590 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb45c0ff1e0 0x7fb45c195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fb4440a1260 0x7fb4440a3710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 --2- 172.21.15.31:0/45552136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb45c1003e0 0x7fb45c196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.400+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 >> 172.21.15.31:0/45552136 conn(0x7fb45c0fa7f0 msgr2=0x7fb45c068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.401+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 shutdown_connections 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:23.401+0000 7fb46300a700 1 -- 172.21.15.31:0/45552136 wait complete. 2023-12-14T18:27:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:23.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-14T18:27:24.141 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:24 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/45552136' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-14T18:27:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:24 smithi204 ceph-mon[43866]: pgmap v275: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:24.644 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:24 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/45552136' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-14T18:27:24.645 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:24 smithi031 ceph-mon[42085]: pgmap v275: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:24.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:24 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/45552136' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-14T18:27:24.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:24 smithi176 ceph-mon[43506]: pgmap v275: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 -- 172.21.15.31:0/2858884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 msgr2=0x7fd920108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 --2- 172.21.15.31:0/2858884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd920108750 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd91800b0a0 tx=0x7fd91800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 -- 172.21.15.31:0/2858884811 shutdown_connections 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 --2- 172.21.15.31:0/2858884811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd920108750 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 --2- 172.21.15.31:0/2858884811 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd920100300 0x7fd920100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 --2- 172.21.15.31:0/2858884811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd920101500 0x7fd920101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 -- 172.21.15.31:0/2858884811 >> 172.21.15.31:0/2858884811 conn(0x7fd9200fb910 msgr2=0x7fd9200fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 -- 172.21.15.31:0/2858884811 shutdown_connections 2023-12-14T18:27:25.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.269+0000 7fd928269700 1 -- 172.21.15.31:0/2858884811 wait complete. 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 Processor -- start 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 -- start start 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd920100300 0x7fd920196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd920101500 0x7fd920196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd92019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92019dcd0 con 0x7fd920101e90 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92019de10 con 0x7fd920101500 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd928269700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92019dfa0 con 0x7fd920100300 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd926806700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd92019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd926806700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd92019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59060/0 (socket says 172.21.15.31:59060) 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd926806700 1 -- 172.21.15.31:0/3054759267 learned_addr learned my addr 172.21.15.31:0/3054759267 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd926005700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd920100300 0x7fd920196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.270+0000 7fd925804700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd920101500 0x7fd920196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd926806700 1 -- 172.21.15.31:0/3054759267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd920100300 msgr2=0x7fd920196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd926806700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd920100300 0x7fd920196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd926806700 1 -- 172.21.15.31:0/3054759267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd920101500 msgr2=0x7fd920196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd926806700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd920101500 0x7fd920196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd926806700 1 -- 172.21.15.31:0/3054759267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918009d00 con 0x7fd920101e90 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd926806700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd92019d700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd918007a60 tx=0x7fd91800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800e040 con 0x7fd920101e90 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92010bc80 con 0x7fd920101e90 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9180094a0 con 0x7fd920101e90 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180053b0 con 0x7fd920101e90 2023-12-14T18:27:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.271+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92010baf0 con 0x7fd920101e90 2023-12-14T18:27:25.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.273+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd920066ea0 con 0x7fd920101e90 2023-12-14T18:27:25.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.274+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd91801d9c0 con 0x7fd920101e90 2023-12-14T18:27:25.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.274+0000 7fd9177fe700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd90c0a1260 0x7fd90c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.274+0000 7fd926005700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd90c0a1260 0x7fd90c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.275+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd9180ce470 con 0x7fd920101e90 2023-12-14T18:27:25.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.275+0000 7fd926005700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd90c0a1260 0x7fd90c0a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd920101360 tx=0x7fd91c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.279+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd9180977e0 con 0x7fd920101e90 2023-12-14T18:27:25.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.489+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fd9201071c0 con 0x7fd920101e90 2023-12-14T18:27:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.490+0000 7fd9177fe700 1 -- 172.21.15.31:0/3054759267 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5609 (secure 0 0 0) 0x7fd91809ae00 con 0x7fd920101e90 2023-12-14T18:27:25.491 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 87 2023-12-14T18:27:25.491 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 37e25396-9aad-11ee-95a5-87774f69a715 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-12-14T18:19:54.751625+0000 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2023-12-14T18:26:48.546060+0000 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 27 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client luminous 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release reef 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-12-14T18:27:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 12 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6802/4133067944,v1:172.21.15.31:6803/4133067944] [v2:172.21.15.31:6804/4133067944,v1:172.21.15.31:6805/4133067944] exists,up 60ee717b-c58b-4f49-b38f-1ba95d414f5b 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6800/3594574093,v1:172.21.15.176:6801/3594574093] [v2:172.21.15.176:6802/3594574093,v1:172.21.15.176:6803/3594574093] exists,up 79738a41-e749-4359-85db-3cc4494bf2f1 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/4107722229,v1:172.21.15.204:6801/4107722229] [v2:172.21.15.204:6802/4107722229,v1:172.21.15.204:6803/4107722229] exists,up 5119d0ba-0044-4206-b4e1-898df98d5f0c 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6810/3446506620,v1:172.21.15.31:6811/3446506620] [v2:172.21.15.31:6812/3446506620,v1:172.21.15.31:6813/3446506620] exists,up bfb73492-97c9-4615-9eed-75737af46d2e 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6808/2531574099,v1:172.21.15.176:6809/2531574099] [v2:172.21.15.176:6810/2531574099,v1:172.21.15.176:6811/2531574099] exists,up a1c7deda-98af-495d-9680-d1b95ceb100b 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/1402310939,v1:172.21.15.204:6809/1402310939] [v2:172.21.15.204:6810/1402310939,v1:172.21.15.204:6811/1402310939] exists,up e58c9b5f-bcfc-48af-a8bf-e1744c662b84 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 43 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6818/2676550981,v1:172.21.15.31:6819/2676550981] [v2:172.21.15.31:6820/2676550981,v1:172.21.15.31:6821/2676550981] exists,up 78200a1a-a3f7-48fb-aeb7-c335cc8b9e86 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 48 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6816/800429312,v1:172.21.15.176:6817/800429312] [v2:172.21.15.176:6818/800429312,v1:172.21.15.176:6819/800429312] exists,up 2d0a6115-2273-425b-84a6-086924f6da23 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.8 up in weight 1 up_from 55 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/2332045081,v1:172.21.15.204:6817/2332045081] [v2:172.21.15.204:6818/2332045081,v1:172.21.15.204:6819/2332045081] exists,up bb114398-c403-4e55-83ea-cdab039ce2e4 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.9 up in weight 1 up_from 63 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6826/1064380347,v1:172.21.15.31:6827/1064380347] [v2:172.21.15.31:6828/1064380347,v1:172.21.15.31:6829/1064380347] exists,up 4d9d2a00-4cc2-40e3-8587-405fa776464d 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.176:6824/3741647642,v1:172.21.15.176:6825/3741647642] [v2:172.21.15.176:6826/3741647642,v1:172.21.15.176:6827/3741647642] exists,up caf93ab4-d4cc-4d56-a291-abbc32df979b 2023-12-14T18:27:25.493 INFO:teuthology.orchestra.run.smithi031.stdout:osd.11 up in weight 1 up_from 76 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/2063130933,v1:172.21.15.204:6825/2063130933] [v2:172.21.15.204:6826/2063130933,v1:172.21.15.204:6827/2063130933] exists,up a3cf56d3-54b6-43d9-91cd-1b869bcf5146 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.6 [8,11] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.b [8,11] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.c [2,1] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 2.1b [2,4] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.11 [4,8] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 3.13 [4,9] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.13 [9,8] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.16 [9,10] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.1c [5,8] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:pg_upmap_items 4.1f [9,8] 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/1680573410 expires 2023-12-15T18:20:47.403710+0000 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/2330625991 expires 2023-12-15T18:20:47.403710+0000 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/2330625991 expires 2023-12-15T18:20:47.403710+0000 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/384230437 expires 2023-12-15T18:20:47.403710+0000 2023-12-14T18:27:25.494 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6800/2094677585 expires 2023-12-15T18:20:25.264468+0000 2023-12-14T18:27:25.495 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/132110935 expires 2023-12-15T18:20:25.264468+0000 2023-12-14T18:27:25.495 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:6801/2094677585 expires 2023-12-15T18:20:25.264468+0000 2023-12-14T18:27:25.495 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2892448986 expires 2023-12-15T18:20:25.264468+0000 2023-12-14T18:27:25.495 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/463659266 expires 2023-12-15T18:20:47.403710+0000 2023-12-14T18:27:25.495 INFO:teuthology.orchestra.run.smithi031.stdout:blocklist 172.21.15.31:0/2117792145 expires 2023-12-15T18:20:25.264468+0000 2023-12-14T18:27:25.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.492+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd90c0a1260 msgr2=0x7fd90c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.492+0000 7fd928269700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd90c0a1260 0x7fd90c0a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd920101360 tx=0x7fd91c009260 comp rx=0 tx=0).stop 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.492+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 msgr2=0x7fd92019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.492+0000 7fd928269700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd92019d700 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd918007a60 tx=0x7fd91800bd60 comp rx=0 tx=0).stop 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 shutdown_connections 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd920101e90 0x7fd92019d700 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd90c0a1260 0x7fd90c0a3710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd920100300 0x7fd920196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 --2- 172.21.15.31:0/3054759267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd920101500 0x7fd920196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 >> 172.21.15.31:0/3054759267 conn(0x7fd9200fb910 msgr2=0x7fd9200fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 shutdown_connections 2023-12-14T18:27:25.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:25.493+0000 7fd928269700 1 -- 172.21.15.31:0/3054759267 wait complete. 2023-12-14T18:27:25.806 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:25 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3054759267' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-14T18:27:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:25 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3054759267' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-14T18:27:25.882 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-14T18:27:25.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:25 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3054759267' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 -- 172.21.15.31:0/1155617201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d70101e40 msgr2=0x7f7d70108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 --2- 172.21.15.31:0/1155617201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d70101e40 0x7f7d70108700 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800b0a0 tx=0x7f7d6800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 -- 172.21.15.31:0/1155617201 shutdown_connections 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 --2- 172.21.15.31:0/1155617201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d70101e40 0x7f7d70108700 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 --2- 172.21.15.31:0/1155617201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d700fe4e0 0x7f7d700fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 --2- 172.21.15.31:0/1155617201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d701014b0 0x7f7d70101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 -- 172.21.15.31:0/1155617201 >> 172.21.15.31:0/1155617201 conn(0x7f7d700f9e80 msgr2=0x7f7d700fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 -- 172.21.15.31:0/1155617201 shutdown_connections 2023-12-14T18:27:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.049+0000 7f7d7791f700 1 -- 172.21.15.31:0/1155617201 wait complete. 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 Processor -- start 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 -- start start 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d700fe4e0 0x7f7d70100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 0x7f7d700ff460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d70101e40 0x7f7d700ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d70100140 con 0x7f7d701014b0 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701002b0 con 0x7f7d70101e40 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d7791f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701a0160 con 0x7f7d700fe4e0 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d74eba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 0x7f7d700ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d74eba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 0x7f7d700ff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59078/0 (socket says 172.21.15.31:59078) 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d74eba700 1 -- 172.21.15.31:0/4117269402 learned_addr learned my addr 172.21.15.31:0/4117269402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d75ebc700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d70101e40 0x7f7d700ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.050+0000 7f7d756bb700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d700fe4e0 0x7f7d70100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d74eba700 1 -- 172.21.15.31:0/4117269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d700fe4e0 msgr2=0x7f7d70100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d74eba700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d700fe4e0 0x7f7d70100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d74eba700 1 -- 172.21.15.31:0/4117269402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d70101e40 msgr2=0x7f7d700ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d74eba700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d70101e40 0x7f7d700ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d74eba700 1 -- 172.21.15.31:0/4117269402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68009d00 con 0x7f7d701014b0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d74eba700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 0x7f7d700ff460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000d950 tx=0x7f7d6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000f890 con 0x7f7d701014b0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d60010460 con 0x7f7d701014b0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7010bbe0 con 0x7f7d701014b0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000e610 con 0x7f7d701014b0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.051+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7010ba50 con 0x7f7d701014b0 2023-12-14T18:27:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.052+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d701071a0 con 0x7f7d701014b0 2023-12-14T18:27:26.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.054+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7d600108d0 con 0x7f7d701014b0 2023-12-14T18:27:26.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.054+0000 7f7d667fc700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7d5c0a11e0 0x7f7d5c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.055+0000 7f7d756bb700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7d5c0a11e0 0x7f7d5c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.055+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7d600ccbd0 con 0x7f7d701014b0 2023-12-14T18:27:26.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.055+0000 7f7d756bb700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7d5c0a11e0 0x7f7d5c0a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d70067bd0 tx=0x7f7d6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.058+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7d60095e50 con 0x7f7d701014b0 2023-12-14T18:27:26.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.269+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d7002cfe0 con 0x7f7d701014b0 2023-12-14T18:27:26.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.271+0000 7f7d667fc700 1 -- 172.21.15.31:0/4117269402 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f7d60016090 con 0x7f7d701014b0 2023-12-14T18:27:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:26.272 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7d5c0a11e0 msgr2=0x7f7d5c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7d5c0a11e0 0x7f7d5c0a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d70067bd0 tx=0x7f7d6c009260 comp rx=0 tx=0).stop 2023-12-14T18:27:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 msgr2=0x7f7d700ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 0x7f7d700ff460 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000d950 tx=0x7f7d6000dc60 comp rx=0 tx=0).stop 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 shutdown_connections 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d70101e40 0x7f7d700ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7d5c0a11e0 0x7f7d5c0a3690 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d700fe4e0 0x7f7d70100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 --2- 172.21.15.31:0/4117269402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d701014b0 0x7f7d700ff460 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 >> 172.21.15.31:0/4117269402 conn(0x7f7d700f9e80 msgr2=0x7f7d700fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 shutdown_connections 2023-12-14T18:27:26.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.273+0000 7f7d7791f700 1 -- 172.21.15.31:0/4117269402 wait complete. 2023-12-14T18:27:26.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:26.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 -- 172.21.15.31:0/3499714302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88100ca0 msgr2=0x7f0d88103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3499714302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88100ca0 0x7f0d88103080 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009990 tx=0x7f0d78009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 -- 172.21.15.31:0/3499714302 shutdown_connections 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3499714302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d88105ee0 0x7f0d88108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3499714302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88100ca0 0x7f0d88103080 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3499714302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d881035c0 0x7f0d881059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 -- 172.21.15.31:0/3499714302 >> 172.21.15.31:0/3499714302 conn(0x7f0d880fa740 msgr2=0x7f0d880fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.428+0000 7f0d8c917700 1 -- 172.21.15.31:0/3499714302 shutdown_connections 2023-12-14T18:27:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.429+0000 7f0d8c917700 1 -- 172.21.15.31:0/3499714302 wait complete. 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.429+0000 7f0d8c917700 1 Processor -- start 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 -- start start 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d88100ca0 0x7f0d88195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881035c0 0x7f0d88196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 0x7f0d8819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819db40 con 0x7f0d88105ee0 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819dc80 con 0x7f0d88100ca0 2023-12-14T18:27:26.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8c917700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819de10 con 0x7f0d881035c0 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 0x7f0d8819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 0x7f0d8819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59106/0 (socket says 172.21.15.31:59106) 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 -- 172.21.15.31:0/3143998887 learned_addr learned my addr 172.21.15.31:0/3143998887 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8659c700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d88100ca0 0x7f0d88195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d7dd9b700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881035c0 0x7f0d88196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 -- 172.21.15.31:0/3143998887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881035c0 msgr2=0x7f0d88196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881035c0 0x7f0d88196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 -- 172.21.15.31:0/3143998887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d88100ca0 msgr2=0x7f0d88195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d88100ca0 0x7f0d88195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d86d9d700 1 -- 172.21.15.31:0/3143998887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780095f0 con 0x7f0d88105ee0 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.430+0000 7f0d8659c700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d88100ca0 0x7f0d88195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.431+0000 7f0d86d9d700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 0x7f0d8819d570 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d80007ee0 tx=0x7f0d8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:26.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.431+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000dd20 con 0x7f0d88105ee0 2023-12-14T18:27:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.431+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d8000f040 con 0x7f0d88105ee0 2023-12-14T18:27:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.431+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8810bb80 con 0x7f0d88105ee0 2023-12-14T18:27:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.431+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800048a0 con 0x7f0d88105ee0 2023-12-14T18:27:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.431+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8810b9f0 con 0x7f0d88105ee0 2023-12-14T18:27:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.432+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d880fc2b0 con 0x7f0d88105ee0 2023-12-14T18:27:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.434+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0d80003a00 con 0x7f0d88105ee0 2023-12-14T18:27:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.434+0000 7f0d7ffff700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0d740a11e0 0x7f0d740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.435+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0d800ccea0 con 0x7f0d88105ee0 2023-12-14T18:27:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.435+0000 7f0d8659c700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0d740a11e0 0x7f0d740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.435+0000 7f0d8659c700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0d740a11e0 0x7f0d740a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b2e0 tx=0x7f0d78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:26.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.438+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0d80096120 con 0x7f0d88105ee0 2023-12-14T18:27:26.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.647+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d880679f0 con 0x7f0d88105ee0 2023-12-14T18:27:26.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.647+0000 7f0d7ffff700 1 -- 172.21.15.31:0/3143998887 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d800230a0 con 0x7f0d88105ee0 2023-12-14T18:27:26.649 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0d740a11e0 msgr2=0x7f0d740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0d740a11e0 0x7f0d740a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800b2e0 tx=0x7f0d78005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 msgr2=0x7f0d8819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 0x7f0d8819d570 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d80007ee0 tx=0x7f0d8000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 shutdown_connections 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d88105ee0 0x7f0d8819d570 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d88100ca0 0x7f0d88195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f0d740a11e0 0x7f0d740a3690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 --2- 172.21.15.31:0/3143998887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881035c0 0x7f0d88196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 >> 172.21.15.31:0/3143998887 conn(0x7f0d880fa740 msgr2=0x7f0d88101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 shutdown_connections 2023-12-14T18:27:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.650+0000 7f0d8c917700 1 -- 172.21.15.31:0/3143998887 wait complete. 2023-12-14T18:27:26.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.804+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/2281116151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14100cd0 msgr2=0x7f9d141030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.804+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/2281116151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14100cd0 0x7f9d141030b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009990 tx=0x7f9d10009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.804+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/2281116151 shutdown_connections 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.804+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/2281116151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d14105f10 0x7f9d14108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/2281116151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14100cd0 0x7f9d141030b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/2281116151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d141035f0 0x7f9d141059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/2281116151 >> 172.21.15.31:0/2281116151 conn(0x7f9d140fa750 msgr2=0x7f9d140fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:26.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/2281116151 shutdown_connections 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/2281116151 wait complete. 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 Processor -- start 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 -- start start 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d14100cd0 0x7f9d14195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d141035f0 0x7f9d141964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 0x7f9d1419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1419db00 con 0x7f9d14105f10 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.805+0000 7f9d1bc1d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1419dc40 con 0x7f9d141035f0 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1bc1d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1419ddd0 con 0x7f9d14100cd0 2023-12-14T18:27:26.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 0x7f9d1419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 0x7f9d1419d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59114/0 (socket says 172.21.15.31:59114) 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 -- 172.21.15.31:0/3665791395 learned_addr learned my addr 172.21.15.31:0/3665791395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d199b9700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d14100cd0 0x7f9d14195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d191b8700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d141035f0 0x7f9d141964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 -- 172.21.15.31:0/3665791395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d14100cd0 msgr2=0x7f9d14195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d14100cd0 0x7f9d14195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 -- 172.21.15.31:0/3665791395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d141035f0 msgr2=0x7f9d141964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d141035f0 0x7f9d141964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 -- 172.21.15.31:0/3665791395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100095f0 con 0x7f9d14105f10 2023-12-14T18:27:26.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1a1ba700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 0x7f9d1419d530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c007ee0 tx=0x7f9d0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00ddd0 con 0x7f9d14105f10 2023-12-14T18:27:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d0c00f040 con 0x7f9d14105f10 2023-12-14T18:27:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1410bbb0 con 0x7f9d14105f10 2023-12-14T18:27:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0048e0 con 0x7f9d14105f10 2023-12-14T18:27:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.806+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1410ba20 con 0x7f9d14105f10 2023-12-14T18:27:26.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.808+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d140fc2c0 con 0x7f9d14105f10 2023-12-14T18:27:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.809+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9d0c003a00 con 0x7f9d14105f10 2023-12-14T18:27:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.809+0000 7f9d0affd700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d000a11e0 0x7f9d000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.809+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9d0c0cc700 con 0x7f9d14105f10 2023-12-14T18:27:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.809+0000 7f9d199b9700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d000a11e0 0x7f9d000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:26.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.810+0000 7f9d199b9700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d000a11e0 0x7f9d000a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000b5c0 tx=0x7f9d10005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:26.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:26.814+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d0c04e730 con 0x7f9d14105f10 2023-12-14T18:27:26.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:26 smithi204 ceph-mon[43866]: pgmap v276: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:26.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:26 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4117269402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:26.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:26 smithi031 ceph-mon[42085]: pgmap v276: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:26.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:26 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4117269402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:26.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:26 smithi176 ceph-mon[43506]: pgmap v276: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:26.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:26 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4117269402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:27.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.023+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d14061700 con 0x7f9d14105f10 2023-12-14T18:27:27.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.023+0000 7f9d0affd700 1 -- 172.21.15.31:0/3665791395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d0c0959e0 con 0x7f9d14105f10 2023-12-14T18:27:27.024 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:27.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.025+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d000a11e0 msgr2=0x7f9d000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.025+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d000a11e0 0x7f9d000a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000b5c0 tx=0x7f9d10005fd0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 msgr2=0x7f9d1419d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 0x7f9d1419d530 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c007ee0 tx=0x7f9d0c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 shutdown_connections 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d14105f10 0x7f9d1419d530 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9d000a11e0 0x7f9d000a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d14100cd0 0x7f9d14195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 --2- 172.21.15.31:0/3665791395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9d141035f0 0x7f9d141964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 >> 172.21.15.31:0/3665791395 conn(0x7f9d140fa750 msgr2=0x7f9d14101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 shutdown_connections 2023-12-14T18:27:27.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.026+0000 7f9d1bc1d700 1 -- 172.21.15.31:0/3665791395 wait complete. 2023-12-14T18:27:27.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 -- 172.21.15.31:0/3893398953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788105f40 msgr2=0x7f1788108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 --2- 172.21.15.31:0/3893398953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788105f40 0x7f1788108340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f178000b0a0 tx=0x7f178000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 -- 172.21.15.31:0/3893398953 shutdown_connections 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 --2- 172.21.15.31:0/3893398953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788105f40 0x7f1788108340 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 --2- 172.21.15.31:0/3893398953 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788100d00 0x7f17881030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 --2- 172.21.15.31:0/3893398953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788103620 0x7f1788105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 -- 172.21.15.31:0/3893398953 >> 172.21.15.31:0/3893398953 conn(0x7f17880fa740 msgr2=0x7f17880fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 -- 172.21.15.31:0/3893398953 shutdown_connections 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 -- 172.21.15.31:0/3893398953 wait complete. 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.180+0000 7f178df06700 1 Processor -- start 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 -- start start 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788100d00 0x7f1788100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 0x7f17880fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788105f40 0x7f17880ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17880ff990 con 0x7f1788103620 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17880ffb00 con 0x7f1788105f40 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f178df06700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17881a0100 con 0x7f1788100d00 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 0x7f17880fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 0x7f17880fecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59136/0 (socket says 172.21.15.31:59136) 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 -- 172.21.15.31:0/484654585 learned_addr learned my addr 172.21.15.31:0/484654585 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f17877fe700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788100d00 0x7f1788100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1787fff700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788105f40 0x7f17880ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 -- 172.21.15.31:0/484654585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788100d00 msgr2=0x7f1788100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788100d00 0x7f1788100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 -- 172.21.15.31:0/484654585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788105f40 msgr2=0x7f17880ff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788105f40 0x7f17880ff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 -- 172.21.15.31:0/484654585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009d00 con 0x7f1788103620 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1787fff700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788105f40 0x7f17880ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f17877fe700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788100d00 0x7f1788100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.181+0000 7f1786ffd700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 0x7f17880fecb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d950 tx=0x7f177c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.182+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c009b40 con 0x7f1788103620 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.182+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f177c009ca0 con 0x7f1788103620 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.182+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178810bb80 con 0x7f1788103620 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.182+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c00f620 con 0x7f1788103620 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.182+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178810b9f0 con 0x7f1788103620 2023-12-14T18:27:27.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.183+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17880fc2b0 con 0x7f1788103620 2023-12-14T18:27:27.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.184+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f177c010460 con 0x7f1788103620 2023-12-14T18:27:27.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.185+0000 7f1784ff9700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17700a1260 0x7f17700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.185+0000 7f17877fe700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17700a1260 0x7f17700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.185+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f177c0cd0b0 con 0x7f1788103620 2023-12-14T18:27:27.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.185+0000 7f17877fe700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17700a1260 0x7f17700a3710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1778009cb0 tx=0x7f1778009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:27.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.189+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f177c096390 con 0x7f1788103620 2023-12-14T18:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.401+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17880618b0 con 0x7f1788103620 2023-12-14T18:27:27.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.401+0000 7f1784ff9700 1 -- 172.21.15.31:0/484654585 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f177c0999b0 con 0x7f1788103620 2023-12-14T18:27:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.403+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17700a1260 msgr2=0x7f17700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.403+0000 7f178df06700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17700a1260 0x7f17700a3710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1778009cb0 tx=0x7f1778009260 comp rx=0 tx=0).stop 2023-12-14T18:27:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.403+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 msgr2=0x7f17880fecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.403+0000 7f178df06700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 0x7f17880fecb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f177c00d950 tx=0x7f177c00dd10 comp rx=0 tx=0).stop 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 shutdown_connections 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1788105f40 0x7f17880ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17700a1260 0x7f17700a3710 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1788100d00 0x7f1788100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 --2- 172.21.15.31:0/484654585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1788103620 0x7f17880fecb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 >> 172.21.15.31:0/484654585 conn(0x7f17880fa740 msgr2=0x7f1788101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 shutdown_connections 2023-12-14T18:27:27.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.404+0000 7f178df06700 1 -- 172.21.15.31:0/484654585 wait complete. 2023-12-14T18:27:27.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:27.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 -- 172.21.15.31:0/3381027427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 msgr2=0x7f9068101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 --2- 172.21.15.31:0/3381027427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f9068101890 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f905c009990 tx=0x7f905c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 -- 172.21.15.31:0/3381027427 shutdown_connections 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 --2- 172.21.15.31:0/3381027427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9068101dd0 0x7f9068108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 --2- 172.21.15.31:0/3381027427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9068100250 0x7f9068100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 --2- 172.21.15.31:0/3381027427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f9068101890 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 -- 172.21.15.31:0/3381027427 >> 172.21.15.31:0/3381027427 conn(0x7f90680fb850 msgr2=0x7f90680fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 -- 172.21.15.31:0/3381027427 shutdown_connections 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.560+0000 7f9070654700 1 -- 172.21.15.31:0/3381027427 wait complete. 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 Processor -- start 2023-12-14T18:27:27.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 -- start start 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9068100250 0x7f9068195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f90681964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9068101dd0 0x7f906819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906819db20 con 0x7f9068101440 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906819dc60 con 0x7f9068100250 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f9070654700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906819ddf0 con 0x7f9068101dd0 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f90681964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f90681964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59156/0 (socket says 172.21.15.31:59156) 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 -- 172.21.15.31:0/561170383 learned_addr learned my addr 172.21.15.31:0/561170383 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906ebf1700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9068101dd0 0x7f906819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906e3f0700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9068100250 0x7f9068195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 -- 172.21.15.31:0/561170383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9068101dd0 msgr2=0x7f906819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9068101dd0 0x7f906819d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 -- 172.21.15.31:0/561170383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9068100250 msgr2=0x7f9068195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9068100250 0x7f9068195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.561+0000 7f906dbef700 1 -- 172.21.15.31:0/561170383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060009520 con 0x7f9068101440 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.562+0000 7f906dbef700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f90681964e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f905c00b620 tx=0x7f905c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.562+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c01d070 con 0x7f9068101440 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.562+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f905c005030 con 0x7f9068101440 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.562+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c0095f0 con 0x7f9068101440 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.562+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c004ba0 con 0x7f9068101440 2023-12-14T18:27:27.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.562+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906810bbb0 con 0x7f9068101440 2023-12-14T18:27:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.563+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9068067160 con 0x7f9068101440 2023-12-14T18:27:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.564+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f905c0051a0 con 0x7f9068101440 2023-12-14T18:27:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.565+0000 7f90677fe700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f90500a1260 0x7f90500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.565+0000 7f906e3f0700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f90500a1260 0x7f90500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.565+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f905c0cdd60 con 0x7f9068101440 2023-12-14T18:27:27.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.566+0000 7f906e3f0700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f90500a1260 0x7f90500a3710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f90681012a0 tx=0x7f905800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:27.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.570+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f905c0970d0 con 0x7f9068101440 2023-12-14T18:27:27.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:27 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3143998887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:27 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3665791395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:27 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/484654585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.780+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9068107120 con 0x7f9068101440 2023-12-14T18:27:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.780+0000 7f90677fe700 1 -- 172.21.15.31:0/561170383 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f905c09a6f0 con 0x7f9068101440 2023-12-14T18:27:27.782 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f90500a1260 msgr2=0x7f90500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f90500a1260 0x7f90500a3710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f90681012a0 tx=0x7f905800b040 comp rx=0 tx=0).stop 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 msgr2=0x7f90681964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f90681964e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f905c00b620 tx=0x7f905c004730 comp rx=0 tx=0).stop 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 shutdown_connections 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f90500a1260 0x7f90500a3710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9068101dd0 0x7f906819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9068100250 0x7f9068195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 --2- 172.21.15.31:0/561170383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9068101440 0x7f90681964e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 >> 172.21.15.31:0/561170383 conn(0x7f90680fb850 msgr2=0x7f90680fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:27.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 shutdown_connections 2023-12-14T18:27:27.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.783+0000 7f9070654700 1 -- 172.21.15.31:0/561170383 wait complete. 2023-12-14T18:27:27.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:27 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3143998887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:27 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3665791395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:27 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/484654585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 -- 172.21.15.31:0/3160704026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec100c90 msgr2=0x7fefec107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 --2- 172.21.15.31:0/3160704026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec100c90 0x7fefec107550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fefe400b0a0 tx=0x7fefe400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 -- 172.21.15.31:0/3160704026 shutdown_connections 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 --2- 172.21.15.31:0/3160704026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec100c90 0x7fefec107550 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 --2- 172.21.15.31:0/3160704026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec0ff110 0x7fefec0ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 --2- 172.21.15.31:0/3160704026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 0x7fefec100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 -- 172.21.15.31:0/3160704026 >> 172.21.15.31:0/3160704026 conn(0x7fefec0fa6f0 msgr2=0x7fefec0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:27.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 -- 172.21.15.31:0/3160704026 shutdown_connections 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.936+0000 7feff39a4700 1 -- 172.21.15.31:0/3160704026 wait complete. 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 Processor -- start 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 -- start start 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 0x7fefec196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 0x7fefec196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec100c90 0x7fefec19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec19dcd0 con 0x7fefec0ff110 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec19de10 con 0x7fefec100300 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff39a4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec19dfa0 con 0x7fefec100c90 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 0x7fefec196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 0x7fefec196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59174/0 (socket says 172.21.15.31:59174) 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 -- 172.21.15.31:0/3310214765 learned_addr learned my addr 172.21.15.31:0/3310214765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1f41700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec100c90 0x7fefec19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff0f3f700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 0x7fefec196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 -- 172.21.15.31:0/3310214765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec100c90 msgr2=0x7fefec19d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec100c90 0x7fefec19d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 -- 172.21.15.31:0/3310214765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 msgr2=0x7fefec196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 0x7fefec196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1740700 1 -- 172.21.15.31:0/3310214765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4009d00 con 0x7fefec0ff110 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff1f41700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec100c90 0x7fefec19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.937+0000 7feff0f3f700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 0x7fefec196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:27.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.938+0000 7feff1740700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 0x7fefec196150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00d980 tx=0x7fefdc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.938+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00de60 con 0x7fefec0ff110 2023-12-14T18:27:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.938+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefdc0044e0 con 0x7fefec0ff110 2023-12-14T18:27:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.938+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefec10aad0 con 0x7fefec0ff110 2023-12-14T18:27:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.938+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc003be0 con 0x7fefec0ff110 2023-12-14T18:27:27.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.938+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec10a940 con 0x7fefec0ff110 2023-12-14T18:27:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.939+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec0679f0 con 0x7fefec0ff110 2023-12-14T18:27:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.940+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fefdc009bb0 con 0x7fefec0ff110 2023-12-14T18:27:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.941+0000 7fefe27fc700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fefd80a11e0 0x7fefd80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.941+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fefdc0ccf50 con 0x7fefec0ff110 2023-12-14T18:27:27.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.941+0000 7feff0f3f700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fefd80a11e0 0x7fefd80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:27.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.941+0000 7feff0f3f700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fefd80a11e0 0x7fefd80a3690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009b50 tx=0x7fefe8009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:27.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:27.944+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fefdc0961d0 con 0x7fefec0ff110 2023-12-14T18:27:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:27 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3143998887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:27 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3665791395' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:27 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/484654585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.203+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefec19ef50 con 0x7fefec0ff110 2023-12-14T18:27:28.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.204+0000 7fefe27fc700 1 -- 172.21.15.31:0/3310214765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7fefdc009940 con 0x7fefec0ff110 2023-12-14T18:27:28.205 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:28.206 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fefd80a11e0 msgr2=0x7fefd80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fefd80a11e0 0x7fefd80a3690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009b50 tx=0x7fefe8009550 comp rx=0 tx=0).stop 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 msgr2=0x7fefec196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 0x7fefec196150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00d980 tx=0x7fefdc00dd40 comp rx=0 tx=0).stop 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 shutdown_connections 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fefd80a11e0 0x7fefd80a3690 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefec100c90 0x7fefec19d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefec0ff110 0x7fefec196150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 --2- 172.21.15.31:0/3310214765 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefec100300 0x7fefec196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.206+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 >> 172.21.15.31:0/3310214765 conn(0x7fefec0fa6f0 msgr2=0x7fefec067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.207+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 shutdown_connections 2023-12-14T18:27:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.207+0000 7feff39a4700 1 -- 172.21.15.31:0/3310214765 wait complete. 2023-12-14T18:27:28.209 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:28.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 -- 172.21.15.31:0/921063771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708100c90 msgr2=0x7f7708103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 --2- 172.21.15.31:0/921063771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708100c90 0x7f7708103070 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7704009990 tx=0x7f7704009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 -- 172.21.15.31:0/921063771 shutdown_connections 2023-12-14T18:27:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 --2- 172.21.15.31:0/921063771 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708105ed0 0x7f7708108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 --2- 172.21.15.31:0/921063771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708100c90 0x7f7708103070 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 --2- 172.21.15.31:0/921063771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77081035b0 0x7f7708105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 -- 172.21.15.31:0/921063771 >> 172.21.15.31:0/921063771 conn(0x7f77080fa6f0 msgr2=0x7f77080fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 -- 172.21.15.31:0/921063771 shutdown_connections 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.360+0000 7f770f447700 1 -- 172.21.15.31:0/921063771 wait complete. 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 Processor -- start 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 -- start start 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708100c90 0x7f7708195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 0x7f77081964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7708105ed0 0x7f770819d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770819daf0 con 0x7f77081035b0 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770819dc30 con 0x7f7708100c90 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770f447700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770819ddc0 con 0x7f7708105ed0 2023-12-14T18:27:28.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770c9e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 0x7f77081964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770c9e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 0x7f77081964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59194/0 (socket says 172.21.15.31:59194) 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770c9e2700 1 -- 172.21.15.31:0/4016122547 learned_addr learned my addr 172.21.15.31:0/4016122547 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770d1e3700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708100c90 0x7f7708195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770d9e4700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7708105ed0 0x7f770819d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.361+0000 7f770c9e2700 1 -- 172.21.15.31:0/4016122547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7708105ed0 msgr2=0x7f770819d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770c9e2700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7708105ed0 0x7f770819d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770c9e2700 1 -- 172.21.15.31:0/4016122547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708100c90 msgr2=0x7f7708195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770c9e2700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708100c90 0x7f7708195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770c9e2700 1 -- 172.21.15.31:0/4016122547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040095f0 con 0x7f77081035b0 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770d9e4700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7708105ed0 0x7f770819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770c9e2700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 0x7f77081964b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f76f800d950 tx=0x7f76f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8009940 con 0x7f77081035b0 2023-12-14T18:27:28.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f8010460 con 0x7f77081035b0 2023-12-14T18:27:28.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800f620 con 0x7f77081035b0 2023-12-14T18:27:28.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770810bb90 con 0x7f77081035b0 2023-12-14T18:27:28.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.362+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770810ba00 con 0x7f77081035b0 2023-12-14T18:27:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.363+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77080fc260 con 0x7f77081035b0 2023-12-14T18:27:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.364+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f76f80108d0 con 0x7f77081035b0 2023-12-14T18:27:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.365+0000 7f76fe7fc700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f76f40a11e0 0x7f76f40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.365+0000 7f770d1e3700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f76f40a11e0 0x7f76f40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.366+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76f80ccc40 con 0x7f77081035b0 2023-12-14T18:27:28.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.366+0000 7f770d1e3700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f76f40a11e0 0x7f76f40a3690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f770400b2e0 tx=0x7f7704005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:28.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.369+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f76f8095f20 con 0x7f77081035b0 2023-12-14T18:27:28.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.580+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77080679f0 con 0x7f77081035b0 2023-12-14T18:27:28.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.582+0000 7f76fe7fc700 1 -- 172.21.15.31:0/4016122547 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f76f8016090 con 0x7f77081035b0 2023-12-14T18:27:28.583 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:28.584 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f76f40a11e0 msgr2=0x7f76f40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f76f40a11e0 0x7f76f40a3690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f770400b2e0 tx=0x7f7704005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 msgr2=0x7f77081964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 0x7f77081964b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f76f800d950 tx=0x7f76f800dc60 comp rx=0 tx=0).stop 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 shutdown_connections 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f76f40a11e0 0x7f76f40a3690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7708105ed0 0x7f770819d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7708100c90 0x7f7708195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 --2- 172.21.15.31:0/4016122547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77081035b0 0x7f77081964b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 >> 172.21.15.31:0/4016122547 conn(0x7f77080fa6f0 msgr2=0x7f77081018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 shutdown_connections 2023-12-14T18:27:28.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.584+0000 7f770f447700 1 -- 172.21.15.31:0/4016122547 wait complete. 2023-12-14T18:27:28.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:28.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 -- 172.21.15.31:0/644646189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388100230 msgr2=0x7f7388100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 --2- 172.21.15.31:0/644646189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388100230 0x7f7388100640 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7370009990 tx=0x7f7370009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 -- 172.21.15.31:0/644646189 shutdown_connections 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 --2- 172.21.15.31:0/644646189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7388101db0 0x7f7388108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 --2- 172.21.15.31:0/644646189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388100230 0x7f7388100640 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 --2- 172.21.15.31:0/644646189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7388101420 0x7f7388101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 -- 172.21.15.31:0/644646189 >> 172.21.15.31:0/644646189 conn(0x7f73880fb850 msgr2=0x7f73880fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 -- 172.21.15.31:0/644646189 shutdown_connections 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.740+0000 7f738d00c700 1 -- 172.21.15.31:0/644646189 wait complete. 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 Processor -- start 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 -- start start 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7388100230 0x7f7388196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 0x7f7388196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7388101db0 0x7f738819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738819dbd0 con 0x7f7388101420 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738819dd10 con 0x7f7388101db0 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738d00c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738819dea0 con 0x7f7388100230 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.741+0000 7f738659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 0x7f7388196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 0x7f7388196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59214/0 (socket says 172.21.15.31:59214) 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 -- 172.21.15.31:0/1209288157 learned_addr learned my addr 172.21.15.31:0/1209288157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f7386d9d700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7388100230 0x7f7388196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738759e700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7388101db0 0x7f738819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 -- 172.21.15.31:0/1209288157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7388100230 msgr2=0x7f7388196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7388100230 0x7f7388196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 -- 172.21.15.31:0/1209288157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7388101db0 msgr2=0x7f738819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7388101db0 0x7f738819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 -- 172.21.15.31:0/1209288157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700095f0 con 0x7f7388101420 2023-12-14T18:27:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738659c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 0x7f7388196590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f737800e9a0 tx=0x7f737800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378009790 con 0x7f7388101420 2023-12-14T18:27:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738810bba0 con 0x7f7388101420 2023-12-14T18:27:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7378010430 con 0x7f7388101420 2023-12-14T18:27:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378018680 con 0x7f7388101420 2023-12-14T18:27:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.742+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738810ba10 con 0x7f7388101420 2023-12-14T18:27:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.743+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73881077c0 con 0x7f7388101420 2023-12-14T18:27:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.744+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f73780105a0 con 0x7f7388101420 2023-12-14T18:27:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.745+0000 7f737ffff700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73740a11e0 0x7f73740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.745+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7378014070 con 0x7f7388101420 2023-12-14T18:27:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.745+0000 7f7386d9d700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73740a11e0 0x7f73740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.746+0000 7f7386d9d700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73740a11e0 0x7f73740a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7370006010 tx=0x7f7370005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.749+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7378096440 con 0x7f7388101420 2023-12-14T18:27:28.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:28 smithi204 ceph-mon[43866]: pgmap v277: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:28.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:28 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/561170383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:28.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:28 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3310214765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:28.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.956+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7388061700 con 0x7f7388101420 2023-12-14T18:27:28.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.956+0000 7f737ffff700 1 -- 172.21.15.31:0/1209288157 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73780260a0 con 0x7f7388101420 2023-12-14T18:27:28.958 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:28.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73740a11e0 msgr2=0x7f73740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73740a11e0 0x7f73740a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7370006010 tx=0x7f7370005960 comp rx=0 tx=0).stop 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 msgr2=0x7f7388196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 0x7f7388196590 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f737800e9a0 tx=0x7f737800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 shutdown_connections 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7388101db0 0x7f738819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f73740a11e0 0x7f73740a3690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7388100230 0x7f7388196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 --2- 172.21.15.31:0/1209288157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7388101420 0x7f7388196590 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 >> 172.21.15.31:0/1209288157 conn(0x7f73880fb850 msgr2=0x7f7388104900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 shutdown_connections 2023-12-14T18:27:28.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:28.959+0000 7f738d00c700 1 -- 172.21.15.31:0/1209288157 wait complete. 2023-12-14T18:27:28.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:28 smithi031 ceph-mon[42085]: pgmap v277: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:28.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:28 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/561170383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:28.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:28 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3310214765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:28.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:28 smithi176 ceph-mon[43506]: pgmap v277: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:28 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/561170383' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:28.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:28 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3310214765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 -- 172.21.15.31:0/2749369844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954100ca0 msgr2=0x7f5954103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 --2- 172.21.15.31:0/2749369844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954100ca0 0x7f5954103080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f593c0098f0 tx=0x7f593c009c00 comp rx=0 tx=0).stop 2023-12-14T18:27:29.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 -- 172.21.15.31:0/2749369844 shutdown_connections 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 --2- 172.21.15.31:0/2749369844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5954105ee0 0x7f5954108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 --2- 172.21.15.31:0/2749369844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954100ca0 0x7f5954103080 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 --2- 172.21.15.31:0/2749369844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 0x7f59541059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 -- 172.21.15.31:0/2749369844 >> 172.21.15.31:0/2749369844 conn(0x7f59540fa740 msgr2=0x7f59540fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 -- 172.21.15.31:0/2749369844 shutdown_connections 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.128+0000 7f595a139700 1 -- 172.21.15.31:0/2749369844 wait complete. 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 Processor -- start 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 -- start start 2023-12-14T18:27:29.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5954100ca0 0x7f5954196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 0x7f5954196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 0x7f595419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595419dbc0 con 0x7f5954105ee0 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595419dd00 con 0x7f59541035c0 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f595a139700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595419de90 con 0x7f5954100ca0 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f5953fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 0x7f595419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f5953fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 0x7f595419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59218/0 (socket says 172.21.15.31:59218) 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f5953fff700 1 -- 172.21.15.31:0/3215362193 learned_addr learned my addr 172.21.15.31:0/3215362193 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f59537fe700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5954100ca0 0x7f5954196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.129+0000 7f5952ffd700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 0x7f5954196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5953fff700 1 -- 172.21.15.31:0/3215362193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5954100ca0 msgr2=0x7f5954196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5953fff700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5954100ca0 0x7f5954196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5953fff700 1 -- 172.21.15.31:0/3215362193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 msgr2=0x7f5954196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5953fff700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 0x7f5954196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5953fff700 1 -- 172.21.15.31:0/3215362193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440095f0 con 0x7f5954105ee0 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5952ffd700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 0x7f5954196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5953fff700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 0x7f595419d5f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f594c00ef30 tx=0x7f594c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c018070 con 0x7f5954105ee0 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593c009520 con 0x7f5954105ee0 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594c00f040 con 0x7f5954105ee0 2023-12-14T18:27:29.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00cdb0 con 0x7f5954105ee0 2023-12-14T18:27:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.130+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595410bbf0 con 0x7f5954105ee0 2023-12-14T18:27:29.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.131+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59540fc2b0 con 0x7f5954105ee0 2023-12-14T18:27:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.132+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f594c003a00 con 0x7f5954105ee0 2023-12-14T18:27:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.133+0000 7f5950ff9700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f59400a11e0 0x7f59400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.133+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f594c0cd3b0 con 0x7f5954105ee0 2023-12-14T18:27:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.133+0000 7f59537fe700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f59400a11e0 0x7f59400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.134+0000 7f59537fe700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f59400a11e0 0x7f59400a3690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f593c005e90 tx=0x7f593c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:29.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.137+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f594c0966c0 con 0x7f5954105ee0 2023-12-14T18:27:29.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.343+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59540679f0 con 0x7f5954105ee0 2023-12-14T18:27:29.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.344+0000 7f5950ff9700 1 -- 172.21.15.31:0/3215362193 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f594c099ce0 con 0x7f5954105ee0 2023-12-14T18:27:29.345 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f59400a11e0 msgr2=0x7f59400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f59400a11e0 0x7f59400a3690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f593c005e90 tx=0x7f593c019040 comp rx=0 tx=0).stop 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 msgr2=0x7f595419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 0x7f595419d5f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f594c00ef30 tx=0x7f594c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 shutdown_connections 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5954105ee0 0x7f595419d5f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f59400a11e0 0x7f59400a3690 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5954100ca0 0x7f5954196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 --2- 172.21.15.31:0/3215362193 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59541035c0 0x7f5954196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 >> 172.21.15.31:0/3215362193 conn(0x7f59540fa740 msgr2=0x7f5954101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.346+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 shutdown_connections 2023-12-14T18:27:29.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.347+0000 7f595a139700 1 -- 172.21.15.31:0/3215362193 wait complete. 2023-12-14T18:27:29.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:29.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 -- 172.21.15.31:0/698306670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 msgr2=0x7f5178101cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 --2- 172.21.15.31:0/698306670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f5178101cc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5168009990 tx=0x7f5168009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 -- 172.21.15.31:0/698306670 shutdown_connections 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 --2- 172.21.15.31:0/698306670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780fecb0 0x7f5178108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 --2- 172.21.15.31:0/698306670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f5178101cc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 --2- 172.21.15.31:0/698306670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51780fe320 0x7f51780fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 -- 172.21.15.31:0/698306670 >> 172.21.15.31:0/698306670 conn(0x7f51780f9fb0 msgr2=0x7f51780fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 -- 172.21.15.31:0/698306670 shutdown_connections 2023-12-14T18:27:29.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.503+0000 7f517f006700 1 -- 172.21.15.31:0/698306670 wait complete. 2023-12-14T18:27:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 Processor -- start 2023-12-14T18:27:29.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 -- start start 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780fe320 0x7f5178196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51780fecb0 0x7f5178196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f517819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819db80 con 0x7f51781018b0 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819dcc0 con 0x7f51780fecb0 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517f006700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517819de50 con 0x7f51780fe320 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517d5a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f517819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517d5a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f517819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59226/0 (socket says 172.21.15.31:59226) 2023-12-14T18:27:29.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517d5a3700 1 -- 172.21.15.31:0/3957079670 learned_addr learned my addr 172.21.15.31:0/3957079670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.504+0000 7f517cda2700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780fe320 0x7f5178196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f5177fff700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51780fecb0 0x7f5178196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517d5a3700 1 -- 172.21.15.31:0/3957079670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780fe320 msgr2=0x7f5178196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517d5a3700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780fe320 0x7f5178196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517d5a3700 1 -- 172.21.15.31:0/3957079670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51780fecb0 msgr2=0x7f5178196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517d5a3700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51780fecb0 0x7f5178196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517d5a3700 1 -- 172.21.15.31:0/3957079670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680095f0 con 0x7f51781018b0 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517d5a3700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f517819d5b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f517000b970 tx=0x7f517000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000d860 con 0x7f51781018b0 2023-12-14T18:27:29.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f517000de80 con 0x7f51781018b0 2023-12-14T18:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517810bca0 con 0x7f51781018b0 2023-12-14T18:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170017400 con 0x7f51781018b0 2023-12-14T18:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.505+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517810bb10 con 0x7f51781018b0 2023-12-14T18:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.506+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5178107a40 con 0x7f51781018b0 2023-12-14T18:27:29.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.508+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f517000e630 con 0x7f51781018b0 2023-12-14T18:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.508+0000 7f5175ffb700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f51640a11e0 0x7f51640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.508+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5170006e40 con 0x7f51781018b0 2023-12-14T18:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.508+0000 7f517cda2700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f51640a11e0 0x7f51640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.509+0000 7f517cda2700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f51640a11e0 0x7f51640a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f516800b5c0 tx=0x7f5168005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:29.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.513+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5170095cb0 con 0x7f51781018b0 2023-12-14T18:27:29.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4016122547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:29.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1209288157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:29.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3215362193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.722+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51780619b0 con 0x7f51781018b0 2023-12-14T18:27:29.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.723+0000 7f5175ffb700 1 -- 172.21.15.31:0/3957079670 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51700992d0 con 0x7f51781018b0 2023-12-14T18:27:29.724 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f51640a11e0 msgr2=0x7f51640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f51640a11e0 0x7f51640a3690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f516800b5c0 tx=0x7f5168005fd0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 msgr2=0x7f517819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f517819d5b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f517000b970 tx=0x7f517000bc80 comp rx=0 tx=0).stop 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 shutdown_connections 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51781018b0 0x7f517819d5b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f51640a11e0 0x7f51640a3690 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51780fe320 0x7f5178196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 --2- 172.21.15.31:0/3957079670 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51780fecb0 0x7f5178196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 >> 172.21.15.31:0/3957079670 conn(0x7f51780f9fb0 msgr2=0x7f5178104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 shutdown_connections 2023-12-14T18:27:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.725+0000 7f517f006700 1 -- 172.21.15.31:0/3957079670 wait complete. 2023-12-14T18:27:29.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4016122547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1209288157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3215362193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/395933597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58105ee0 msgr2=0x7f6e58108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/395933597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58105ee0 0x7f6e58108340 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b0a0 tx=0x7f6e5000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/395933597 shutdown_connections 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/395933597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e58105ee0 0x7f6e58108340 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/395933597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58100ca0 0x7f6e58103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/395933597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e581035c0 0x7f6e581059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/395933597 >> 172.21.15.31:0/395933597 conn(0x7f6e580fa740 msgr2=0x7f6e580fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/395933597 shutdown_connections 2023-12-14T18:27:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/395933597 wait complete. 2023-12-14T18:27:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 Processor -- start 2023-12-14T18:27:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.879+0000 7f6e5e2e5700 1 -- start start 2023-12-14T18:27:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5e2e5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e58100ca0 0x7f6e58196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5e2e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 0x7f6e58196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5e2e5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58105ee0 0x7f6e5819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5e2e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5819dbc0 con 0x7f6e581035c0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5e2e5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5819dd00 con 0x7f6e58105ee0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5e2e5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5819de90 con 0x7f6e58100ca0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 0x7f6e58196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 0x7f6e58196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59240/0 (socket says 172.21.15.31:59240) 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 -- 172.21.15.31:0/1021664718 learned_addr learned my addr 172.21.15.31:0/1021664718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e57fff700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e58100ca0 0x7f6e58196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5c882700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58105ee0 0x7f6e5819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 -- 172.21.15.31:0/1021664718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e58100ca0 msgr2=0x7f6e58196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e58100ca0 0x7f6e58196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 -- 172.21.15.31:0/1021664718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58105ee0 msgr2=0x7f6e5819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58105ee0 0x7f6e5819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e577fe700 1 -- 172.21.15.31:0/1021664718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e50009d00 con 0x7f6e581035c0 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.880+0000 7f6e5c882700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58105ee0 0x7f6e5819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.881+0000 7f6e577fe700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 0x7f6e58196580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d980 tx=0x7f6e4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.881+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40004220 con 0x7f6e581035c0 2023-12-14T18:27:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.881+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5810bbe0 con 0x7f6e581035c0 2023-12-14T18:27:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.881+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e40007640 con 0x7f6e581035c0 2023-12-14T18:27:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.881+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005610 con 0x7f6e581035c0 2023-12-14T18:27:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.881+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5810ba50 con 0x7f6e581035c0 2023-12-14T18:27:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.882+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580fc2b0 con 0x7f6e581035c0 2023-12-14T18:27:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.883+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6e40004750 con 0x7f6e581035c0 2023-12-14T18:27:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.884+0000 7f6e557fa700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6e440a11e0 0x7f6e440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.884+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6e400cd110 con 0x7f6e581035c0 2023-12-14T18:27:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.884+0000 7f6e57fff700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6e440a11e0 0x7f6e440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.884+0000 7f6e57fff700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6e440a11e0 0x7f6e440a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009c60 tx=0x7f6e48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:29.887+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e400963f0 con 0x7f6e581035c0 2023-12-14T18:27:29.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4016122547' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:29.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1209288157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:29.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3215362193' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.095+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e58061930 con 0x7f6e581035c0 2023-12-14T18:27:30.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.095+0000 7f6e557fa700 1 -- 172.21.15.31:0/1021664718 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6e400160a0 con 0x7f6e581035c0 2023-12-14T18:27:30.096 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.097+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6e440a11e0 msgr2=0x7f6e440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.097+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6e440a11e0 0x7f6e440a3690 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009c60 tx=0x7f6e48009260 comp rx=0 tx=0).stop 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.097+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 msgr2=0x7f6e58196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.097+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 0x7f6e58196580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000d980 tx=0x7f6e4000dc90 comp rx=0 tx=0).stop 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 shutdown_connections 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e58105ee0 0x7f6e5819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6e440a11e0 0x7f6e440a3690 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e58100ca0 0x7f6e58196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 --2- 172.21.15.31:0/1021664718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e581035c0 0x7f6e58196580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 >> 172.21.15.31:0/1021664718 conn(0x7f6e580fa740 msgr2=0x7f6e58101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 shutdown_connections 2023-12-14T18:27:30.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.098+0000 7f6e5e2e5700 1 -- 172.21.15.31:0/1021664718 wait complete. 2023-12-14T18:27:30.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 -- 172.21.15.31:0/3525714748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 msgr2=0x7f53c0103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/3525714748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0103100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009990 tx=0x7f53b4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 -- 172.21.15.31:0/3525714748 shutdown_connections 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/3525714748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53c0105f60 0x7f53c0108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/3525714748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0103100 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/3525714748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0103640 0x7f53c0105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 -- 172.21.15.31:0/3525714748 >> 172.21.15.31:0/3525714748 conn(0x7f53c00fa720 msgr2=0x7f53c00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 -- 172.21.15.31:0/3525714748 shutdown_connections 2023-12-14T18:27:30.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.249+0000 7f53c6a6d700 1 -- 172.21.15.31:0/3525714748 wait complete. 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 Processor -- start 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 -- start start 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53c0103640 0x7f53c0196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0105f60 0x7f53c019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c019dad0 con 0x7f53c0100d20 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c019dc10 con 0x7f53c0105f60 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c6a6d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c019dda0 con 0x7f53c0103640 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c4809700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c4809700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59256/0 (socket says 172.21.15.31:59256) 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c4809700 1 -- 172.21.15.31:0/87263498 learned_addr learned my addr 172.21.15.31:0/87263498 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:30.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53bffff700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53c0103640 0x7f53c0196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c500a700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0105f60 0x7f53c019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c4809700 1 -- 172.21.15.31:0/87263498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53c0103640 msgr2=0x7f53c0196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.250+0000 7f53c4809700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53c0103640 0x7f53c0196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c4809700 1 -- 172.21.15.31:0/87263498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0105f60 msgr2=0x7f53c019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c4809700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0105f60 0x7f53c019d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c4809700 1 -- 172.21.15.31:0/87263498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b40095f0 con 0x7f53c0100d20 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c500a700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0105f60 0x7f53c019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c4809700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0195f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f53b40059d0 tx=0x7f53b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b401d070 con 0x7f53c0100d20 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53b400fc40 con 0x7f53c0100d20 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c010bba0 con 0x7f53c0100d20 2023-12-14T18:27:30.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40173f0 con 0x7f53c0100d20 2023-12-14T18:27:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.251+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c010ba10 con 0x7f53c0100d20 2023-12-14T18:27:30.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.252+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c00fc290 con 0x7f53c0100d20 2023-12-14T18:27:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.253+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f53b4022470 con 0x7f53c0100d20 2023-12-14T18:27:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.254+0000 7f53bdffb700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f53b00a11e0 0x7f53b00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.254+0000 7f53bffff700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f53b00a11e0 0x7f53b00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.254+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f53b40ce1e0 con 0x7f53c0100d20 2023-12-14T18:27:30.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.255+0000 7f53bffff700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f53b00a11e0 0x7f53b00a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009c60 tx=0x7f53ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:30.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.259+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f53b405c730 con 0x7f53c0100d20 2023-12-14T18:27:30.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.519+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53c0061900 con 0x7f53c0100d20 2023-12-14T18:27:30.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.520+0000 7f53bdffb700 1 -- 172.21.15.31:0/87263498 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f53b4097550 con 0x7f53c0100d20 2023-12-14T18:27:30.521 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:30.521 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f53b00a11e0 msgr2=0x7f53b00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f53b00a11e0 0x7f53b00a3690 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009c60 tx=0x7f53ac009260 comp rx=0 tx=0).stop 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 msgr2=0x7f53c0195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0195f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f53b40059d0 tx=0x7f53b4004730 comp rx=0 tx=0).stop 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 shutdown_connections 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0105f60 0x7f53c019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53c0100d20 0x7f53c0195f50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f53b00a11e0 0x7f53b00a3690 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 --2- 172.21.15.31:0/87263498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f53c0103640 0x7f53c0196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 >> 172.21.15.31:0/87263498 conn(0x7f53c00fa720 msgr2=0x7f53c0101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 shutdown_connections 2023-12-14T18:27:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.522+0000 7f53c6a6d700 1 -- 172.21.15.31:0/87263498 wait complete. 2023-12-14T18:27:30.525 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:30.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 -- 172.21.15.31:0/228954692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42301014b0 msgr2=0x7f4230101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 --2- 172.21.15.31:0/228954692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42301014b0 0x7f4230101900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4224009990 tx=0x7f4224009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 -- 172.21.15.31:0/228954692 shutdown_connections 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 --2- 172.21.15.31:0/228954692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4230101e40 0x7f4230108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 --2- 172.21.15.31:0/228954692 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42300fe4e0 0x7f42300fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 --2- 172.21.15.31:0/228954692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42301014b0 0x7f4230101900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.674+0000 7f42364be700 1 -- 172.21.15.31:0/228954692 >> 172.21.15.31:0/228954692 conn(0x7f42300f9e80 msgr2=0x7f42300fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:30.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.675+0000 7f42364be700 1 -- 172.21.15.31:0/228954692 shutdown_connections 2023-12-14T18:27:30.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.675+0000 7f42364be700 1 -- 172.21.15.31:0/228954692 wait complete. 2023-12-14T18:27:30.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.675+0000 7f42364be700 1 Processor -- start 2023-12-14T18:27:30.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.675+0000 7f42364be700 1 -- start start 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f42364be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 0x7f4230196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f42364be700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42301014b0 0x7f4230196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f42364be700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4230101e40 0x7f423019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f42364be700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423019dba0 con 0x7f42300fe4e0 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f42364be700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423019dce0 con 0x7f4230101e40 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f42364be700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423019de70 con 0x7f42301014b0 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 0x7f4230196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 0x7f4230196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59272/0 (socket says 172.21.15.31:59272) 2023-12-14T18:27:30.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 -- 172.21.15.31:0/131252644 learned_addr learned my addr 172.21.15.31:0/131252644 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422f7fe700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42301014b0 0x7f4230196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f4234a5b700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4230101e40 0x7f423019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 -- 172.21.15.31:0/131252644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42301014b0 msgr2=0x7f4230196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42301014b0 0x7f4230196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 -- 172.21.15.31:0/131252644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4230101e40 msgr2=0x7f423019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4230101e40 0x7f423019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 -- 172.21.15.31:0/131252644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240095f0 con 0x7f42300fe4e0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.676+0000 7f422ffff700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 0x7f4230196020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f422000e970 tx=0x7f422000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.677+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220009790 con 0x7f42300fe4e0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.677+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f422000c7e0 con 0x7f42300fe4e0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.677+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220003c70 con 0x7f42300fe4e0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.677+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423010bbd0 con 0x7f42300fe4e0 2023-12-14T18:27:30.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.677+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423010ba40 con 0x7f42300fe4e0 2023-12-14T18:27:30.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.678+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4230107240 con 0x7f42300fe4e0 2023-12-14T18:27:30.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.679+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f422000cc50 con 0x7f42300fe4e0 2023-12-14T18:27:30.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.680+0000 7f422d7fa700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f42180a1260 0x7f42180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:30.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.680+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4220014070 con 0x7f42300fe4e0 2023-12-14T18:27:30.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.680+0000 7f422f7fe700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f42180a1260 0x7f42180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:30.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.681+0000 7f422f7fe700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f42180a1260 0x7f42180a3710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f422400b2e0 tx=0x7f4224005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:30.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.684+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4220096310 con 0x7f42300fe4e0 2023-12-14T18:27:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:30 smithi204 ceph-mon[43866]: pgmap v278: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:30 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3957079670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:30 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1021664718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:30 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/87263498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:30.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.895+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f423002cfe0 con 0x7f42300fe4e0 2023-12-14T18:27:30.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.897+0000 7f422d7fa700 1 -- 172.21.15.31:0/131252644 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f4220099930 con 0x7f42300fe4e0 2023-12-14T18:27:30.898 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:30.898 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f42180a1260 msgr2=0x7f42180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f42180a1260 0x7f42180a3710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f422400b2e0 tx=0x7f4224005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 msgr2=0x7f4230196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 0x7f4230196020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f422000e970 tx=0x7f422000ec80 comp rx=0 tx=0).stop 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 shutdown_connections 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4230101e40 0x7f423019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42300fe4e0 0x7f4230196020 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.899+0000 7f42364be700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f42180a1260 0x7f42180a3710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.900+0000 7f42364be700 1 --2- 172.21.15.31:0/131252644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42301014b0 0x7f4230196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:30.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.900+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 >> 172.21.15.31:0/131252644 conn(0x7f42300f9e80 msgr2=0x7f42300fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:30.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.900+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 shutdown_connections 2023-12-14T18:27:30.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:30.900+0000 7f42364be700 1 -- 172.21.15.31:0/131252644 wait complete. 2023-12-14T18:27:30.913 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:30 smithi031 ceph-mon[42085]: pgmap v278: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:30.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:30 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3957079670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:30 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1021664718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:30 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/87263498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:30.915 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-14T18:25:55.286836+0000', 'modified': '2023-12-14T18:26:22.578285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24538, 'mds_1': 24536, 'mds_2': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24490': {'gid': 24490, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.31:6837/3783760943', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6836', 'nonce': 3783760943}, {'type': 'v1', 'addr': '172.21.15.31:6837', 'nonce': 3783760943}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.176:6837/2569700606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6836', 'nonce': 2569700606}, {'type': 'v1', 'addr': '172.21.15.176:6837', 'nonce': 2569700606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/3595337412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 3595337412}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3595337412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.204:6837/1057123586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 1057123586}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 1057123586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.176:6839/2954758257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.176:6838', 'nonce': 2954758257}, {'type': 'v1', 'addr': '172.21.15.176:6839', 'nonce': 2954758257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.204:6839/1902825357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1902825357}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1902825357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-14T18:27:30.915 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-14T18:27:30.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-14T18:27:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:30 smithi176 ceph-mon[43506]: pgmap v278: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:30 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3957079670' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:30 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1021664718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:30 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/87263498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 -- 172.21.15.31:0/3372567387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584100ca0 msgr2=0x7fd584103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3372567387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584100ca0 0x7fd584103080 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd574009990 tx=0x7fd574009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 -- 172.21.15.31:0/3372567387 shutdown_connections 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3372567387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd584105ee0 0x7fd584108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3372567387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584100ca0 0x7fd584103080 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3372567387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5841035c0 0x7fd5841059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 -- 172.21.15.31:0/3372567387 >> 172.21.15.31:0/3372567387 conn(0x7fd5840fa740 msgr2=0x7fd5840fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 -- 172.21.15.31:0/3372567387 shutdown_connections 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.053+0000 7fd58a02c700 1 -- 172.21.15.31:0/3372567387 wait complete. 2023-12-14T18:27:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 Processor -- start 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 -- start start 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd584100ca0 0x7fd584100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5841035c0 0x7fd5840fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 0x7fd5840ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840ff930 con 0x7fd584105ee0 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840ffaa0 con 0x7fd584100ca0 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.054+0000 7fd58a02c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5841a0030 con 0x7fd5841035c0 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 0x7fd5840ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 0x7fd5840ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59290/0 (socket says 172.21.15.31:59290) 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 -- 172.21.15.31:0/3746320622 learned_addr learned my addr 172.21.15.31:0/3746320622 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd5837fe700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd584100ca0 0x7fd584100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd582ffd700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5841035c0 0x7fd5840fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 -- 172.21.15.31:0/3746320622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5841035c0 msgr2=0x7fd5840fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5841035c0 0x7fd5840fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 -- 172.21.15.31:0/3746320622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd584100ca0 msgr2=0x7fd584100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd584100ca0 0x7fd584100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 -- 172.21.15.31:0/3746320622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740095f0 con 0x7fd584105ee0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd583fff700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 0x7fd5840ff190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd57c007ee0 tx=0x7fd57c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd580ff9700 1 -- 172.21.15.31:0/3746320622 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017070 con 0x7fd584105ee0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd580ff9700 1 -- 172.21.15.31:0/3746320622 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57c00f040 con 0x7fd584105ee0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58410bbf0 con 0x7fd584105ee0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd580ff9700 1 -- 172.21.15.31:0/3746320622 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c005580 con 0x7fd584105ee0 2023-12-14T18:27:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.055+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58410ba60 con 0x7fd584105ee0 2023-12-14T18:27:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.058+0000 7fd580ff9700 1 -- 172.21.15.31:0/3746320622 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd57c00db20 con 0x7fd584105ee0 2023-12-14T18:27:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd580ff9700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd56c0a11e0 0x7fd56c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5837fe700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd56c0a11e0 0x7fd56c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd580ff9700 1 -- 172.21.15.31:0/3746320622 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd57c0cbeb0 con 0x7fd584105ee0 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 Processor -- start 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 -- start start 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f52228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5222e00 0x5596f5223720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f5224120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f510b590 con 0x5596f52224b0 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f517a820 con 0x5596f5222e00 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5961bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5173cb0 con 0x5596f5223cb0 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.059+0000 7fd5837fe700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd56c0a11e0 0x7fd56c0a3690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd57400b2e0 tx=0x7fd574005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f52228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f52228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59298/0 (socket says 172.21.15.31:59298) 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 -- 172.21.15.31:0/1215481138 learned_addr learned my addr 172.21.15.31:0/1215481138 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd582ffd700 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5222e00 0x5596f5223720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd583fff700 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f5224120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 -- 172.21.15.31:0/1215481138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 msgr2=0x5596f5224120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f5224120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 -- 172.21.15.31:0/1215481138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5222e00 msgr2=0x5596f5223720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5222e00 0x5596f5223720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 -- 172.21.15.31:0/1215481138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f4ff5360 con 0x5596f52224b0 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd583fff700 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f5224120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd5837fe700 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f52228c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd57400bfa0 tx=0x7fd574020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3566ae6efe5df3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd569ffb700 1 -- 172.21.15.31:0/1215481138 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574004700 con 0x5596f52224b0 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd569ffb700 1 -- 172.21.15.31:0/1215481138 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57401e040 con 0x5596f52224b0 2023-12-14T18:27:31.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.060+0000 7fd569ffb700 1 -- 172.21.15.31:0/1215481138 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400fbf0 con 0x5596f52224b0 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 -- 172.21.15.31:0/1215481138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 msgr2=0x5596f52228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f52228c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd57400bfa0 tx=0x7fd574020000 comp rx=0 tx=0).stop 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 -- 172.21.15.31:0/1215481138 shutdown_connections 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f5224120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f52228c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1215481138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5222e00 0x5596f5223720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 -- 172.21.15.31:0/1215481138 >> 172.21.15.31:0/1215481138 conn(0x5596f5217c50 msgr2=0x5596f521d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 -- 172.21.15.31:0/1215481138 shutdown_connections 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 -- 172.21.15.31:0/1215481138 wait complete. 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.061+0000 7fd5961bab80 1 -- start start 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 Processor -- start 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f52224b0 0x5596f522c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f522c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f5233670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f510ff40 con 0x5596f5222e00 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f510fbe0 con 0x5596f5223cb0 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f4ff7650 con 0x5596f52224b0 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5961bab80 10 client.0 _ll_register_callbacks cb 0x5596f51338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd582ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f522c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd582ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f522c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59312/0 (socket says 172.21.15.31:59312) 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd582ffd700 1 -- 172.21.15.31:0/2458590303 learned_addr learned my addr 172.21.15.31:0/2458590303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd5837fe700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f52224b0 0x5596f522c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.062+0000 7fd583fff700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f5233670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd582ffd700 1 -- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f52224b0 msgr2=0x5596f522c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd582ffd700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f52224b0 0x5596f522c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd582ffd700 1 -- 172.21.15.31:0/2458590303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 msgr2=0x5596f5233670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd582ffd700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f5233670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd582ffd700 1 -- 172.21.15.31:0/2458590303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd584108f30 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd5837fe700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f52224b0 0x5596f522c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd582ffd700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f522c630 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd5780126b0 tx=0x7fd5780129c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52021a4848100d2d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd562ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd562ffd700 1 -- 172.21.15.31:0/2458590303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578017070 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd562ffd700 1 -- 172.21.15.31:0/2458590303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd578004050 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596f5215260 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd562ffd700 1 -- 172.21.15.31:0/2458590303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd578004210 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd5961bab80 10 client.15354 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5596f513a120 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.063+0000 7fd562ffd700 1 -- 172.21.15.31:0/2458590303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596f5215260 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd562ffd700 1 -- 172.21.15.31:0/2458590303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd578010450 con 0x5596f5222e00 2023-12-14T18:27:31.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5961bab80 10 client.15354 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5961bab80 10 client.15354 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5961bab80 20 client.15354 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5961bab80 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f5237340 0x5596f5236740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5961bab80 4 client.15354 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596f516cc90 con 0x5596f5237340 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.064+0000 7fd5837fe700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f5237340 0x5596f5236740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.065+0000 7fd562ffd700 10 client.15354 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.065+0000 7fd5837fe700 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f5237340 0x5596f5236740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca88d0a30e53185b server_cookie=74710fdd83ceae36 in_seq=0 out_seq=0 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.065+0000 7fd562ffd700 1 -- 172.21.15.31:0/2458590303 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5596f516cc90 con 0x5596f5237340 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.065+0000 7fd562ffd700 10 client.15354 handle_command_reply: tid=0 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.065+0000 7fd5961bab80 1 client.15354 shutdown 2023-12-14T18:27:31.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 msgr2=0x5596f522c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f522c630 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd5780126b0 tx=0x7fd5780129c0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 shutdown_connections 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f5237340 0x5596f5236740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f5233670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd562ffd700 0 client.15354 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f52224b0 0x5596f522c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 --2- 172.21.15.31:0/2458590303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f522c630 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 >> 172.21.15.31:0/2458590303 conn(0x5596f5217c50 msgr2=0x5596f521bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 shutdown_connections 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 1 -- 172.21.15.31:0/2458590303 wait complete. 2023-12-14T18:27:31.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.066+0000 7fd5961bab80 20 client.15354 trim_cache size 0 max 16384 2023-12-14T18:27:31.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 Processor -- start 2023-12-14T18:27:31.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 -- start start 2023-12-14T18:27:31.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f52224b0 0x5596f522ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f5233630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f521c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f516cc90 con 0x5596f5222e00 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f5142a50 con 0x5596f52224b0 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5961bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f511d580 con 0x5596f5223cb0 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd582ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f5233630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd582ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f5233630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59328/0 (socket says 172.21.15.31:59328) 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd582ffd700 1 -- 172.21.15.31:0/3768108207 learned_addr learned my addr 172.21.15.31:0/3768108207 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd583fff700 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f521c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.085+0000 7fd5837fe700 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f52224b0 0x5596f522ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd582ffd700 1 -- 172.21.15.31:0/3768108207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 msgr2=0x5596f521c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd582ffd700 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f521c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd582ffd700 1 -- 172.21.15.31:0/3768108207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f52224b0 msgr2=0x5596f522ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd582ffd700 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f52224b0 0x5596f522ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd582ffd700 1 -- 172.21.15.31:0/3768108207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f5139e70 con 0x5596f5222e00 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd583fff700 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f521c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd582ffd700 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f5233630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd578010860 tx=0x7fd578009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bbd533fe680088 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd56a7fc700 1 -- 172.21.15.31:0/3768108207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578017070 con 0x5596f5222e00 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd56a7fc700 1 -- 172.21.15.31:0/3768108207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5780092d0 con 0x5596f5222e00 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.086+0000 7fd56a7fc700 1 -- 172.21.15.31:0/3768108207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578003680 con 0x5596f5222e00 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 -- 172.21.15.31:0/3768108207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 msgr2=0x5596f5233630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f5233630 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd578010860 tx=0x7fd578009750 comp rx=0 tx=0).stop 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 -- 172.21.15.31:0/3768108207 shutdown_connections 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5223cb0 0x5596f521c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f52224b0 0x5596f522ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 --2- 172.21.15.31:0/3768108207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f5222e00 0x5596f5233630 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 -- 172.21.15.31:0/3768108207 >> 172.21.15.31:0/3768108207 conn(0x5596f5217c50 msgr2=0x5596f521d270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 -- 172.21.15.31:0/3768108207 shutdown_connections 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 -- 172.21.15.31:0/3768108207 wait complete. 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 -- start start 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.087+0000 7fd5961bab80 1 Processor -- start 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f522e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5222e00 0x5596f5233720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f522ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f528b1c0 con 0x5596f52224b0 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f528b330 con 0x5596f5223cb0 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f528b4f0 con 0x5596f5222e00 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5961bab80 10 client.0 _ll_register_callbacks cb 0x7fd578009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f522e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f522e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59344/0 (socket says 172.21.15.31:59344) 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 -- 172.21.15.31:0/1559714808 learned_addr learned my addr 172.21.15.31:0/1559714808 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd583fff700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f522ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd582ffd700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5222e00 0x5596f5233720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 -- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5222e00 msgr2=0x5596f5233720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5222e00 0x5596f5233720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 -- 172.21.15.31:0/1559714808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 msgr2=0x5596f522ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f522ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 -- 172.21.15.31:0/1559714808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58410ba60 con 0x5596f52224b0 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd583fff700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f522ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd5837fe700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f522e8a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ce0 tx=0x7fd57400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92a2aa8fa078a90 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.088+0000 7fd562ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd562ffd700 1 -- 172.21.15.31:0/1559714808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574034040 con 0x5596f52224b0 2023-12-14T18:27:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd562ffd700 1 -- 172.21.15.31:0/1559714808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd57403bc30 con 0x5596f52224b0 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd57401e040 con 0x5596f52224b0 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd562ffd700 1 -- 172.21.15.31:0/1559714808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd57403bdf0 con 0x5596f52224b0 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 10 client.15360 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fd578021040 con 0x5596f52224b0 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd562ffd700 1 -- 172.21.15.31:0/1559714808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd57401e040 con 0x5596f52224b0 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd562ffd700 1 -- 172.21.15.31:0/1559714808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd574024410 con 0x5596f52224b0 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 10 client.15360 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 10 client.15360 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 20 client.15360 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f528fe80 0x5596f528c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 4 client.15360 mds_command: new command op to 24514 tid=0[{"prefix": "status"}] 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5596f516c9b0 con 0x5596f528fe80 2023-12-14T18:27:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.089+0000 7fd582ffd700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f528fe80 0x5596f528c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd562ffd700 10 client.15360 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd582ffd700 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f528fe80 0x5596f528c810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69357cea48638dcd server_cookie=5b9cd7a7bdb2b322 in_seq=0 out_seq=0 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd562ffd700 1 -- 172.21.15.31:0/1559714808 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5596f516c9b0 con 0x5596f528fe80 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd562ffd700 10 client.15360 handle_command_reply: tid=0 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 client.15360 shutdown 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 msgr2=0x5596f522e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f522e8a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ce0 tx=0x7fd57400bfd0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 shutdown_connections 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5596f528fe80 0x5596f528c810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5596f5223cb0 0x5596f522ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5596f52224b0 0x5596f522e8a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd562ffd700 0 client.15360 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 --2- 172.21.15.31:0/1559714808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5596f5222e00 0x5596f5233720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 >> 172.21.15.31:0/1559714808 conn(0x5596f5217c50 msgr2=0x5596f5236740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 shutdown_connections 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.090+0000 7fd5961bab80 1 -- 172.21.15.31:0/1559714808 wait complete. 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd5961bab80 20 client.15360 trim_cache size 0 max 16384 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 2, 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24514, 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:27:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 69.519881561000005, 2023-12-14T18:27:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 24, 2023-12-14T18:27:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-12-14T18:27:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 85, 2023-12-14T18:27:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 110.925867758 2023-12-14T18:27:31.100 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd56c0a11e0 msgr2=0x7fd56c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd56c0a11e0 0x7fd56c0a3690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd57400b2e0 tx=0x7fd574005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 msgr2=0x7fd5840ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 0x7fd5840ff190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd57c007ee0 tx=0x7fd57c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 shutdown_connections 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd584105ee0 0x7fd5840ff190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd584100ca0 0x7fd584100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fd56c0a11e0 0x7fd56c0a3690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 --2- 172.21.15.31:0/3746320622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5841035c0 0x7fd5840fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 >> 172.21.15.31:0/3746320622 conn(0x7fd5840fa740 msgr2=0x7fd584101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 shutdown_connections 2023-12-14T18:27:31.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.091+0000 7fd58a02c700 1 -- 172.21.15.31:0/3746320622 wait complete. 2023-12-14T18:27:31.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-14T18:27:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 -- 172.21.15.31:0/3640471022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf00ff120 msgr2=0x7fedf00ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 --2- 172.21.15.31:0/3640471022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf00ff120 0x7fedf00ff530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009990 tx=0x7fedd8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 -- 172.21.15.31:0/3640471022 shutdown_connections 2023-12-14T18:27:31.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 --2- 172.21.15.31:0/3640471022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf0100ca0 0x7fedf0107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 --2- 172.21.15.31:0/3640471022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf00ff120 0x7fedf00ff530 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 --2- 172.21.15.31:0/3640471022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf0100310 0x7fedf0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 -- 172.21.15.31:0/3640471022 >> 172.21.15.31:0/3640471022 conn(0x7fedf00fa740 msgr2=0x7fedf00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 -- 172.21.15.31:0/3640471022 shutdown_connections 2023-12-14T18:27:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 -- 172.21.15.31:0/3640471022 wait complete. 2023-12-14T18:27:31.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.243+0000 7fedf54cb700 1 Processor -- start 2023-12-14T18:27:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 -- start start 2023-12-14T18:27:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf00ff120 0x7fedf00689a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 0x7fedf019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf0100ca0 0x7fedf019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf019db00 con 0x7fedf0100310 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf019dc70 con 0x7fedf00ff120 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedf54cb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf019de30 con 0x7fedf0100ca0 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 0x7fedf019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 0x7fedf019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59352/0 (socket says 172.21.15.31:59352) 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 -- 172.21.15.31:0/2546726776 learned_addr learned my addr 172.21.15.31:0/2546726776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedeeffd700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf00ff120 0x7fedf00689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedef7fe700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf0100ca0 0x7fedf019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 -- 172.21.15.31:0/2546726776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf0100ca0 msgr2=0x7fedf019d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf0100ca0 0x7fedf019d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 -- 172.21.15.31:0/2546726776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf00ff120 msgr2=0x7fedf00689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf00ff120 0x7fedf00689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.244+0000 7fedee7fc700 1 -- 172.21.15.31:0/2546726776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd80095f0 con 0x7fedf0100310 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.245+0000 7fedee7fc700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 0x7fedf019acd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fede000d980 tx=0x7fede000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.245+0000 7fede7fff700 1 -- 172.21.15.31:0/2546726776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0004220 con 0x7fedf0100310 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.245+0000 7fede7fff700 1 -- 172.21.15.31:0/2546726776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fede0007640 con 0x7fedf0100310 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.245+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf010aa90 con 0x7fedf0100310 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.245+0000 7fede7fff700 1 -- 172.21.15.31:0/2546726776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0005610 con 0x7fedf0100310 2023-12-14T18:27:31.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.245+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf010a900 con 0x7fedf0100310 2023-12-14T18:27:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.247+0000 7fede7fff700 1 -- 172.21.15.31:0/2546726776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fede0004750 con 0x7fedf0100310 2023-12-14T18:27:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.248+0000 7fede7fff700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7feddc0a1260 0x7feddc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.248+0000 7fede7fff700 1 -- 172.21.15.31:0/2546726776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fede00ccca0 con 0x7fedf0100310 2023-12-14T18:27:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.248+0000 7fedeeffd700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7feddc0a1260 0x7feddc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 Processor -- start 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 -- start start 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7feddc0a1260 0x7feddc0a3710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b5c0 tx=0x7fedd8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb368c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb37720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb38120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca1f590 con 0x56046cb364b0 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca8e820 con 0x56046cb36e00 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fee01659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca87cb0 con 0x56046cb37cb0 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb368c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59362/0 (socket says 172.21.15.31:59362) 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 -- 172.21.15.31:0/1102892212 learned_addr learned my addr 172.21.15.31:0/1102892212 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedee7fc700 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb37720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedef7fe700 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb38120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 -- 172.21.15.31:0/1102892212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 msgr2=0x56046cb38120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb38120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 -- 172.21.15.31:0/1102892212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 msgr2=0x56046cb37720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb37720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.249+0000 7fedeeffd700 1 -- 172.21.15.31:0/1102892212 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046c909360 con 0x56046cb364b0 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.250+0000 7fedeeffd700 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb368c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fedf0100170 tx=0x7fedd8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b5d263193a34e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.250+0000 7fede57fa700 1 -- 172.21.15.31:0/1102892212 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd800bdd0 con 0x56046cb364b0 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.250+0000 7fede57fa700 1 -- 172.21.15.31:0/1102892212 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd801e040 con 0x56046cb364b0 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.250+0000 7fede57fa700 1 -- 172.21.15.31:0/1102892212 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004760 con 0x56046cb364b0 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 -- 172.21.15.31:0/1102892212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 msgr2=0x56046cb368c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb368c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fedf0100170 tx=0x7fedd8020000 comp rx=0 tx=0).stop 2023-12-14T18:27:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 -- 172.21.15.31:0/1102892212 shutdown_connections 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb38120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb368c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 --2- 172.21.15.31:0/1102892212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb37720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 -- 172.21.15.31:0/1102892212 >> 172.21.15.31:0/1102892212 conn(0x56046cb2bc50 msgr2=0x56046cb313e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 -- 172.21.15.31:0/1102892212 shutdown_connections 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 -- 172.21.15.31:0/1102892212 wait complete. 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 -- start start 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.251+0000 7fee01659b80 1 Processor -- start 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb364b0 0x56046cb40070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca23f40 con 0x56046cb37cb0 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca23be0 con 0x56046cb36e00 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046c90b650 con 0x56046cb364b0 2023-12-14T18:27:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fee01659b80 10 client.0 _ll_register_callbacks cb 0x56046ca478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59364/0 (socket says 172.21.15.31:59364) 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 -- 172.21.15.31:0/1467681051 learned_addr learned my addr 172.21.15.31:0/1467681051 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedeeffd700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb364b0 0x56046cb40070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedee7fc700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 -- 172.21.15.31:0/1467681051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb364b0 msgr2=0x56046cb40070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb364b0 0x56046cb40070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 -- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 msgr2=0x56046cb405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 -- 172.21.15.31:0/1467681051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf0107e90 con 0x56046cb37cb0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedef7fe700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb475f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fede8007ac0 tx=0x7fede8007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1cca62e7e5e54d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedd67fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.252+0000 7fedd67fc700 1 -- 172.21.15.31:0/1467681051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8009e00 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fedd67fc700 1 -- 172.21.15.31:0/1467681051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fede8003a00 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56046cb29260 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fedd67fc700 1 -- 172.21.15.31:0/1467681051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede8003bc0 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 10 client.15372 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56046ca4e120 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fedd67fc700 1 -- 172.21.15.31:0/1467681051 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56046cb29260 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fedd67fc700 1 -- 172.21.15.31:0/1467681051 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fede8014070 con 0x56046cb37cb0 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 20 client.15372 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cb4b3f0 0x56046cb4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 4 client.15372 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:31.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.253+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56046ca80c90 con 0x56046cb4b3f0 2023-12-14T18:27:31.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.254+0000 7fedeeffd700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cb4b3f0 0x56046cb4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.254+0000 7fedd67fc700 10 client.15372 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:31.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.254+0000 7fedeeffd700 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cb4b3f0 0x56046cb4a7a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82caf313290f4e32 server_cookie=78132e8c065edbe1 in_seq=0 out_seq=0 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fedd67fc700 1 -- 172.21.15.31:0/1467681051 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56046ca80c90 con 0x56046cb4b3f0 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fedd67fc700 10 client.15372 handle_command_reply: tid=0 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 client.15372 shutdown 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 msgr2=0x56046cb475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb475f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fede8007ac0 tx=0x7fede8007dd0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 shutdown_connections 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb475f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cb4b3f0 0x56046cb4a7a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb364b0 0x56046cb40070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 --2- 172.21.15.31:0/1467681051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 >> 172.21.15.31:0/1467681051 conn(0x56046cb2bc50 msgr2=0x56046cb2fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fedd67fc700 0 client.15372 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 shutdown_connections 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 1 -- 172.21.15.31:0/1467681051 wait complete. 2023-12-14T18:27:31.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.255+0000 7fee01659b80 20 client.15372 trim_cache size 0 max 16384 2023-12-14T18:27:31.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.274+0000 7fee01659b80 1 Processor -- start 2023-12-14T18:27:31.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.274+0000 7fee01659b80 1 -- start start 2023-12-14T18:27:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fee01659b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb47e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fee01659b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb47510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fee01659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fee01659b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046cb326e0 con 0x56046cb364b0 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fee01659b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca80c90 con 0x56046cb36e00 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fee01659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046ca56a50 con 0x56046cb37cb0 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb47e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb47e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59370/0 (socket says 172.21.15.31:59370) 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 -- 172.21.15.31:0/4032812301 learned_addr learned my addr 172.21.15.31:0/4032812301 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedef7fe700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedee7fc700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb47510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 -- 172.21.15.31:0/4032812301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 msgr2=0x56046cb4a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb4a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 -- 172.21.15.31:0/4032812301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 msgr2=0x56046cb47510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb47510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 -- 172.21.15.31:0/4032812301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56046cb3f100 con 0x56046cb364b0 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedef7fe700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedee7fc700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb47510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fedeeffd700 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb47e20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fedd800a010 tx=0x7fedd801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b978d25a70ab40 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fede5ffb700 1 -- 172.21.15.31:0/4032812301 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8009290 con 0x56046cb364b0 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fede5ffb700 1 -- 172.21.15.31:0/4032812301 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd800be10 con 0x56046cb364b0 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.275+0000 7fede5ffb700 1 -- 172.21.15.31:0/4032812301 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8003bc0 con 0x56046cb364b0 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 -- 172.21.15.31:0/4032812301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 msgr2=0x56046cb47e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb47e20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fedd800a010 tx=0x7fedd801a040 comp rx=0 tx=0).stop 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 -- 172.21.15.31:0/4032812301 shutdown_connections 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb37cb0 0x56046cb4a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb364b0 0x56046cb47e20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 --2- 172.21.15.31:0/4032812301 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb36e00 0x56046cb47510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 -- 172.21.15.31:0/4032812301 >> 172.21.15.31:0/4032812301 conn(0x56046cb2bc50 msgr2=0x56046cb30980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 -- 172.21.15.31:0/4032812301 shutdown_connections 2023-12-14T18:27:31.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.276+0000 7fee01659b80 1 -- 172.21.15.31:0/4032812301 wait complete. 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 -- start start 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 Processor -- start 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb364b0 0x56046cb42270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb36e00 0x56046cb427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb9f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046cb477e0 con 0x56046cb37cb0 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046cb47950 con 0x56046cb364b0 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56046cb9f7d0 con 0x56046cb36e00 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fee01659b80 10 client.0 _ll_register_callbacks cb 0x7fede8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb9f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedeeffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb364b0 0x56046cb42270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb9f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59372/0 (socket says 172.21.15.31:59372) 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 -- 172.21.15.31:0/3827636968 learned_addr learned my addr 172.21.15.31:0/3827636968 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedee7fc700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb36e00 0x56046cb427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 -- 172.21.15.31:0/3827636968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb36e00 msgr2=0x56046cb427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb36e00 0x56046cb427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 -- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb364b0 msgr2=0x56046cb42270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.277+0000 7fedef7fe700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb364b0 0x56046cb42270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedef7fe700 1 -- 172.21.15.31:0/3827636968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56046c909360 con 0x56046cb37cb0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedeeffd700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb364b0 0x56046cb42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedef7fe700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb9f290 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fede8016980 tx=0x7fede8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2833ed4831a4f59 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedd67fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedd67fc700 1 -- 172.21.15.31:0/3827636968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede801eeb0 con 0x56046cb37cb0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd801e040 con 0x56046cb37cb0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedd67fc700 1 -- 172.21.15.31:0/3827636968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fede8016450 con 0x56046cb37cb0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedd67fc700 1 -- 172.21.15.31:0/3827636968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede8016610 con 0x56046cb37cb0 2023-12-14T18:27:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fee01659b80 10 client.15378 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56046ca4de70 con 0x56046cb37cb0 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedd67fc700 1 -- 172.21.15.31:0/3827636968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fede8031080 con 0x56046cb37cb0 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.278+0000 7fedd67fc700 1 -- 172.21.15.31:0/3827636968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fede8007af0 con 0x56046cb37cb0 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fee01659b80 10 client.15378 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fee01659b80 10 client.15378 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fee01659b80 20 client.15378 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fee01659b80 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cba4420 0x56046cba0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fee01659b80 4 client.15378 mds_command: new command op to 24536 tid=0[{"prefix": "status"}] 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56046ca809b0 con 0x56046cba4420 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fedeeffd700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cba4420 0x56046cba0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fedeeffd700 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cba4420 0x56046cba0d50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e31940f791a11c9 server_cookie=3e2265460fc51cea in_seq=0 out_seq=0 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.279+0000 7fedd67fc700 10 client.15378 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedd67fc700 1 -- 172.21.15.31:0/3827636968 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x56046ca809b0 con 0x56046cba4420 2023-12-14T18:27:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedd67fc700 10 client.15378 handle_command_reply: tid=0 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 client.15378 shutdown 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 msgr2=0x56046cb9f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb9f290 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fede8016980 tx=0x7fede8009750 comp rx=0 tx=0).stop 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 shutdown_connections 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56046cb37cb0 0x56046cb9f290 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56046cb364b0 0x56046cb42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56046cba4420 0x56046cba0d50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 --2- 172.21.15.31:0/3827636968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56046cb36e00 0x56046cb427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedd67fc700 0 client.15378 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 >> 172.21.15.31:0/3827636968 conn(0x56046cb2bc50 msgr2=0x56046cb47e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 shutdown_connections 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 1 -- 172.21.15.31:0/3827636968 wait complete. 2023-12-14T18:27:31.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fee01659b80 20 client.15378 trim_cache size 0 max 16384 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 1, 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24536, 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 70.723300189, 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 24, 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 85, 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 101.955538322 2023-12-14T18:27:31.288 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7feddc0a1260 msgr2=0x7feddc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedf54cb700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7feddc0a1260 0x7feddc0a3710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b5c0 tx=0x7fedd8005960 comp rx=0 tx=0).stop 2023-12-14T18:27:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 msgr2=0x7fedf019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.280+0000 7fedf54cb700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 0x7fedf019acd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fede000d980 tx=0x7fede000dc90 comp rx=0 tx=0).stop 2023-12-14T18:27:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 shutdown_connections 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7feddc0a1260 0x7feddc0a3710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedf0100ca0 0x7fedf019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf00ff120 0x7fedf00689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 --2- 172.21.15.31:0/2546726776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedf0100310 0x7fedf019acd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 >> 172.21.15.31:0/2546726776 conn(0x7fedf00fa740 msgr2=0x7fedf00fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 shutdown_connections 2023-12-14T18:27:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.281+0000 7fedf54cb700 1 -- 172.21.15.31:0/2546726776 wait complete. 2023-12-14T18:27:31.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.433+0000 7fc36d316700 1 -- 172.21.15.31:0/1019865035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3680ff140 msgr2=0x7fc3680ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.433+0000 7fc36d316700 1 --2- 172.21.15.31:0/1019865035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3680ff140 0x7fc3680ff550 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc358009990 tx=0x7fc358009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 -- 172.21.15.31:0/1019865035 shutdown_connections 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 --2- 172.21.15.31:0/1019865035 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc368100cc0 0x7fc368107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 --2- 172.21.15.31:0/1019865035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3680ff140 0x7fc3680ff550 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 --2- 172.21.15.31:0/1019865035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc368100330 0x7fc368100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 -- 172.21.15.31:0/1019865035 >> 172.21.15.31:0/1019865035 conn(0x7fc3680fa780 msgr2=0x7fc3680fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 -- 172.21.15.31:0/1019865035 shutdown_connections 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 -- 172.21.15.31:0/1019865035 wait complete. 2023-12-14T18:27:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 Processor -- start 2023-12-14T18:27:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.434+0000 7fc36d316700 1 -- start start 2023-12-14T18:27:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc36d316700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 0x7fc368069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc36d316700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 0x7fc36819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc36d316700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc368100cc0 0x7fc36819d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc36d316700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819dae0 con 0x7fc368100330 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc36d316700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819dc50 con 0x7fc368100cc0 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc36d316700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36819de10 con 0x7fc3680ff140 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 0x7fc368069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc3667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 0x7fc36819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 0x7fc368069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:59424/0 (socket says 172.21.15.31:59424) 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 -- 172.21.15.31:0/69887726 learned_addr learned my addr 172.21.15.31:0/69887726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc3667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 0x7fc36819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59396/0 (socket says 172.21.15.31:59396) 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc3677fe700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc368100cc0 0x7fc36819d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 -- 172.21.15.31:0/69887726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc368100cc0 msgr2=0x7fc36819d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc368100cc0 0x7fc36819d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 -- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 msgr2=0x7fc36819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 0x7fc36819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 -- 172.21.15.31:0/69887726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580095f0 con 0x7fc3680ff140 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc3667fc700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 0x7fc36819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.435+0000 7fc366ffd700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 0x7fc368069bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc3580047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.436+0000 7fc34ffff700 1 -- 172.21.15.31:0/69887726 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35801d070 con 0x7fc3680ff140 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.436+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36810aab0 con 0x7fc3680ff140 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.436+0000 7fc34ffff700 1 -- 172.21.15.31:0/69887726 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc358022470 con 0x7fc3680ff140 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.436+0000 7fc34ffff700 1 -- 172.21.15.31:0/69887726 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358017650 con 0x7fc3680ff140 2023-12-14T18:27:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.436+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36810a920 con 0x7fc3680ff140 2023-12-14T18:27:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.438+0000 7fc34ffff700 1 -- 172.21.15.31:0/69887726 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc3580225e0 con 0x7fc3680ff140 2023-12-14T18:27:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.439+0000 7fc34ffff700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc3500a11e0 0x7fc3500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.439+0000 7fc34ffff700 1 -- 172.21.15.31:0/69887726 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3580ce3f0 con 0x7fc3680ff140 2023-12-14T18:27:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.439+0000 7fc3667fc700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc3500a11e0 0x7fc3500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.439+0000 7fc3794a4b80 1 Processor -- start 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.439+0000 7fc3794a4b80 1 -- start start 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc3500a11e0 0x7fc3500a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011490 tx=0x7fc35c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7d4b0 0x55d158a7d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 0x55d158a7e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158966590 con 0x55d158a7de00 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1589d5820 con 0x55d158a7d4b0 2023-12-14T18:27:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1589cecb0 con 0x55d158a7ecb0 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 0x55d158a7e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 0x55d158a7e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59402/0 (socket says 172.21.15.31:59402) 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 -- 172.21.15.31:0/1351280506 learned_addr learned my addr 172.21.15.31:0/1351280506 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc366ffd700 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7d4b0 0x55d158a7d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3677fe700 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 -- 172.21.15.31:0/1351280506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 msgr2=0x55d158a7f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a7f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 -- 172.21.15.31:0/1351280506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7d4b0 msgr2=0x55d158a7d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7d4b0 0x55d158a7d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 -- 172.21.15.31:0/1351280506 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d158850360 con 0x55d158a7de00 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.440+0000 7fc3667fc700 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 0x55d158a7e720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0043c0 tx=0x7fc35c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2bbf3fe7845d3b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc34d7fa700 1 -- 172.21.15.31:0/1351280506 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c003d10 con 0x55d158a7de00 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc34d7fa700 1 -- 172.21.15.31:0/1351280506 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c003e70 con 0x55d158a7de00 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc34d7fa700 1 -- 172.21.15.31:0/1351280506 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00e7d0 con 0x55d158a7de00 2023-12-14T18:27:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1351280506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 msgr2=0x55d158a7e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 0x55d158a7e720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0043c0 tx=0x7fc35c014000 comp rx=0 tx=0).stop 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1351280506 shutdown_connections 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a7f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7d4b0 0x55d158a7d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1351280506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7de00 0x55d158a7e720 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.441+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1351280506 >> 172.21.15.31:0/1351280506 conn(0x55d158a72c50 msgr2=0x55d158a783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1351280506 shutdown_connections 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1351280506 wait complete. 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 -- start start 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 Processor -- start 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7d4b0 0x55d158a87180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 0x55d158a8e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15896af40 con 0x55d158a7ecb0 2023-12-14T18:27:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15896abe0 con 0x55d158a7de00 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158852650 con 0x55d158a7d4b0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3794a4b80 10 client.0 _ll_register_callbacks cb 0x55d15898e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 0x55d158a8e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc366ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7d4b0 0x55d158a87180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 0x55d158a8e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59418/0 (socket says 172.21.15.31:59418) 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.442+0000 7fc3677fe700 1 -- 172.21.15.31:0/2770465528 learned_addr learned my addr 172.21.15.31:0/2770465528 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3667fc700 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3677fe700 1 -- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7d4b0 msgr2=0x55d158a87180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3677fe700 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7d4b0 0x55d158a87180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3677fe700 1 -- 172.21.15.31:0/2770465528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 msgr2=0x55d158a876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3677fe700 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3677fe700 1 -- 172.21.15.31:0/2770465528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c017070 con 0x55d158a7ecb0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3677fe700 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 0x55d158a8e700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc360007a80 tx=0x7fc360007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162f663a2b03d4d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc34aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc34aff5700 1 -- 172.21.15.31:0/2770465528 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360012b40 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d158a70260 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc34aff5700 1 -- 172.21.15.31:0/2770465528 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc360012ca0 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc34aff5700 1 -- 172.21.15.31:0/2770465528 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc360012e60 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3794a4b80 10 client.15390 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d158995120 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.443+0000 7fc34aff5700 1 -- 172.21.15.31:0/2770465528 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d158a70260 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc34aff5700 1 -- 172.21.15.31:0/2770465528 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc360014040 con 0x55d158a7ecb0 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc3794a4b80 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc3794a4b80 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc3794a4b80 20 client.15390 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158a92510 0x55d158a91860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc3794a4b80 4 client.15390 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1589c7c90 con 0x55d158a92510 2023-12-14T18:27:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc366ffd700 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158a92510 0x55d158a91860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc34aff5700 10 client.15390 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.444+0000 7fc366ffd700 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158a92510 0x55d158a91860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e761cab5ed037a16 server_cookie=5b84509ea9820f6f in_seq=0 out_seq=0 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc34aff5700 1 -- 172.21.15.31:0/2770465528 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d1589c7c90 con 0x55d158a92510 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc34aff5700 10 client.15390 handle_command_reply: tid=0 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 client.15390 shutdown 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 msgr2=0x55d158a8e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 0x55d158a8e700 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc360007a80 tx=0x7fc360007d90 comp rx=0 tx=0).stop 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 shutdown_connections 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158a92510 0x55d158a91860 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7ecb0 0x55d158a8e700 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7d4b0 0x55d158a87180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc34aff5700 0 client.15390 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2770465528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 >> 172.21.15.31:0/2770465528 conn(0x55d158a72c50 msgr2=0x55d158a76ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 shutdown_connections 2023-12-14T18:27:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.445+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2770465528 wait complete. 2023-12-14T18:27:31.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.446+0000 7fc3794a4b80 20 client.15390 trim_cache size 0 max 16384 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 Processor -- start 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 -- start start 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a7f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a85040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15898d290 con 0x55d158a7d4b0 2023-12-14T18:27:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1588dac50 con 0x55d158a7de00 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158a78090 con 0x55d158a7ecb0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc366ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc366ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a7f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59424/0 (socket says 172.21.15.31:59424) 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.464+0000 7fc366ffd700 1 -- 172.21.15.31:0/1391262062 learned_addr learned my addr 172.21.15.31:0/1391262062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc3667fc700 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc3677fe700 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a85040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc366ffd700 1 -- 172.21.15.31:0/1391262062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 msgr2=0x55d158a85040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc366ffd700 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a85040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc366ffd700 1 -- 172.21.15.31:0/1391262062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 msgr2=0x55d158a85f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc366ffd700 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a85f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc366ffd700 1 -- 172.21.15.31:0/1391262062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d158994e70 con 0x55d158a7d4b0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc366ffd700 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a7f100 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc358097e90 tx=0x7fc35801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f4f8bf3e1caeb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc34dffb700 1 -- 172.21.15.31:0/1391262062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3580b1070 con 0x55d158a7d4b0 2023-12-14T18:27:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc34dffb700 1 -- 172.21.15.31:0/1391262062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3580a37e0 con 0x55d158a7d4b0 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.465+0000 7fc34dffb700 1 -- 172.21.15.31:0/1391262062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35809f150 con 0x55d158a7d4b0 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1391262062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 msgr2=0x55d158a7f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a7f100 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc358097e90 tx=0x7fc35801a040 comp rx=0 tx=0).stop 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1391262062 shutdown_connections 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a85040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a7f100 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/1391262062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a85f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1391262062 >> 172.21.15.31:0/1391262062 conn(0x55d158a72c50 msgr2=0x55d158a77980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1391262062 shutdown_connections 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- 172.21.15.31:0/1391262062 wait complete. 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- start start 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 Processor -- start 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a83630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a83bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158aea5a0 con 0x55d158a7d4b0 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158aea710 con 0x55d158a7de00 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d158aea8d0 con 0x55d158a7ecb0 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc3794a4b80 10 client.0 _ll_register_callbacks cb 0x7fc360009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.466+0000 7fc366ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a83630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a83bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a83630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:48898/0 (socket says 172.21.15.31:48898) 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a83bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:59478/0 (socket says 172.21.15.31:59478) 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 -- 172.21.15.31:0/2370877030 learned_addr learned my addr 172.21.15.31:0/2370877030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 -- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 msgr2=0x55d158a83bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a83bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 -- 172.21.15.31:0/2370877030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 msgr2=0x55d158a830f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a830f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 -- 172.21.15.31:0/2370877030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36810a920 con 0x55d158a7de00 2023-12-14T18:27:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3677fe700 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a83bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3667fc700 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a83630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc35c025000 tx=0x7fc35c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74545a48ca99829 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc34aff5700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc34aff5700 1 -- 172.21.15.31:0/2370877030 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c02b070 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc35c003e70 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc34aff5700 1 -- 172.21.15.31:0/2370877030 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc35c0054d0 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc34aff5700 1 -- 172.21.15.31:0/2370877030 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc35c005690 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.467+0000 7fc3794a4b80 10 client.25277 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d15897b950 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc34aff5700 1 -- 172.21.15.31:0/2370877030 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc35c003e70 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc34aff5700 1 -- 172.21.15.31:0/2370877030 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc35c024020 con 0x55d158a7de00 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 10 client.25277 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 10 client.25277 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:27:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 20 client.25277 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158aef580 0x55d158aebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 4 client.25277 mds_command: new command op to 24538 tid=0[{"prefix": "status"}] 2023-12-14T18:27:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d1589c79b0 con 0x55d158aef580 2023-12-14T18:27:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.468+0000 7fc366ffd700 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158aef580 0x55d158aebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc366ffd700 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158aef580 0x55d158aebf70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9dd3652ccc1ad5e server_cookie=819213ae344e0abf in_seq=0 out_seq=0 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc34aff5700 10 client.25277 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc34aff5700 1 -- 172.21.15.31:0/2370877030 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d1589c79b0 con 0x55d158aef580 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc34aff5700 10 client.25277 handle_command_reply: tid=0 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 client.25277 shutdown 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 msgr2=0x55d158a83630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a83630 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc35c025000 tx=0x7fc35c005270 comp rx=0 tx=0).stop 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 shutdown_connections 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d158aef580 0x55d158aebf70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d158a7ecb0 0x55d158a83bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d158a7d4b0 0x55d158a830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 --2- 172.21.15.31:0/2370877030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d158a7de00 0x55d158a83630 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc34aff5700 0 client.25277 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 >> 172.21.15.31:0/2370877030 conn(0x55d158a72c50 msgr2=0x55d158a85040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 shutdown_connections 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 1 -- 172.21.15.31:0/2370877030 wait complete. 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.469+0000 7fc3794a4b80 20 client.25277 trim_cache size 0 max 16384 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "37e25396-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 24538, 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-12-14T18:27:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-12-14T18:27:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 96.179598709999993, 2023-12-14T18:27:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 24, 2023-12-14T18:27:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 87, 2023-12-14T18:27:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 85, 2023-12-14T18:27:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 107.762662679 2023-12-14T18:27:31.477 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc3500a11e0 msgr2=0x7fc3500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc3500a11e0 0x7fc3500a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc35c011490 tx=0x7fc35c009260 comp rx=0 tx=0).stop 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 msgr2=0x7fc368069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 0x7fc368069bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc3580047a0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 shutdown_connections 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc368100cc0 0x7fc36819d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc3500a11e0 0x7fc3500a3690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3680ff140 0x7fc368069bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 --2- 172.21.15.31:0/69887726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc368100330 0x7fc36819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 >> 172.21.15.31:0/69887726 conn(0x7fc3680fa780 msgr2=0x7fc368103910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 shutdown_connections 2023-12-14T18:27:31.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:31.470+0000 7fc36d316700 1 -- 172.21.15.31:0/69887726 wait complete. 2023-12-14T18:27:31.485 INFO:teuthology.run_tasks:Running task kclient... 2023-12-14T18:27:31.497 INFO:tasks.kclient:Mounting kernel clients... 2023-12-14T18:27:31.498 INFO:tasks.kclient:config is {'client.0': {}} 2023-12-14T18:27:31.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:31.498 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-12-14T18:27:31.517 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-14T18:27:31.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:31.518 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-14T18:27:31.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:31.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-12-14T18:27:31.678 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-12-14T18:27:31.678 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-14T18:27:31.678 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:31.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph fs ls 2023-12-14T18:27:31.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:31 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/131252644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:31.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:31 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/131252644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:31 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/131252644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:27:32.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:32 smithi204 ceph-mon[43866]: pgmap v279: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:32.897 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:32 smithi031 ceph-mon[42085]: pgmap v279: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:32.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:32 smithi176 ceph-mon[43506]: pgmap v279: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:33.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 -- 172.21.15.31:0/334964907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 msgr2=0x7f817c10c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:33.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 --2- 172.21.15.31:0/334964907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c10c0a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f817400b0d0 tx=0x7f817400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 -- 172.21.15.31:0/334964907 shutdown_connections 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 --2- 172.21.15.31:0/334964907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c10c0a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 --2- 172.21.15.31:0/334964907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f817c0ff480 0x7f817c0ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 --2- 172.21.15.31:0/334964907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f817c0ffe20 0x7f817c103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 -- 172.21.15.31:0/334964907 >> 172.21.15.31:0/334964907 conn(0x7f817c0770d0 msgr2=0x7f817c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 -- 172.21.15.31:0/334964907 shutdown_connections 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.074+0000 7f8182856700 1 -- 172.21.15.31:0/334964907 wait complete. 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.075+0000 7f8182856700 1 Processor -- start 2023-12-14T18:27:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.075+0000 7f8182856700 1 -- start start 2023-12-14T18:27:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8182856700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f817c0ff480 0x7f817c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8182856700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f817c0ffe20 0x7f817c197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8182856700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c1a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:33.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8182856700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c198160 con 0x7f817c104520 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8182856700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c1a28d0 con 0x7f817c0ffe20 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8182856700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817c1a2a40 con 0x7f817c0ff480 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c1a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c1a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59446/0 (socket says 172.21.15.31:59446) 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 -- 172.21.15.31:0/3698308337 learned_addr learned my addr 172.21.15.31:0/3698308337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f817b7fe700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f817c0ffe20 0x7f817c197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f817bfff700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f817c0ff480 0x7f817c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 -- 172.21.15.31:0/3698308337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f817c0ff480 msgr2=0x7f817c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:33.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f817c0ff480 0x7f817c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 -- 172.21.15.31:0/3698308337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f817c0ffe20 msgr2=0x7f817c197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f817c0ffe20 0x7f817c197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.076+0000 7f8180df3700 1 -- 172.21.15.31:0/3698308337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009d00 con 0x7f817c104520 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.077+0000 7f8180df3700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c1a2390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8174007870 tx=0x7f81740078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.077+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817400e040 con 0x7f817c104520 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.077+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8174003e10 con 0x7f817c104520 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.077+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817c10f260 con 0x7f817c104520 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.077+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817401ba10 con 0x7f817c104520 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.077+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817c10f0d0 con 0x7f817c104520 2023-12-14T18:27:33.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.078+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817c10f3f0 con 0x7f817c104520 2023-12-14T18:27:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.079+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8174024400 con 0x7f817c104520 2023-12-14T18:27:33.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.080+0000 7f81797fa700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81640a1260 0x7f81640a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:33.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.080+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81740ce420 con 0x7f817c104520 2023-12-14T18:27:33.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.080+0000 7f817bfff700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81640a1260 0x7f81640a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:33.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.080+0000 7f817bfff700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81640a1260 0x7f81640a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f816c005950 tx=0x7f816c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:33.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.084+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8174097790 con 0x7f817c104520 2023-12-14T18:27:33.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.342+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f817c066ea0 con 0x7f817c104520 2023-12-14T18:27:33.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.342+0000 7f81797fa700 1 -- 172.21.15.31:0/3698308337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7f817409adb0 con 0x7f817c104520 2023-12-14T18:27:33.345 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-14T18:27:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81640a1260 msgr2=0x7f81640a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81640a1260 0x7f81640a3710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f816c005950 tx=0x7f816c00b410 comp rx=0 tx=0).stop 2023-12-14T18:27:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 msgr2=0x7f817c1a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c1a2390 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8174007870 tx=0x7f81740078a0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 shutdown_connections 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f817c104520 0x7f817c1a2390 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81640a1260 0x7f81640a3710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f817c0ff480 0x7f817c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 --2- 172.21.15.31:0/3698308337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f817c0ffe20 0x7f817c197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 >> 172.21.15.31:0/3698308337 conn(0x7f817c0770d0 msgr2=0x7f817c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 shutdown_connections 2023-12-14T18:27:33.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:33.345+0000 7f8182856700 1 -- 172.21.15.31:0/3698308337 wait complete. 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-14T18:27:33.732 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-14T18:27:33.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:33.733 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-12-14T18:27:33.756 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-14T18:27:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-14T18:27:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-12-14T18:27:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:27:33.756 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-14T18:27:33.757 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 41841sec preferred_lft 41841sec 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-12-14T18:27:33.758 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-12-14T18:27:33.758 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-14T18:27:33.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-14T18:27:33.759 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:27:33.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:33.904 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-14T18:27:33.938 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-12-14T18:27:33.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:33.940 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-12-14T18:27:34.004 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-12-14T18:27:34.004 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-14T18:27:34.004 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-14T18:27:34.004 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-14T18:27:34.005 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-14T18:27:34.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.005 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:27:34.005 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:27:34.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-14T18:27:34.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-14T18:27:34.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-14T18:27:34.006 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:27:34.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.131 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-12-14T18:27:34.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.152 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-12-14T18:27:34.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.215 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:27:34.215 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:27:34.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-14T18:27:34.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-14T18:27:34.215 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:27:34.328 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-14T18:27:34.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.328 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-14T18:27:34.329 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:27:34.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:34 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3698308337' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:27:34.562 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:34 smithi031 ceph-mon[42085]: pgmap v280: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:34.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.564 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-12-14T18:27:34.565 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-12-14T18:27:34.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-12-14T18:27:34.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-12-14T18:27:34.565 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-12-14T18:27:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:34 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3698308337' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:27:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:34 smithi204 ceph-mon[43866]: pgmap v280: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:34.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:27:34.680 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:34.700 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-14T18:27:34.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:27:34.700 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:34 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3698308337' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:27:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:34 smithi176 ceph-mon[43506]: pgmap v280: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:27:34.762 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-12-14T18:27:34.762 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-12-14T18:27:34.762 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/7844f291-1382-4478-8c9e-c751cc2b39de 2023-12-14T18:27:34.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/7844f291-1382-4478-8c9e-c751cc2b39de /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-12-14T18:27:34.862 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-12-14T18:27:34.862 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2023-12-14T18:27:34.862 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-12-14T18:27:34.971 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2023-12-14T18:27:34.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2023-12-14T18:27:34.971 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-12-14T18:27:34.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.176:3300,172.21.15.204:3300" 2023-12-14T18:27:34.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:3300,172.21.15.176:3300,172.21.15.204:3300:/volumes/_nogroup/sv_1/7844f291-1382-4478-8c9e-c751cc2b39de 2023-12-14T18:27:34.971 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=37e25396-9aad-11ee-95a5-87774f69a715" will pass to kernel 2023-12-14T18:27:34.974 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-14T18:27:34.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:34.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:35.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:35.010 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-14T18:27:35.109 INFO:teuthology.orchestra.run.smithi031.stdout:client25286 2023-12-14T18:27:35.110 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:27:35.110 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:27:35.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:35.133 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-14T18:27:35.236 INFO:teuthology.orchestra.run.smithi031.stdout:client25286 2023-12-14T18:27:35.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:35.236 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-14T18:27:35.297 INFO:teuthology.orchestra.run.smithi031.stdout:client25286 2023-12-14T18:27:35.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:27:35.298 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/37e25396-9aad-11ee-95a5-87774f69a715.client25286/status') 2023-12-14T18:27:35.371 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.25286 (3)172.21.15.31:0/2217836768 2023-12-14T18:27:35.371 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2023-12-14T18:27:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2023-12-14T18:27:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2023-12-14T18:27:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:71 bytes copied, 2.9659e-05 s, 2.4 MB/s 2023-12-14T18:27:35.373 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-14T18:27:35.386 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-12-14T18:27:35.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.544+0000 7f34f9fa3700 1 -- 172.21.15.31:0/61209254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f40734d0 msgr2=0x7f34f4073940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.544+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/61209254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f40734d0 0x7f34f4073940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009990 tx=0x7f34dc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 -- 172.21.15.31:0/61209254 shutdown_connections 2023-12-14T18:27:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/61209254 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f41080e0 0x7f34f410a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/61209254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34f4074ab0 0x7f34f4072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/61209254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f40734d0 0x7f34f4073940 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 -- 172.21.15.31:0/61209254 >> 172.21.15.31:0/61209254 conn(0x7f34f40fb810 msgr2=0x7f34f40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 -- 172.21.15.31:0/61209254 shutdown_connections 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 -- 172.21.15.31:0/61209254 wait complete. 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.545+0000 7f34f9fa3700 1 Processor -- start 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 -- start start 2023-12-14T18:27:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f40734d0 0x7f34f41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34f4074ab0 0x7f34f4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 0x7f34f419f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f419fc40 con 0x7f34f41080e0 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f419fd80 con 0x7f34f40734d0 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f9fa3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f419ff10 con 0x7f34f4074ab0 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 0x7f34f419f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 0x7f34f419f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52944/0 (socket says 172.21.15.31:52944) 2023-12-14T18:27:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f3fff700 1 -- 172.21.15.31:0/1242914107 learned_addr learned my addr 172.21.15.31:0/1242914107 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f37fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f40734d0 0x7f34f41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.546+0000 7f34f2ffd700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34f4074ab0 0x7f34f4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f3fff700 1 -- 172.21.15.31:0/1242914107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34f4074ab0 msgr2=0x7f34f4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f3fff700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34f4074ab0 0x7f34f4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f3fff700 1 -- 172.21.15.31:0/1242914107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f40734d0 msgr2=0x7f34f41980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f3fff700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f40734d0 0x7f34f41980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f3fff700 1 -- 172.21.15.31:0/1242914107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0095f0 con 0x7f34f41080e0 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f3fff700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 0x7f34f419f670 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f34ec007ee0 tx=0x7f34ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec017070 con 0x7f34f41080e0 2023-12-14T18:27:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f410dd70 con 0x7f34f41080e0 2023-12-14T18:27:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ec00f040 con 0x7f34f41080e0 2023-12-14T18:27:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec005580 con 0x7f34f41080e0 2023-12-14T18:27:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.547+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f410dbe0 con 0x7f34f41080e0 2023-12-14T18:27:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.548+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f4106820 con 0x7f34f41080e0 2023-12-14T18:27:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.549+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f34ec00db20 con 0x7f34f41080e0 2023-12-14T18:27:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.550+0000 7f34f0ff9700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34e00a11e0 0x7f34e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.550+0000 7f34f37fe700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34e00a11e0 0x7f34e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.550+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f34ec0cc4e0 con 0x7f34f41080e0 2023-12-14T18:27:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.551+0000 7f34f37fe700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34e00a11e0 0x7f34e00a3690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34e400ac50 tx=0x7f34e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.557+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f34ec0957c0 con 0x7f34f41080e0 2023-12-14T18:27:35.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.770+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34f402cfe0 con 0x7f34f41080e0 2023-12-14T18:27:35.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.772+0000 7f34f0ff9700 1 -- 172.21.15.31:0/1242914107 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f34ec098de0 con 0x7f34f41080e0 2023-12-14T18:27:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:35.775 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:35.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.774+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34e00a11e0 msgr2=0x7f34e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.774+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34e00a11e0 0x7f34e00a3690 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34e400ac50 tx=0x7f34e400a380 comp rx=0 tx=0).stop 2023-12-14T18:27:35.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.774+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 msgr2=0x7f34f419f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.774+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 0x7f34f419f670 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f34ec007ee0 tx=0x7f34ec00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 shutdown_connections 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f41080e0 0x7f34f419f670 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34f40734d0 0x7f34f41980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f34e00a11e0 0x7f34e00a3690 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 --2- 172.21.15.31:0/1242914107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34f4074ab0 0x7f34f4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 >> 172.21.15.31:0/1242914107 conn(0x7f34f40fb810 msgr2=0x7f34f410a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:35.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 shutdown_connections 2023-12-14T18:27:35.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.775+0000 7f34f9fa3700 1 -- 172.21.15.31:0/1242914107 wait complete. 2023-12-14T18:27:35.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:35.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 -- 172.21.15.31:0/1766541647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840100230 msgr2=0x7f8840100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 --2- 172.21.15.31:0/1766541647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840100230 0x7f8840100640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f883c009990 tx=0x7f883c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 -- 172.21.15.31:0/1766541647 shutdown_connections 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 --2- 172.21.15.31:0/1766541647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840101db0 0x7f8840108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 --2- 172.21.15.31:0/1766541647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840100230 0x7f8840100640 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 --2- 172.21.15.31:0/1766541647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101420 0x7f8840101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 -- 172.21.15.31:0/1766541647 >> 172.21.15.31:0/1766541647 conn(0x7f88400fb850 msgr2=0x7f88400fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 -- 172.21.15.31:0/1766541647 shutdown_connections 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 -- 172.21.15.31:0/1766541647 wait complete. 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 Processor -- start 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.932+0000 7f8847883700 1 -- start start 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8847883700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840100230 0x7f8840195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8847883700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 0x7f88401964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8847883700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101db0 0x7f884019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8847883700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019dae0 con 0x7f8840101420 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8847883700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019dc20 con 0x7f8840101db0 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8847883700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019ddb0 con 0x7f8840100230 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 0x7f88401964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 0x7f88401964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52964/0 (socket says 172.21.15.31:52964) 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 -- 172.21.15.31:0/3809432291 learned_addr learned my addr 172.21.15.31:0/3809432291 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8845e20700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101db0 0x7f884019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f884561f700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840100230 0x7f8840195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 -- 172.21.15.31:0/3809432291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840100230 msgr2=0x7f8840195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840100230 0x7f8840195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 -- 172.21.15.31:0/3809432291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101db0 msgr2=0x7f884019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101db0 0x7f884019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 -- 172.21.15.31:0/3809432291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0095f0 con 0x7f8840101420 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.933+0000 7f8844e1e700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 0x7f88401964a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f883000d950 tx=0x7f883000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.934+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830009940 con 0x7f8840101420 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.934+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8830010460 con 0x7f8840101420 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.934+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884010bbe0 con 0x7f8840101420 2023-12-14T18:27:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.934+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000f620 con 0x7f8840101420 2023-12-14T18:27:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.934+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884010ba50 con 0x7f8840101420 2023-12-14T18:27:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.935+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8840107720 con 0x7f8840101420 2023-12-14T18:27:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.936+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f88300108d0 con 0x7f8840101420 2023-12-14T18:27:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.937+0000 7f88367fc700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f882c0a11e0 0x7f882c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.937+0000 7f884561f700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f882c0a11e0 0x7f882c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.937+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f88300ccca0 con 0x7f8840101420 2023-12-14T18:27:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.937+0000 7f884561f700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f882c0a11e0 0x7f882c0a3690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f883c00b5c0 tx=0x7f883c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:35.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:35.940+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8830095f20 con 0x7f8840101420 2023-12-14T18:27:36.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.146+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8840061700 con 0x7f8840101420 2023-12-14T18:27:36.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.146+0000 7f88367fc700 1 -- 172.21.15.31:0/3809432291 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8830020020 con 0x7f8840101420 2023-12-14T18:27:36.148 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.148+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f882c0a11e0 msgr2=0x7f882c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f882c0a11e0 0x7f882c0a3690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f883c00b5c0 tx=0x7f883c005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 msgr2=0x7f88401964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 0x7f88401964a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f883000d950 tx=0x7f883000dc60 comp rx=0 tx=0).stop 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 shutdown_connections 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101db0 0x7f884019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f882c0a11e0 0x7f882c0a3690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840100230 0x7f8840195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 --2- 172.21.15.31:0/3809432291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8840101420 0x7f88401964a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 >> 172.21.15.31:0/3809432291 conn(0x7f88400fb850 msgr2=0x7f8840104860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 shutdown_connections 2023-12-14T18:27:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.149+0000 7f8847883700 1 -- 172.21.15.31:0/3809432291 wait complete. 2023-12-14T18:27:36.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 -- 172.21.15.31:0/1151474706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c105f70 msgr2=0x7f447c108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 --2- 172.21.15.31:0/1151474706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c105f70 0x7f447c108370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f447400b0a0 tx=0x7f447400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 -- 172.21.15.31:0/1151474706 shutdown_connections 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 --2- 172.21.15.31:0/1151474706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c105f70 0x7f447c108370 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 --2- 172.21.15.31:0/1151474706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447c100d30 0x7f447c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 --2- 172.21.15.31:0/1151474706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c103650 0x7f447c105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 -- 172.21.15.31:0/1151474706 >> 172.21.15.31:0/1151474706 conn(0x7f447c0fa770 msgr2=0x7f447c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 -- 172.21.15.31:0/1151474706 shutdown_connections 2023-12-14T18:27:36.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.304+0000 7f44825d0700 1 -- 172.21.15.31:0/1151474706 wait complete. 2023-12-14T18:27:36.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 Processor -- start 2023-12-14T18:27:36.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 -- start start 2023-12-14T18:27:36.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447c100d30 0x7f447c196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 0x7f447c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c105f70 0x7f447c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c19dc10 con 0x7f447c103650 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c19dd50 con 0x7f447c105f70 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.305+0000 7f44825d0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c19dee0 con 0x7f447c100d30 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 0x7f447c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 0x7f447c1965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52978/0 (socket says 172.21.15.31:52978) 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 -- 172.21.15.31:0/1011369961 learned_addr learned my addr 172.21.15.31:0/1011369961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447bfff700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447c100d30 0x7f447c196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f4480b6d700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c105f70 0x7f447c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 -- 172.21.15.31:0/1011369961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447c100d30 msgr2=0x7f447c196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447c100d30 0x7f447c196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 -- 172.21.15.31:0/1011369961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c105f70 msgr2=0x7f447c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c105f70 0x7f447c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 -- 172.21.15.31:0/1011369961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474009d00 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f4480b6d700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c105f70 0x7f447c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f447b7fe700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 0x7f447c1965d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f447000e9a0 tx=0x7f447000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009790 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c10bc10 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447000cbd0 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470018680 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.306+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c10ba80 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.307+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c0fc2e0 con 0x7f447c103650 2023-12-14T18:27:36.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.308+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f447000cd40 con 0x7f447c103650 2023-12-14T18:27:36.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.309+0000 7f44797fa700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f44640a11e0 0x7f44640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.309+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4470014070 con 0x7f447c103650 2023-12-14T18:27:36.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.309+0000 7f447bfff700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f44640a11e0 0x7f44640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.310+0000 7f447bfff700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f44640a11e0 0x7f44640a3690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f446c009c60 tx=0x7f446c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.314+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4470051730 con 0x7f447c103650 2023-12-14T18:27:36.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.522+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f447c061930 con 0x7f447c103650 2023-12-14T18:27:36.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.522+0000 7f44797fa700 1 -- 172.21.15.31:0/1011369961 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44700962e0 con 0x7f447c103650 2023-12-14T18:27:36.524 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f44640a11e0 msgr2=0x7f44640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f44640a11e0 0x7f44640a3690 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f446c009c60 tx=0x7f446c009260 comp rx=0 tx=0).stop 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 msgr2=0x7f447c1965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 0x7f447c1965d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f447000e9a0 tx=0x7f447000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 shutdown_connections 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f447c105f70 0x7f447c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f44640a11e0 0x7f44640a3690 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.524+0000 7f44825d0700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447c100d30 0x7f447c196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.525+0000 7f44825d0700 1 --2- 172.21.15.31:0/1011369961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f447c103650 0x7f447c1965d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.525+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 >> 172.21.15.31:0/1011369961 conn(0x7f447c0fa770 msgr2=0x7f447c101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.525+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 shutdown_connections 2023-12-14T18:27:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.525+0000 7f44825d0700 1 -- 172.21.15.31:0/1011369961 wait complete. 2023-12-14T18:27:36.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:36.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 -- 172.21.15.31:0/3788692893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 msgr2=0x7f84a4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 --2- 172.21.15.31:0/3788692893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4103110 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8494009990 tx=0x7f8494009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 -- 172.21.15.31:0/3788692893 shutdown_connections 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 --2- 172.21.15.31:0/3788692893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a4105f70 0x7f84a4108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 --2- 172.21.15.31:0/3788692893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4103110 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 --2- 172.21.15.31:0/3788692893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 0x7f84a4105a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 -- 172.21.15.31:0/3788692893 >> 172.21.15.31:0/3788692893 conn(0x7f84a40fa750 msgr2=0x7f84a40fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 -- 172.21.15.31:0/3788692893 shutdown_connections 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.680+0000 7f84a9042700 1 -- 172.21.15.31:0/3788692893 wait complete. 2023-12-14T18:27:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 Processor -- start 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 -- start start 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 0x7f84a4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a4105f70 0x7f84a419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419fcd0 con 0x7f84a4100d30 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419fe10 con 0x7f84a4103650 2023-12-14T18:27:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a9042700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a419ffa0 con 0x7f84a4105f70 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52992/0 (socket says 172.21.15.31:52992) 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a2d9d700 1 -- 172.21.15.31:0/98856583 learned_addr learned my addr 172.21.15.31:0/98856583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.681+0000 7f84a359e700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a4105f70 0x7f84a419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a259c700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 0x7f84a4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a2d9d700 1 -- 172.21.15.31:0/98856583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a4105f70 msgr2=0x7f84a419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a2d9d700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a4105f70 0x7f84a419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a2d9d700 1 -- 172.21.15.31:0/98856583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 msgr2=0x7f84a4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a2d9d700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 0x7f84a4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a2d9d700 1 -- 172.21.15.31:0/98856583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940095f0 con 0x7f84a4100d30 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a259c700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 0x7f84a4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a2d9d700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4198150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f849400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849401d070 con 0x7f84a4100d30 2023-12-14T18:27:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f849400fcb0 con 0x7f84a4100d30 2023-12-14T18:27:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a410bbb0 con 0x7f84a4100d30 2023-12-14T18:27:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494017650 con 0x7f84a4100d30 2023-12-14T18:27:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.682+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a410ba20 con 0x7f84a4100d30 2023-12-14T18:27:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.683+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4107ae0 con 0x7f84a4100d30 2023-12-14T18:27:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.684+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8494021410 con 0x7f84a4100d30 2023-12-14T18:27:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.685+0000 7f848bfff700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f848c0a11e0 0x7f848c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.685+0000 7f84a259c700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f848c0a11e0 0x7f848c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.686+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84940cea10 con 0x7f84a4100d30 2023-12-14T18:27:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.686+0000 7f84a259c700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f848c0a11e0 0x7f848c0a3690 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8498005d90 tx=0x7f8498005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.689+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8494097d80 con 0x7f84a4100d30 2023-12-14T18:27:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:36 smithi204 ceph-mon[43866]: pgmap v281: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:36 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1242914107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:36 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3809432291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:36 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1011369961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:36.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.896+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84a40619b0 con 0x7f84a4100d30 2023-12-14T18:27:36.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.897+0000 7f848bfff700 1 -- 172.21.15.31:0/98856583 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f849409b3a0 con 0x7f84a4100d30 2023-12-14T18:27:36.899 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:36.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f848c0a11e0 msgr2=0x7f848c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f848c0a11e0 0x7f848c0a3690 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8498005d90 tx=0x7f8498005d20 comp rx=0 tx=0).stop 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 msgr2=0x7f84a4198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4198150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f849400f460 comp rx=0 tx=0).stop 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 shutdown_connections 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f848c0a11e0 0x7f848c0a3690 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a4105f70 0x7f84a419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84a4100d30 0x7f84a4198150 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 --2- 172.21.15.31:0/98856583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84a4103650 0x7f84a4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 >> 172.21.15.31:0/98856583 conn(0x7f84a40fa750 msgr2=0x7f84a40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 shutdown_connections 2023-12-14T18:27:36.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:36.899+0000 7f84a9042700 1 -- 172.21.15.31:0/98856583 wait complete. 2023-12-14T18:27:36.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:36 smithi031 ceph-mon[42085]: pgmap v281: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:36.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:36 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1242914107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:36.914 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:36 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3809432291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:36.915 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:36 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1011369961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:36.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:36 smithi176 ceph-mon[43506]: pgmap v281: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:27:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:36 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1242914107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:36 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3809432291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:36 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1011369961' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 -- 172.21.15.31:0/3053395247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db81017f0 msgr2=0x7f8db8101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/3053395247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db81017f0 0x7f8db8101c00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009990 tx=0x7f8da8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 -- 172.21.15.31:0/3053395247 shutdown_connections 2023-12-14T18:27:37.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/3053395247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db80febf0 0x7f8db81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/3053395247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db81017f0 0x7f8db8101c00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/3053395247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8db80fe260 0x7f8db80fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 -- 172.21.15.31:0/3053395247 >> 172.21.15.31:0/3053395247 conn(0x7f8db80f9ed0 msgr2=0x7f8db80fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 -- 172.21.15.31:0/3053395247 shutdown_connections 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.053+0000 7f8dbdd04700 1 -- 172.21.15.31:0/3053395247 wait complete. 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 Processor -- start 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 -- start start 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8db80fe260 0x7f8db8100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 0x7f8db80ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81017f0 0x7f8db80ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db8100210 con 0x7f8db80febf0 2023-12-14T18:27:37.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db8100380 con 0x7f8db80fe260 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8dbdd04700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db819ff50 con 0x7f8db81017f0 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.054+0000 7f8db6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 0x7f8db80ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 0x7f8db80ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53020/0 (socket says 172.21.15.31:53020) 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 -- 172.21.15.31:0/1312538718 learned_addr learned my addr 172.21.15.31:0/1312538718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db7fff700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81017f0 0x7f8db80ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db77fe700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8db80fe260 0x7f8db8100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 -- 172.21.15.31:0/1312538718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81017f0 msgr2=0x7f8db80ffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81017f0 0x7f8db80ffa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 -- 172.21.15.31:0/1312538718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8db80fe260 msgr2=0x7f8db8100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8db80fe260 0x7f8db8100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 -- 172.21.15.31:0/1312538718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da80095f0 con 0x7f8db80febf0 2023-12-14T18:27:37.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db6ffd700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 0x7f8db80ff530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c9b0 tx=0x7f8dac00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac007b00 con 0x7f8db80febf0 2023-12-14T18:27:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dac007c60 con 0x7f8db80febf0 2023-12-14T18:27:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db810bb70 con 0x7f8db80febf0 2023-12-14T18:27:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac018660 con 0x7f8db80febf0 2023-12-14T18:27:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.055+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db810b9e0 con 0x7f8db80febf0 2023-12-14T18:27:37.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.056+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db81a06a0 con 0x7f8db80febf0 2023-12-14T18:27:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.057+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8dac00f450 con 0x7f8db80febf0 2023-12-14T18:27:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.058+0000 7f8db4ff9700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8da00a11e0 0x7f8da00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.058+0000 7f8db77fe700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8da00a11e0 0x7f8da00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.059+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8dac0cd040 con 0x7f8db80febf0 2023-12-14T18:27:37.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.059+0000 7f8db77fe700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8da00a11e0 0x7f8da00a3690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8da8006010 tx=0x7f8da800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:37.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.062+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8dac0963b0 con 0x7f8db80febf0 2023-12-14T18:27:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.273+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8db81071f0 con 0x7f8db80febf0 2023-12-14T18:27:37.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.273+0000 7f8db4ff9700 1 -- 172.21.15.31:0/1312538718 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8dac0999d0 con 0x7f8db80febf0 2023-12-14T18:27:37.275 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8da00a11e0 msgr2=0x7f8da00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8da00a11e0 0x7f8da00a3690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8da8006010 tx=0x7f8da800b540 comp rx=0 tx=0).stop 2023-12-14T18:27:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 msgr2=0x7f8db80ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 0x7f8db80ff530 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8dac00c9b0 tx=0x7f8dac00ccc0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 shutdown_connections 2023-12-14T18:27:37.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8da00a11e0 0x7f8da00a3690 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81017f0 0x7f8db80ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8db80fe260 0x7f8db8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 --2- 172.21.15.31:0/1312538718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db80febf0 0x7f8db80ff530 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 >> 172.21.15.31:0/1312538718 conn(0x7f8db80f9ed0 msgr2=0x7f8db80fc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 shutdown_connections 2023-12-14T18:27:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.276+0000 7f8dbdd04700 1 -- 172.21.15.31:0/1312538718 wait complete. 2023-12-14T18:27:37.292 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:37.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/149667337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ff3a0 msgr2=0x7fa5ac0ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/149667337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ff3a0 0x7fa5ac0ff810 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009990 tx=0x7fa5a8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/149667337 shutdown_connections 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/149667337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/149667337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5ac0fea50 0x7fa5ac0fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/149667337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ff3a0 0x7fa5ac0ff810 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/149667337 >> 172.21.15.31:0/149667337 conn(0x7fa5ac0fa6e0 msgr2=0x7fa5ac0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/149667337 shutdown_connections 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.428+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/149667337 wait complete. 2023-12-14T18:27:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 Processor -- start 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 -- start start 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0fea50 0x7fa5ac198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5ac0ff3a0 0x7fa5ac198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac19fc90 con 0x7fa5ac0ffd50 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac19fdd0 con 0x7fa5ac0ff3a0 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b3ec2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac19ff60 con 0x7fa5ac0fea50 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b245f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b245f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53036/0 (socket says 172.21.15.31:53036) 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b1c5e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0fea50 0x7fa5ac198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b245f700 1 -- 172.21.15.31:0/1326996745 learned_addr learned my addr 172.21.15.31:0/1326996745 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.429+0000 7fa5b145d700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5ac0ff3a0 0x7fa5ac198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b1c5e700 1 -- 172.21.15.31:0/1326996745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5ac0ff3a0 msgr2=0x7fa5ac198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b1c5e700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5ac0ff3a0 0x7fa5ac198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b1c5e700 1 -- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 msgr2=0x7fa5ac19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b1c5e700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b1c5e700 1 -- 172.21.15.31:0/1326996745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80095f0 con 0x7fa5ac0fea50 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b245f700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b1c5e700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0fea50 0x7fa5ac198110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00d950 tx=0x7fa59c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00f890 con 0x7fa5ac0fea50 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa59c010460 con 0x7fa5ac0fea50 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac10dd20 con 0x7fa5ac0fea50 2023-12-14T18:27:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00e610 con 0x7fa5ac0fea50 2023-12-14T18:27:37.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.430+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac10db90 con 0x7fa5ac0fea50 2023-12-14T18:27:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.431+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac109830 con 0x7fa5ac0fea50 2023-12-14T18:27:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.433+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa59c0108d0 con 0x7fa5ac0fea50 2023-12-14T18:27:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.433+0000 7fa5a2ffd700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5980a11e0 0x7fa5980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.433+0000 7fa5b145d700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5980a11e0 0x7fa5980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.433+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa59c0ccf90 con 0x7fa5ac0fea50 2023-12-14T18:27:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.434+0000 7fa5b145d700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5980a11e0 0x7fa5980a3690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800b5c0 tx=0x7fa5a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.437+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa59c096210 con 0x7fa5ac0fea50 2023-12-14T18:27:37.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.650+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5ac02cfe0 con 0x7fa5ac0fea50 2023-12-14T18:27:37.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.653+0000 7fa5a2ffd700 1 -- 172.21.15.31:0/1326996745 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7fa59c016090 con 0x7fa5ac0fea50 2023-12-14T18:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:37.655 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:37.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.654+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5980a11e0 msgr2=0x7fa5980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.654+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5980a11e0 0x7fa5980a3690 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5a800b5c0 tx=0x7fa5a8005960 comp rx=0 tx=0).stop 2023-12-14T18:27:37.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.654+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0fea50 msgr2=0x7fa5ac198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.654+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0fea50 0x7fa5ac198110 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00d950 tx=0x7fa59c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 shutdown_connections 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5ac0ffd50 0x7fa5ac19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa5980a11e0 0x7fa5980a3690 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5ac0fea50 0x7fa5ac198110 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 --2- 172.21.15.31:0/1326996745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa5ac0ff3a0 0x7fa5ac198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 >> 172.21.15.31:0/1326996745 conn(0x7fa5ac0fa6e0 msgr2=0x7fa5ac0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 shutdown_connections 2023-12-14T18:27:37.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.655+0000 7fa5b3ec2700 1 -- 172.21.15.31:0/1326996745 wait complete. 2023-12-14T18:27:37.671 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:37.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.811+0000 7f526c931700 1 -- 172.21.15.31:0/3588797538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101410 msgr2=0x7f5268101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.811+0000 7f526c931700 1 --2- 172.21.15.31:0/3588797538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101410 0x7f5268101860 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5258009990 tx=0x7f5258009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 -- 172.21.15.31:0/3588797538 shutdown_connections 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 --2- 172.21.15.31:0/3588797538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5268101da0 0x7f5268108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 --2- 172.21.15.31:0/3588797538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5268100220 0x7f5268100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 --2- 172.21.15.31:0/3588797538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101410 0x7f5268101860 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 -- 172.21.15.31:0/3588797538 >> 172.21.15.31:0/3588797538 conn(0x7f52680fb840 msgr2=0x7f52680fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 -- 172.21.15.31:0/3588797538 shutdown_connections 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 -- 172.21.15.31:0/3588797538 wait complete. 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.812+0000 7f526c931700 1 Processor -- start 2023-12-14T18:27:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 -- start start 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5268100220 0x7f5268195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5268101410 0x7f5268196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 0x7f526819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819db70 con 0x7f5268101da0 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819dcb0 con 0x7f5268100220 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526c931700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819de40 con 0x7f5268101410 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 0x7f526819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 0x7f526819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53040/0 (socket says 172.21.15.31:53040) 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 -- 172.21.15.31:0/1940352079 learned_addr learned my addr 172.21.15.31:0/1940352079 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f526659c700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5268100220 0x7f5268195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5265d9b700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5268101410 0x7f5268196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 -- 172.21.15.31:0/1940352079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5268101410 msgr2=0x7f5268196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5268101410 0x7f5268196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 -- 172.21.15.31:0/1940352079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5268100220 msgr2=0x7f5268195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5268100220 0x7f5268195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 -- 172.21.15.31:0/1940352079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52580095f0 con 0x7f5268101da0 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.813+0000 7f5266d9d700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 0x7f526819d5a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5260007ee0 tx=0x7f526000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.814+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260017070 con 0x7f5268101da0 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.814+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526000f040 con 0x7f5268101da0 2023-12-14T18:27:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.814+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526810bbe0 con 0x7f5268101da0 2023-12-14T18:27:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.814+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260005740 con 0x7f5268101da0 2023-12-14T18:27:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.814+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526810ba50 con 0x7f5268101da0 2023-12-14T18:27:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.815+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5268067e40 con 0x7f5268101da0 2023-12-14T18:27:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.816+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f526000dc60 con 0x7f5268101da0 2023-12-14T18:27:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.816+0000 7f525f7fe700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f52540a11e0 0x7f52540a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.817+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52600ccba0 con 0x7f5268101da0 2023-12-14T18:27:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.817+0000 7f526659c700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f52540a11e0 0x7f52540a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.817+0000 7f526659c700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f52540a11e0 0x7f52540a3690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5268101270 tx=0x7f5250007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:37.820+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5260095e80 con 0x7f5268101da0 2023-12-14T18:27:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:37 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/98856583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:37 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1312538718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:37.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:37 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/98856583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:37.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:37 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1312538718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:37 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/98856583' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:37 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1312538718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.027+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5268107100 con 0x7f5268101da0 2023-12-14T18:27:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.027+0000 7f525f7fe700 1 -- 172.21.15.31:0/1940352079 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52600b9030 con 0x7f5268101da0 2023-12-14T18:27:38.030 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.029+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f52540a11e0 msgr2=0x7f52540a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.029+0000 7f526c931700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f52540a11e0 0x7f52540a3690 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5268101270 tx=0x7f5250007400 comp rx=0 tx=0).stop 2023-12-14T18:27:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.029+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 msgr2=0x7f526819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.029+0000 7f526c931700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 0x7f526819d5a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5260007ee0 tx=0x7f526000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 shutdown_connections 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5268101da0 0x7f526819d5a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5268100220 0x7f5268195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f52540a11e0 0x7f52540a3690 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 --2- 172.21.15.31:0/1940352079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5268101410 0x7f5268196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 >> 172.21.15.31:0/1940352079 conn(0x7f52680fb840 msgr2=0x7f52680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 shutdown_connections 2023-12-14T18:27:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.030+0000 7f526c931700 1 -- 172.21.15.31:0/1940352079 wait complete. 2023-12-14T18:27:38.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 -- 172.21.15.31:0/3787334032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8101420 msgr2=0x7fa6b8101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/3787334032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8101420 0x7fa6b8101870 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009990 tx=0x7fa6a4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 -- 172.21.15.31:0/3787334032 shutdown_connections 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/3787334032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b8101db0 0x7fa6b8108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/3787334032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8100230 0x7fa6b8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/3787334032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8101420 0x7fa6b8101870 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 -- 172.21.15.31:0/3787334032 >> 172.21.15.31:0/3787334032 conn(0x7fa6b80fb850 msgr2=0x7fa6b80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 -- 172.21.15.31:0/3787334032 shutdown_connections 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 -- 172.21.15.31:0/3787334032 wait complete. 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.185+0000 7fa6bebc4700 1 Processor -- start 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 -- start start 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 0x7fa6b80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b8101420 0x7fa6b8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8101db0 0x7fa6b8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b8073bc0 con 0x7fa6b8100230 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b8073d30 con 0x7fa6b8101420 2023-12-14T18:27:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bebc4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a0010 con 0x7fa6b8101db0 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 0x7fa6b80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 0x7fa6b80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53056/0 (socket says 172.21.15.31:53056) 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 -- 172.21.15.31:0/2553634399 learned_addr learned my addr 172.21.15.31:0/2553634399 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bd161700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8101db0 0x7fa6b8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6b7fff700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b8101420 0x7fa6b8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 -- 172.21.15.31:0/2553634399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8101db0 msgr2=0x7fa6b8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8101db0 0x7fa6b8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 -- 172.21.15.31:0/2553634399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b8101420 msgr2=0x7fa6b8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b8101420 0x7fa6b8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 -- 172.21.15.31:0/2553634399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a40095f0 con 0x7fa6b8100230 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6bc960700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 0x7fa6b80748a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00d980 tx=0x7fa6ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.186+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00de60 con 0x7fa6b8100230 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.187+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6ac0044e0 con 0x7fa6b8100230 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.187+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b810bb90 con 0x7fa6b8100230 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.187+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac003be0 con 0x7fa6b8100230 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.187+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b810ba00 con 0x7fa6b8100230 2023-12-14T18:27:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.188+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b81a0760 con 0x7fa6b8100230 2023-12-14T18:27:38.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.189+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa6ac009bb0 con 0x7fa6b8100230 2023-12-14T18:27:38.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.189+0000 7fa6b5ffb700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa6a80a11e0 0x7fa6a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.190+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa6ac0ccb60 con 0x7fa6b8100230 2023-12-14T18:27:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.190+0000 7fa6b7fff700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa6a80a11e0 0x7fa6a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.190+0000 7fa6b7fff700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa6a80a11e0 0x7fa6a80a3690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4006010 tx=0x7fa6a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.194+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa6ac095e40 con 0x7fa6b8100230 2023-12-14T18:27:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.403+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6b8107130 con 0x7fa6b8100230 2023-12-14T18:27:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.403+0000 7fa6b5ffb700 1 -- 172.21.15.31:0/2553634399 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6ac009940 con 0x7fa6b8100230 2023-12-14T18:27:38.406 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa6a80a11e0 msgr2=0x7fa6a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa6a80a11e0 0x7fa6a80a3690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4006010 tx=0x7fa6a400b540 comp rx=0 tx=0).stop 2023-12-14T18:27:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 msgr2=0x7fa6b80748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 0x7fa6b80748a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00d980 tx=0x7fa6ac00dd40 comp rx=0 tx=0).stop 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 shutdown_connections 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fa6a80a11e0 0x7fa6a80a3690 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6b8101db0 0x7fa6b8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6b8100230 0x7fa6b80748a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 --2- 172.21.15.31:0/2553634399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6b8101420 0x7fa6b8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 >> 172.21.15.31:0/2553634399 conn(0x7fa6b80fb850 msgr2=0x7fa6b80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 shutdown_connections 2023-12-14T18:27:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.406+0000 7fa6bebc4700 1 -- 172.21.15.31:0/2553634399 wait complete. 2023-12-14T18:27:38.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 -- 172.21.15.31:0/2863002619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 msgr2=0x7f898c1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 --2- 172.21.15.31:0/2863002619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c1018f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8980009990 tx=0x7f8980009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 -- 172.21.15.31:0/2863002619 shutdown_connections 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 --2- 172.21.15.31:0/2863002619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c101e30 0x7f898c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 --2- 172.21.15.31:0/2863002619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f898c0fe4d0 0x7f898c0fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 --2- 172.21.15.31:0/2863002619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c1018f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.560+0000 7f8991242700 1 -- 172.21.15.31:0/2863002619 >> 172.21.15.31:0/2863002619 conn(0x7f898c0f9ed0 msgr2=0x7f898c0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:38.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.561+0000 7f8991242700 1 -- 172.21.15.31:0/2863002619 shutdown_connections 2023-12-14T18:27:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.561+0000 7f8991242700 1 -- 172.21.15.31:0/2863002619 wait complete. 2023-12-14T18:27:38.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 Processor -- start 2023-12-14T18:27:38.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 -- start start 2023-12-14T18:27:38.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 0x7f898c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f898c101e30 0x7f898c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19fcb0 con 0x7f898c1014a0 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19fdf0 con 0x7f898c101e30 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f8991242700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c19ff80 con 0x7f898c0fe4d0 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f898a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f898ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 0x7f898c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f898ad9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 0x7f898c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:57714/0 (socket says 172.21.15.31:57714) 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f898a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53086/0 (socket says 172.21.15.31:53086) 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f898ad9d700 1 -- 172.21.15.31:0/642236349 learned_addr learned my addr 172.21.15.31:0/642236349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.562+0000 7f898b59e700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f898c101e30 0x7f898c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898ad9d700 1 -- 172.21.15.31:0/642236349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f898c101e30 msgr2=0x7f898c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898ad9d700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f898c101e30 0x7f898c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898ad9d700 1 -- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 msgr2=0x7f898c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898ad9d700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898ad9d700 1 -- 172.21.15.31:0/642236349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800095f0 con 0x7f898c0fe4d0 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898a59c700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.563+0000 7f898ad9d700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 0x7f898c198130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c990 tx=0x7f897c00cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.564+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c007940 con 0x7f898c0fe4d0 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.564+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f897c00f450 con 0x7f898c0fe4d0 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.564+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c10bb70 con 0x7f898c0fe4d0 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.564+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c018660 con 0x7f898c0fe4d0 2023-12-14T18:27:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.564+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c10b9e0 con 0x7f898c0fe4d0 2023-12-14T18:27:38.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.565+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c1a0720 con 0x7f898c0fe4d0 2023-12-14T18:27:38.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.566+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f897c007aa0 con 0x7f898c0fe4d0 2023-12-14T18:27:38.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.566+0000 7f8973fff700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89740a1260 0x7f89740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.567+0000 7f898a59c700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89740a1260 0x7f89740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.567+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f897c0ccc10 con 0x7f898c0fe4d0 2023-12-14T18:27:38.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.567+0000 7f898a59c700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89740a1260 0x7f89740a3710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f898000b5c0 tx=0x7f8980005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:38.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.571+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f897c095ef0 con 0x7f898c0fe4d0 2023-12-14T18:27:38.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:38 smithi031 ceph-mon[42085]: pgmap v282: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:38.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1326996745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:38.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1940352079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:38.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:38 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2553634399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.780+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f898c1a0cd0 con 0x7f898c0fe4d0 2023-12-14T18:27:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.781+0000 7f8973fff700 1 -- 172.21.15.31:0/642236349 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f897c099510 con 0x7f898c0fe4d0 2023-12-14T18:27:38.783 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89740a1260 msgr2=0x7f89740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89740a1260 0x7f89740a3710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f898000b5c0 tx=0x7f8980005960 comp rx=0 tx=0).stop 2023-12-14T18:27:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 msgr2=0x7f898c198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 0x7f898c198130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f897c00c990 tx=0x7f897c00cca0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 shutdown_connections 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f898c101e30 0x7f898c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f89740a1260 0x7f89740a3710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f898c0fe4d0 0x7f898c198130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 --2- 172.21.15.31:0/642236349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f898c1014a0 0x7f898c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.783+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 >> 172.21.15.31:0/642236349 conn(0x7f898c0f9ed0 msgr2=0x7f898c0fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.784+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 shutdown_connections 2023-12-14T18:27:38.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.784+0000 7f8991242700 1 -- 172.21.15.31:0/642236349 wait complete. 2023-12-14T18:27:38.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:38 smithi204 ceph-mon[43866]: pgmap v282: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:38 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1326996745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:38 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1940352079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:38.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:38 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2553634399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:38.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 -- 172.21.15.31:0/213633511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 msgr2=0x7f8e80100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 --2- 172.21.15.31:0/213633511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80100760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009990 tx=0x7f8e74009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 -- 172.21.15.31:0/213633511 shutdown_connections 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 --2- 172.21.15.31:0/213633511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e80100ca0 0x7f8e80107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 --2- 172.21.15.31:0/213633511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e800ff120 0x7f8e800ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 --2- 172.21.15.31:0/213633511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80100760 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 -- 172.21.15.31:0/213633511 >> 172.21.15.31:0/213633511 conn(0x7f8e800fa740 msgr2=0x7f8e800fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 -- 172.21.15.31:0/213633511 shutdown_connections 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 -- 172.21.15.31:0/213633511 wait complete. 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.941+0000 7f8e860ac700 1 Processor -- start 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 -- start start 2023-12-14T18:27:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e800ff120 0x7f8e80109a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80108050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e80100ca0 0x7f8e80108590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80108d30 con 0x7f8e80100310 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80108ea0 con 0x7f8e800ff120 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e860ac700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801a0050 con 0x7f8e80100ca0 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53100/0 (socket says 172.21.15.31:53100) 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 -- 172.21.15.31:0/188853988 learned_addr learned my addr 172.21.15.31:0/188853988 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7f7fe700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e800ff120 0x7f8e80109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7ffff700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e80100ca0 0x7f8e80108590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 -- 172.21.15.31:0/188853988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e80100ca0 msgr2=0x7f8e80108590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e80100ca0 0x7f8e80108590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 -- 172.21.15.31:0/188853988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e800ff120 msgr2=0x7f8e80109a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e800ff120 0x7f8e80109a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.942+0000 7f8e7effd700 1 -- 172.21.15.31:0/188853988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e740095f0 con 0x7f8e80100310 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.943+0000 7f8e7effd700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80108050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400c010 tx=0x7f8e74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.943+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7401d070 con 0x7f8e80100310 2023-12-14T18:27:38.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.943+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8010bc50 con 0x7f8e80100310 2023-12-14T18:27:38.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.943+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e74005030 con 0x7f8e80100310 2023-12-14T18:27:38.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.943+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74004b90 con 0x7f8e80100310 2023-12-14T18:27:38.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.943+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8010bac0 con 0x7f8e80100310 2023-12-14T18:27:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.944+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e801066b0 con 0x7f8e80100310 2023-12-14T18:27:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.945+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8e74004d70 con 0x7f8e80100310 2023-12-14T18:27:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.946+0000 7f8e7cff9700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e680a11e0 0x7f8e680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.946+0000 7f8e7f7fe700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e680a11e0 0x7f8e680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.946+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8e740ce090 con 0x7f8e80100310 2023-12-14T18:27:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.947+0000 7f8e7f7fe700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e680a11e0 0x7f8e680a3690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e80100170 tx=0x7f8e70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:38.951+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8e7405a730 con 0x7f8e80100310 2023-12-14T18:27:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:38 smithi176 ceph-mon[43506]: pgmap v282: 129 pgs: 129 active+clean; 639 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-12-14T18:27:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:38 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1326996745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:38 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1940352079' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:38.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:38 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2553634399' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.159+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8e80061700 con 0x7f8e80100310 2023-12-14T18:27:39.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.160+0000 7f8e7cff9700 1 -- 172.21.15.31:0/188853988 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8e7409ae30 con 0x7f8e80100310 2023-12-14T18:27:39.162 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:39.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e680a11e0 msgr2=0x7f8e680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e680a11e0 0x7f8e680a3690 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e80100170 tx=0x7f8e70009260 comp rx=0 tx=0).stop 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 msgr2=0x7f8e80108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80108050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400c010 tx=0x7f8e74004730 comp rx=0 tx=0).stop 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 shutdown_connections 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f8e680a11e0 0x7f8e680a3690 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e80100ca0 0x7f8e80108590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8e800ff120 0x7f8e80109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 --2- 172.21.15.31:0/188853988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e80100310 0x7f8e80108050 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 >> 172.21.15.31:0/188853988 conn(0x7f8e800fa740 msgr2=0x7f8e801037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:39.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 shutdown_connections 2023-12-14T18:27:39.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.162+0000 7f8e860ac700 1 -- 172.21.15.31:0/188853988 wait complete. 2023-12-14T18:27:39.179 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-12-14T18:27:39.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:39.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 -- 172.21.15.31:0/1753033728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100ca0 msgr2=0x7f3284107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 --2- 172.21.15.31:0/1753033728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100ca0 0x7f3284107560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f327c00b0a0 tx=0x7f327c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 -- 172.21.15.31:0/1753033728 shutdown_connections 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 --2- 172.21.15.31:0/1753033728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100ca0 0x7f3284107560 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 --2- 172.21.15.31:0/1753033728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32840ff120 0x7f32840ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 --2- 172.21.15.31:0/1753033728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3284100310 0x7f3284100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.315+0000 7f328b976700 1 -- 172.21.15.31:0/1753033728 >> 172.21.15.31:0/1753033728 conn(0x7f32840fa720 msgr2=0x7f32840fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.316+0000 7f328b976700 1 -- 172.21.15.31:0/1753033728 shutdown_connections 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.316+0000 7f328b976700 1 -- 172.21.15.31:0/1753033728 wait complete. 2023-12-14T18:27:39.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.316+0000 7f328b976700 1 Processor -- start 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.316+0000 7f328b976700 1 -- start start 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f328b976700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 0x7f3284069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f328b976700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100310 0x7f328419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f328b976700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3284100ca0 0x7f328419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f328b976700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328419dba0 con 0x7f3284100310 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f328b976700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328419dd10 con 0x7f3284100ca0 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f328b976700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328419ded0 con 0x7f32840ff120 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 0x7f3284069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 0x7f3284069ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:57734/0 (socket says 172.21.15.31:57734) 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 -- 172.21.15.31:0/2761074991 learned_addr learned my addr 172.21.15.31:0/2761074991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:39.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289f13700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3284100ca0 0x7f328419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 -- 172.21.15.31:0/2761074991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3284100ca0 msgr2=0x7f328419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3284100ca0 0x7f328419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 -- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100310 msgr2=0x7f328419adc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100310 0x7f328419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.317+0000 7f3289712700 1 -- 172.21.15.31:0/2761074991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c009d00 con 0x7f32840ff120 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.318+0000 7f3289712700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 0x7f3284069ba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f328000d980 tx=0x7f328000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.318+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000b950 con 0x7f32840ff120 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.318+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3280007620 con 0x7f32840ff120 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.318+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328410aa90 con 0x7f32840ff120 2023-12-14T18:27:39.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.318+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000f3c0 con 0x7f32840ff120 2023-12-14T18:27:39.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.318+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328410a900 con 0x7f32840ff120 2023-12-14T18:27:39.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.319+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284105fd0 con 0x7f32840ff120 2023-12-14T18:27:39.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.321+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3280010640 con 0x7f32840ff120 2023-12-14T18:27:39.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.322+0000 7f327a7fc700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f32700a1260 0x7f32700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.322+0000 7f3288f11700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f32700a1260 0x7f32700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.322+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32800cd560 con 0x7f32840ff120 2023-12-14T18:27:39.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.322+0000 7f3288f11700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f32700a1260 0x7f32700a3710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3274009b60 tx=0x7f3274009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:39.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.325+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32800967e0 con 0x7f32840ff120 2023-12-14T18:27:39.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.586+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32840619b0 con 0x7f32840ff120 2023-12-14T18:27:39.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.587+0000 7f327a7fc700 1 -- 172.21.15.31:0/2761074991 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f328000ee60 con 0x7f32840ff120 2023-12-14T18:27:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:39.590 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f32700a1260 msgr2=0x7f32700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f32700a1260 0x7f32700a3710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3274009b60 tx=0x7f3274009620 comp rx=0 tx=0).stop 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 msgr2=0x7f3284069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 0x7f3284069ba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f328000d980 tx=0x7f328000dc90 comp rx=0 tx=0).stop 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 shutdown_connections 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3284100ca0 0x7f328419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f32700a1260 0x7f32700a3710 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32840ff120 0x7f3284069ba0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 --2- 172.21.15.31:0/2761074991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3284100310 0x7f328419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 >> 172.21.15.31:0/2761074991 conn(0x7f32840fa720 msgr2=0x7f32840fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 shutdown_connections 2023-12-14T18:27:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.590+0000 7f328b976700 1 -- 172.21.15.31:0/2761074991 wait complete. 2023-12-14T18:27:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:39.607 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-12-14T18:27:39.607 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-12-14T18:27:39.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:27:39.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 -- 172.21.15.31:0/3204138126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5524100c90 msgr2=0x7f5524103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3204138126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5524100c90 0x7f5524103070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5514009990 tx=0x7f5514009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 -- 172.21.15.31:0/3204138126 shutdown_connections 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3204138126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5524105ed0 0x7f5524108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3204138126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5524100c90 0x7f5524103070 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3204138126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55241035b0 0x7f5524105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 -- 172.21.15.31:0/3204138126 >> 172.21.15.31:0/3204138126 conn(0x7f55240fa6f0 msgr2=0x7f55240fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 -- 172.21.15.31:0/3204138126 shutdown_connections 2023-12-14T18:27:39.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.743+0000 7f5529b4e700 1 -- 172.21.15.31:0/3204138126 wait complete. 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.744+0000 7f5529b4e700 1 Processor -- start 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.744+0000 7f5529b4e700 1 -- start start 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.744+0000 7f5529b4e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5524100c90 0x7f5524193dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.744+0000 7f5529b4e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 0x7f5524194310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.744+0000 7f5529b4e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5524105ed0 0x7f552419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.744+0000 7f5529b4e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552419b950 con 0x7f55241035b0 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5529b4e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552419ba90 con 0x7f5524100c90 2023-12-14T18:27:39.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5529b4e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552419bc20 con 0x7f5524105ed0 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 0x7f5524194310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 0x7f5524194310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53126/0 (socket says 172.21.15.31:53126) 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 -- 172.21.15.31:0/3740833694 learned_addr learned my addr 172.21.15.31:0/3740833694 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5523fff700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5524105ed0 0x7f552419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f55237fe700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5524100c90 0x7f5524193dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 -- 172.21.15.31:0/3740833694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5524105ed0 msgr2=0x7f552419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5524105ed0 0x7f552419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 -- 172.21.15.31:0/3740833694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5524100c90 msgr2=0x7f5524193dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5524100c90 0x7f5524193dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 -- 172.21.15.31:0/3740833694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140095f0 con 0x7f55241035b0 2023-12-14T18:27:39.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5522ffd700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 0x7f5524194310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f550c00d950 tx=0x7f550c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:39.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00f890 con 0x7f55241035b0 2023-12-14T18:27:39.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f550c010460 con 0x7f55241035b0 2023-12-14T18:27:39.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55241099c0 con 0x7f55241035b0 2023-12-14T18:27:39.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00e610 con 0x7f55241035b0 2023-12-14T18:27:39.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.745+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524109830 con 0x7f55241035b0 2023-12-14T18:27:39.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.746+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240fc260 con 0x7f55241035b0 2023-12-14T18:27:39.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.748+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f550c0108d0 con 0x7f55241035b0 2023-12-14T18:27:39.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.748+0000 7f5520ff9700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f55100a55f0 0x7f55100a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:39.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.748+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f550c0ccb60 con 0x7f55241035b0 2023-12-14T18:27:39.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.748+0000 7f55237fe700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f55100a55f0 0x7f55100a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:39.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.749+0000 7f55237fe700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f55100a55f0 0x7f55100a7aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f551400b5c0 tx=0x7f5514005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:39.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.753+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f550c095e40 con 0x7f55241035b0 2023-12-14T18:27:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:39 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/642236349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:39 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/188853988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.959+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5524061ab0 con 0x7f55100a55f0 2023-12-14T18:27:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:39 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/642236349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:39 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/188853988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.973+0000 7f5520ff9700 1 -- 172.21.15.31:0/3740833694 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427137 (secure 0 0 0) 0x7f5524061ab0 con 0x7f55100a55f0 2023-12-14T18:27:39.982 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:39.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f55100a55f0 msgr2=0x7f55100a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f55100a55f0 0x7f55100a7aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f551400b5c0 tx=0x7f5514005fd0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 msgr2=0x7f5524194310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:39.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 0x7f5524194310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f550c00d950 tx=0x7f550c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:27:39.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 shutdown_connections 2023-12-14T18:27:39.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f55100a55f0 0x7f55100a7aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5524105ed0 0x7f552419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5524100c90 0x7f5524193dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 --2- 172.21.15.31:0/3740833694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55241035b0 0x7f5524194310 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 >> 172.21.15.31:0/3740833694 conn(0x7f55240fa6f0 msgr2=0x7f5524101910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 shutdown_connections 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:39.977+0000 7f5529b4e700 1 -- 172.21.15.31:0/3740833694 wait complete. 2023-12-14T18:27:39.989 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:27:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:39 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/642236349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:39 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/188853988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:39.993 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-12-14T18:27:39.499954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658965,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":605,"num_read_kb":424,"num_write":343,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1182282,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80804,"kb_used_data":11652,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124861020,"statfs":{"total":1151940427776,"available":1151857684480,"internally_reserved":0,"allocated":11931648,"data_stored":7750211,"data_compressed":319020,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4640,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":41,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003489"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562794+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:48.562794+0000","last_peered":"2023-12-14T18:26:48.562794+0000","last_clean":"2023-12-14T18:26:48.562794+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:48.562794+0000","last_undegraded":"2023-12-14T18:26:48.562794+0000","last_fullsized":"2023-12-14T18:26:48.562794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:33:46.076524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557880+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:48.557880+0000","last_peered":"2023-12-14T18:26:48.557880+0000","last_clean":"2023-12-14T18:26:48.557880+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:48.557880+0000","last_undegraded":"2023-12-14T18:26:48.557880+0000","last_fullsized":"2023-12-14T18:26:48.557880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:10.369082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557355+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:48.557355+0000","last_peered":"2023-12-14T18:26:48.557355+0000","last_clean":"2023-12-14T18:26:48.557355+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:48.557355+0000","last_undegraded":"2023-12-14T18:26:48.557355+0000","last_fullsized":"2023-12-14T18:26:48.557355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:23.331965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563177+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:48.563177+0000","last_peered":"2023-12-14T18:26:48.563177+0000","last_clean":"2023-12-14T18:26:48.563177+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:48.563177+0000","last_undegraded":"2023-12-14T18:26:48.563177+0000","last_fullsized":"2023-12-14T18:26:48.563177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:01.529418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567124+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:48.567124+0000","last_peered":"2023-12-14T18:26:48.567124+0000","last_clean":"2023-12-14T18:26:48.567124+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:48.567124+0000","last_undegraded":"2023-12-14T18:26:48.567124+0000","last_fullsized":"2023-12-14T18:26:48.567124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:53:32.988966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647501+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:26:48.647501+0000","last_peered":"2023-12-14T18:26:48.647501+0000","last_clean":"2023-12-14T18:26:48.647501+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:26:48.647501+0000","last_undegraded":"2023-12-14T18:26:48.647501+0000","last_fullsized":"2023-12-14T18:26:48.647501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:00:59.230921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561700+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:48.561700+0000","last_peered":"2023-12-14T18:26:48.561700+0000","last_clean":"2023-12-14T18:26:48.561700+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:48.561700+0000","last_undegraded":"2023-12-14T18:26:48.561700+0000","last_fullsized":"2023-12-14T18:26:48.561700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:47.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559287+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:48.559287+0000","last_peered":"2023-12-14T18:26:48.559287+0000","last_clean":"2023-12-14T18:26:48.559287+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:48.559287+0000","last_undegraded":"2023-12-14T18:26:48.559287+0000","last_fullsized":"2023-12-14T18:26:48.559287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:13:56.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560906+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:48.560906+0000","last_peered":"2023-12-14T18:26:48.560906+0000","last_clean":"2023-12-14T18:26:48.560906+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:48.560906+0000","last_undegraded":"2023-12-14T18:26:48.560906+0000","last_fullsized":"2023-12-14T18:26:48.560906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:38:42.914156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647960+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:26:48.647960+0000","last_peered":"2023-12-14T18:26:48.647960+0000","last_clean":"2023-12-14T18:26:48.647960+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:26:48.647960+0000","last_undegraded":"2023-12-14T18:26:48.647960+0000","last_fullsized":"2023-12-14T18:26:48.647960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:09:41.346067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562267+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:48.562267+0000","last_peered":"2023-12-14T18:26:48.562267+0000","last_clean":"2023-12-14T18:26:48.562267+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:48.562267+0000","last_undegraded":"2023-12-14T18:26:48.562267+0000","last_fullsized":"2023-12-14T18:26:48.562267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:26.374795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563022+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:48.563022+0000","last_peered":"2023-12-14T18:26:48.563022+0000","last_clean":"2023-12-14T18:26:48.563022+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:48.563022+0000","last_undegraded":"2023-12-14T18:26:48.563022+0000","last_fullsized":"2023-12-14T18:26:48.563022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:38.171386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563106+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:48.563106+0000","last_peered":"2023-12-14T18:26:48.563106+0000","last_clean":"2023-12-14T18:26:48.563106+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:48.563106+0000","last_undegraded":"2023-12-14T18:26:48.563106+0000","last_fullsized":"2023-12-14T18:26:48.563106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:18:44.617868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557419+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:48.557419+0000","last_peered":"2023-12-14T18:26:48.557419+0000","last_clean":"2023-12-14T18:26:48.557419+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:48.557419+0000","last_undegraded":"2023-12-14T18:26:48.557419+0000","last_fullsized":"2023-12-14T18:26:48.557419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:04:10.141803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558264+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:48.558264+0000","last_peered":"2023-12-14T18:26:48.558264+0000","last_clean":"2023-12-14T18:26:48.558264+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:48.558264+0000","last_undegraded":"2023-12-14T18:26:48.558264+0000","last_fullsized":"2023-12-14T18:26:48.558264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:53.022658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559788+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:48.559788+0000","last_peered":"2023-12-14T18:26:48.559788+0000","last_clean":"2023-12-14T18:26:48.559788+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:48.559788+0000","last_undegraded":"2023-12-14T18:26:48.559788+0000","last_fullsized":"2023-12-14T18:26:48.559788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:18.996023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562659+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:48.562659+0000","last_peered":"2023-12-14T18:26:48.562659+0000","last_clean":"2023-12-14T18:26:48.562659+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:48.562659+0000","last_undegraded":"2023-12-14T18:26:48.562659+0000","last_fullsized":"2023-12-14T18:26:48.562659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:00:27.549342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562608+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:26:48.562608+0000","last_peered":"2023-12-14T18:26:48.562608+0000","last_clean":"2023-12-14T18:26:48.562608+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:26:48.562608+0000","last_undegraded":"2023-12-14T18:26:48.562608+0000","last_fullsized":"2023-12-14T18:26:48.562608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:06:20.688646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566623+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:26:48.566623+0000","last_peered":"2023-12-14T18:26:48.566623+0000","last_clean":"2023-12-14T18:26:48.566623+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:26:48.566623+0000","last_undegraded":"2023-12-14T18:26:48.566623+0000","last_fullsized":"2023-12-14T18:26:48.566623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:58.296365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564244+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:26:48.564244+0000","last_peered":"2023-12-14T18:26:48.564244+0000","last_clean":"2023-12-14T18:26:48.564244+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:26:48.564244+0000","last_undegraded":"2023-12-14T18:26:48.564244+0000","last_fullsized":"2023-12-14T18:26:48.564244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:23.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563082+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:26:48.563082+0000","last_peered":"2023-12-14T18:26:48.563082+0000","last_clean":"2023-12-14T18:26:48.563082+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:26:48.563082+0000","last_undegraded":"2023-12-14T18:26:48.563082+0000","last_fullsized":"2023-12-14T18:26:48.563082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:35:18.652626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559422+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:48.559422+0000","last_peered":"2023-12-14T18:26:48.559422+0000","last_clean":"2023-12-14T18:26:48.559422+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:48.559422+0000","last_undegraded":"2023-12-14T18:26:48.559422+0000","last_fullsized":"2023-12-14T18:26:48.559422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:31.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562609+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:48.562609+0000","last_peered":"2023-12-14T18:26:48.562609+0000","last_clean":"2023-12-14T18:26:48.562609+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:48.562609+0000","last_undegraded":"2023-12-14T18:26:48.562609+0000","last_fullsized":"2023-12-14T18:26:48.562609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:43.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559749+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:48.559749+0000","last_peered":"2023-12-14T18:26:48.559749+0000","last_clean":"2023-12-14T18:26:48.559749+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:48.559749+0000","last_undegraded":"2023-12-14T18:26:48.559749+0000","last_fullsized":"2023-12-14T18:26:48.559749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:43.650498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558888+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:48.558888+0000","last_peered":"2023-12-14T18:26:48.558888+0000","last_clean":"2023-12-14T18:26:48.558888+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:48.558888+0000","last_undegraded":"2023-12-14T18:26:48.558888+0000","last_fullsized":"2023-12-14T18:26:48.558888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:50.284685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564072+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:48.564072+0000","last_peered":"2023-12-14T18:26:48.564072+0000","last_clean":"2023-12-14T18:26:48.564072+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:48.564072+0000","last_undegraded":"2023-12-14T18:26:48.564072+0000","last_fullsized":"2023-12-14T18:26:48.564072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:29:21.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568778+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:48.568778+0000","last_peered":"2023-12-14T18:26:48.568778+0000","last_clean":"2023-12-14T18:26:48.568778+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:48.568778+0000","last_undegraded":"2023-12-14T18:26:48.568778+0000","last_fullsized":"2023-12-14T18:26:48.568778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:38.658671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560627+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:48.560627+0000","last_peered":"2023-12-14T18:26:48.560627+0000","last_clean":"2023-12-14T18:26:48.560627+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:48.560627+0000","last_undegraded":"2023-12-14T18:26:48.560627+0000","last_fullsized":"2023-12-14T18:26:48.560627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:54:13.436594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562615+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:48.562615+0000","last_peered":"2023-12-14T18:26:48.562615+0000","last_clean":"2023-12-14T18:26:48.562615+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:48.562615+0000","last_undegraded":"2023-12-14T18:26:48.562615+0000","last_fullsized":"2023-12-14T18:26:48.562615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:22:13.915160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568436+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:48.568436+0000","last_peered":"2023-12-14T18:26:48.568436+0000","last_clean":"2023-12-14T18:26:48.568436+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:48.568436+0000","last_undegraded":"2023-12-14T18:26:48.568436+0000","last_fullsized":"2023-12-14T18:26:48.568436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:51:57.429285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563999+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:48.563999+0000","last_peered":"2023-12-14T18:26:48.563999+0000","last_clean":"2023-12-14T18:26:48.563999+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:48.563999+0000","last_undegraded":"2023-12-14T18:26:48.563999+0000","last_fullsized":"2023-12-14T18:26:48.563999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:21:53.454334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564084+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:26:48.564084+0000","last_peered":"2023-12-14T18:26:48.564084+0000","last_clean":"2023-12-14T18:26:48.564084+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:26:48.564084+0000","last_undegraded":"2023-12-14T18:26:48.564084+0000","last_fullsized":"2023-12-14T18:26:48.564084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:33.154296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:35.636522+0000","last_change":"2023-12-14T18:26:48.574808+0000","last_active":"2023-12-14T18:27:35.636522+0000","last_peered":"2023-12-14T18:27:35.636522+0000","last_clean":"2023-12-14T18:27:35.636522+0000","last_became_active":"2023-12-14T18:26:48.568640+0000","last_became_peered":"2023-12-14T18:26:48.568640+0000","last_unstale":"2023-12-14T18:27:35.636522+0000","last_undegraded":"2023-12-14T18:27:35.636522+0000","last_fullsized":"2023-12-14T18:27:35.636522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:16.685666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566784+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:48.566784+0000","last_peered":"2023-12-14T18:26:48.566784+0000","last_clean":"2023-12-14T18:26:48.566784+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:48.566784+0000","last_undegraded":"2023-12-14T18:26:48.566784+0000","last_fullsized":"2023-12-14T18:26:48.566784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.326220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.573798+0000","last_change":"2023-12-14T18:26:48.573798+0000","last_active":"2023-12-14T18:26:48.573798+0000","last_peered":"2023-12-14T18:26:48.573798+0000","last_clean":"2023-12-14T18:26:48.573798+0000","last_became_active":"2023-12-14T18:26:48.573308+0000","last_became_peered":"2023-12-14T18:26:48.573308+0000","last_unstale":"2023-12-14T18:26:48.573798+0000","last_undegraded":"2023-12-14T18:26:48.573798+0000","last_fullsized":"2023-12-14T18:26:48.573798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:28:46.273622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647410+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:48.647410+0000","last_peered":"2023-12-14T18:26:48.647410+0000","last_clean":"2023-12-14T18:26:48.647410+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:48.647410+0000","last_undegraded":"2023-12-14T18:26:48.647410+0000","last_fullsized":"2023-12-14T18:26:48.647410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:05:15.020350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564341+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:48.564341+0000","last_peered":"2023-12-14T18:26:48.564341+0000","last_clean":"2023-12-14T18:26:48.564341+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:48.564341+0000","last_undegraded":"2023-12-14T18:26:48.564341+0000","last_fullsized":"2023-12-14T18:26:48.564341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:43.990428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570306+0000","last_change":"2023-12-14T18:26:48.570306+0000","last_active":"2023-12-14T18:26:48.570306+0000","last_peered":"2023-12-14T18:26:48.570306+0000","last_clean":"2023-12-14T18:26:48.570306+0000","last_became_active":"2023-12-14T18:26:48.569803+0000","last_became_peered":"2023-12-14T18:26:48.569803+0000","last_unstale":"2023-12-14T18:26:48.570306+0000","last_undegraded":"2023-12-14T18:26:48.570306+0000","last_fullsized":"2023-12-14T18:26:48.570306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:33.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562436+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:48.562436+0000","last_peered":"2023-12-14T18:26:48.562436+0000","last_clean":"2023-12-14T18:26:48.562436+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:48.562436+0000","last_undegraded":"2023-12-14T18:26:48.562436+0000","last_fullsized":"2023-12-14T18:26:48.562436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:53.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559154+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:48.559154+0000","last_peered":"2023-12-14T18:26:48.559154+0000","last_clean":"2023-12-14T18:26:48.559154+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:48.559154+0000","last_undegraded":"2023-12-14T18:26:48.559154+0000","last_fullsized":"2023-12-14T18:26:48.559154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:49.740844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557724+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:48.557724+0000","last_peered":"2023-12-14T18:26:48.557724+0000","last_clean":"2023-12-14T18:26:48.557724+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:48.557724+0000","last_undegraded":"2023-12-14T18:26:48.557724+0000","last_fullsized":"2023-12-14T18:26:48.557724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:44:17.333118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567219+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:48.567219+0000","last_peered":"2023-12-14T18:26:48.567219+0000","last_clean":"2023-12-14T18:26:48.567219+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:48.567219+0000","last_undegraded":"2023-12-14T18:26:48.567219+0000","last_fullsized":"2023-12-14T18:26:48.567219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:45:09.792477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561943+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:26:48.561943+0000","last_peered":"2023-12-14T18:26:48.561943+0000","last_clean":"2023-12-14T18:26:48.561943+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:26:48.561943+0000","last_undegraded":"2023-12-14T18:26:48.561943+0000","last_fullsized":"2023-12-14T18:26:48.561943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:21:43.141339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559411+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:48.559411+0000","last_peered":"2023-12-14T18:26:48.559411+0000","last_clean":"2023-12-14T18:26:48.559411+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:48.559411+0000","last_undegraded":"2023-12-14T18:26:48.559411+0000","last_fullsized":"2023-12-14T18:26:48.559411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:36.307493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:37.638765+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:27:37.638765+0000","last_peered":"2023-12-14T18:27:37.638765+0000","last_clean":"2023-12-14T18:27:37.638765+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:27:37.638765+0000","last_undegraded":"2023-12-14T18:27:37.638765+0000","last_fullsized":"2023-12-14T18:27:37.638765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:20:23.673760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560703+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:48.560703+0000","last_peered":"2023-12-14T18:26:48.560703+0000","last_clean":"2023-12-14T18:26:48.560703+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:48.560703+0000","last_undegraded":"2023-12-14T18:26:48.560703+0000","last_fullsized":"2023-12-14T18:26:48.560703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:33:22.836099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561628+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:26:48.561628+0000","last_peered":"2023-12-14T18:26:48.561628+0000","last_clean":"2023-12-14T18:26:48.561628+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:26:48.561628+0000","last_undegraded":"2023-12-14T18:26:48.561628+0000","last_fullsized":"2023-12-14T18:26:48.561628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:47.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561906+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:48.561906+0000","last_peered":"2023-12-14T18:26:48.561906+0000","last_clean":"2023-12-14T18:26:48.561906+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:48.561906+0000","last_undegraded":"2023-12-14T18:26:48.561906+0000","last_fullsized":"2023-12-14T18:26:48.561906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:00.359853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558750+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:48.558750+0000","last_peered":"2023-12-14T18:26:48.558750+0000","last_clean":"2023-12-14T18:26:48.558750+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:48.558750+0000","last_undegraded":"2023-12-14T18:26:48.558750+0000","last_fullsized":"2023-12-14T18:26:48.558750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:49.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568565+0000","last_change":"2023-12-14T18:26:48.568565+0000","last_active":"2023-12-14T18:26:48.568565+0000","last_peered":"2023-12-14T18:26:48.568565+0000","last_clean":"2023-12-14T18:26:48.568565+0000","last_became_active":"2023-12-14T18:26:48.568090+0000","last_became_peered":"2023-12-14T18:26:48.568090+0000","last_unstale":"2023-12-14T18:26:48.568565+0000","last_undegraded":"2023-12-14T18:26:48.568565+0000","last_fullsized":"2023-12-14T18:26:48.568565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:14:38.160855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563856+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:48.563856+0000","last_peered":"2023-12-14T18:26:48.563856+0000","last_clean":"2023-12-14T18:26:48.563856+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:48.563856+0000","last_undegraded":"2023-12-14T18:26:48.563856+0000","last_fullsized":"2023-12-14T18:26:48.563856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:12.189176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560853+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:48.560853+0000","last_peered":"2023-12-14T18:26:48.560853+0000","last_clean":"2023-12-14T18:26:48.560853+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:48.560853+0000","last_undegraded":"2023-12-14T18:26:48.560853+0000","last_fullsized":"2023-12-14T18:26:48.560853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:55:18.680896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562308+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:26:48.562308+0000","last_peered":"2023-12-14T18:26:48.562308+0000","last_clean":"2023-12-14T18:26:48.562308+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:26:48.562308+0000","last_undegraded":"2023-12-14T18:26:48.562308+0000","last_fullsized":"2023-12-14T18:26:48.562308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:05:51.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":223,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:38.641353+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:27:38.641353+0000","last_peered":"2023-12-14T18:27:38.641353+0000","last_clean":"2023-12-14T18:27:38.641353+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:27:38.641353+0000","last_undegraded":"2023-12-14T18:27:38.641353+0000","last_fullsized":"2023-12-14T18:27:38.641353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:33.081169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64447,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":139,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568542+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:48.568542+0000","last_peered":"2023-12-14T18:26:48.568542+0000","last_clean":"2023-12-14T18:26:48.568542+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:48.568542+0000","last_undegraded":"2023-12-14T18:26:48.568542+0000","last_fullsized":"2023-12-14T18:26:48.568542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:30.839862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564457+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:48.564457+0000","last_peered":"2023-12-14T18:26:48.564457+0000","last_clean":"2023-12-14T18:26:48.564457+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:48.564457+0000","last_undegraded":"2023-12-14T18:26:48.564457+0000","last_fullsized":"2023-12-14T18:26:48.564457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:27.839169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565422+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:48.565422+0000","last_peered":"2023-12-14T18:26:48.565422+0000","last_clean":"2023-12-14T18:26:48.565422+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:48.565422+0000","last_undegraded":"2023-12-14T18:26:48.565422+0000","last_fullsized":"2023-12-14T18:26:48.565422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:34.931404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562476+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:48.562476+0000","last_peered":"2023-12-14T18:26:48.562476+0000","last_clean":"2023-12-14T18:26:48.562476+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:48.562476+0000","last_undegraded":"2023-12-14T18:26:48.562476+0000","last_fullsized":"2023-12-14T18:26:48.562476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:45.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561959+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:48.561959+0000","last_peered":"2023-12-14T18:26:48.561959+0000","last_clean":"2023-12-14T18:26:48.561959+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:48.561959+0000","last_undegraded":"2023-12-14T18:26:48.561959+0000","last_fullsized":"2023-12-14T18:26:48.561959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:54.542913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565425+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:48.565425+0000","last_peered":"2023-12-14T18:26:48.565425+0000","last_clean":"2023-12-14T18:26:48.565425+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:48.565425+0000","last_undegraded":"2023-12-14T18:26:48.565425+0000","last_fullsized":"2023-12-14T18:26:48.565425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:46.938078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566091+0000","last_change":"2023-12-14T18:26:48.566091+0000","last_active":"2023-12-14T18:26:48.566091+0000","last_peered":"2023-12-14T18:26:48.566091+0000","last_clean":"2023-12-14T18:26:48.566091+0000","last_became_active":"2023-12-14T18:26:48.565714+0000","last_became_peered":"2023-12-14T18:26:48.565714+0000","last_unstale":"2023-12-14T18:26:48.566091+0000","last_undegraded":"2023-12-14T18:26:48.566091+0000","last_fullsized":"2023-12-14T18:26:48.566091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:48.478859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562658+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:26:48.562658+0000","last_peered":"2023-12-14T18:26:48.562658+0000","last_clean":"2023-12-14T18:26:48.562658+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:26:48.562658+0000","last_undegraded":"2023-12-14T18:26:48.562658+0000","last_fullsized":"2023-12-14T18:26:48.562658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:22.379528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:32.714881+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:27:32.714881+0000","last_peered":"2023-12-14T18:27:32.714881+0000","last_clean":"2023-12-14T18:27:32.714881+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:27:32.714881+0000","last_undegraded":"2023-12-14T18:27:32.714881+0000","last_fullsized":"2023-12-14T18:27:32.714881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:26:13.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561690+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:48.561690+0000","last_peered":"2023-12-14T18:26:48.561690+0000","last_clean":"2023-12-14T18:26:48.561690+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:48.561690+0000","last_undegraded":"2023-12-14T18:26:48.561690+0000","last_fullsized":"2023-12-14T18:26:48.561690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:10:17.702084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.649376+0000","last_change":"2023-12-14T18:26:48.649376+0000","last_active":"2023-12-14T18:26:48.649376+0000","last_peered":"2023-12-14T18:26:48.649376+0000","last_clean":"2023-12-14T18:26:48.649376+0000","last_became_active":"2023-12-14T18:26:48.648882+0000","last_became_peered":"2023-12-14T18:26:48.648882+0000","last_unstale":"2023-12-14T18:26:48.649376+0000","last_undegraded":"2023-12-14T18:26:48.649376+0000","last_fullsized":"2023-12-14T18:26:48.649376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:12.045140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566931+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:48.566931+0000","last_peered":"2023-12-14T18:26:48.566931+0000","last_clean":"2023-12-14T18:26:48.566931+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:48.566931+0000","last_undegraded":"2023-12-14T18:26:48.566931+0000","last_fullsized":"2023-12-14T18:26:48.566931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:47.382142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.572300+0000","last_change":"2023-12-14T18:26:48.572300+0000","last_active":"2023-12-14T18:26:48.572300+0000","last_peered":"2023-12-14T18:26:48.572300+0000","last_clean":"2023-12-14T18:26:48.572300+0000","last_became_active":"2023-12-14T18:26:48.571895+0000","last_became_peered":"2023-12-14T18:26:48.571895+0000","last_unstale":"2023-12-14T18:26:48.572300+0000","last_undegraded":"2023-12-14T18:26:48.572300+0000","last_fullsized":"2023-12-14T18:26:48.572300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:40:54.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566446+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:48.566446+0000","last_peered":"2023-12-14T18:26:48.566446+0000","last_clean":"2023-12-14T18:26:48.566446+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:48.566446+0000","last_undegraded":"2023-12-14T18:26:48.566446+0000","last_fullsized":"2023-12-14T18:26:48.566446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:44.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570871+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:48.570871+0000","last_peered":"2023-12-14T18:26:48.570871+0000","last_clean":"2023-12-14T18:26:48.570871+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:48.570871+0000","last_undegraded":"2023-12-14T18:26:48.570871+0000","last_fullsized":"2023-12-14T18:26:48.570871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:28.018194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561975+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:48.561975+0000","last_peered":"2023-12-14T18:26:48.561975+0000","last_clean":"2023-12-14T18:26:48.561975+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:48.561975+0000","last_undegraded":"2023-12-14T18:26:48.561975+0000","last_fullsized":"2023-12-14T18:26:48.561975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:37.576788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566649+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:48.566649+0000","last_peered":"2023-12-14T18:26:48.566649+0000","last_clean":"2023-12-14T18:26:48.566649+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:48.566649+0000","last_undegraded":"2023-12-14T18:26:48.566649+0000","last_fullsized":"2023-12-14T18:26:48.566649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:37.853928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562990+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:26:48.562990+0000","last_peered":"2023-12-14T18:26:48.562990+0000","last_clean":"2023-12-14T18:26:48.562990+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:26:48.562990+0000","last_undegraded":"2023-12-14T18:26:48.562990+0000","last_fullsized":"2023-12-14T18:26:48.562990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:32.215966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647772+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:26:48.647772+0000","last_peered":"2023-12-14T18:26:48.647772+0000","last_clean":"2023-12-14T18:26:48.647772+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:26:48.647772+0000","last_undegraded":"2023-12-14T18:26:48.647772+0000","last_fullsized":"2023-12-14T18:26:48.647772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:51:12.578633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563700+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:48.563700+0000","last_peered":"2023-12-14T18:26:48.563700+0000","last_clean":"2023-12-14T18:26:48.563700+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:48.563700+0000","last_undegraded":"2023-12-14T18:26:48.563700+0000","last_fullsized":"2023-12-14T18:26:48.563700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:58.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561937+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:48.561937+0000","last_peered":"2023-12-14T18:26:48.561937+0000","last_clean":"2023-12-14T18:26:48.561937+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:48.561937+0000","last_undegraded":"2023-12-14T18:26:48.561937+0000","last_fullsized":"2023-12-14T18:26:48.561937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:21.783482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647707+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:26:48.647707+0000","last_peered":"2023-12-14T18:26:48.647707+0000","last_clean":"2023-12-14T18:26:48.647707+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:26:48.647707+0000","last_undegraded":"2023-12-14T18:26:48.647707+0000","last_fullsized":"2023-12-14T18:26:48.647707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:26:41.629317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563228+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:48.563228+0000","last_peered":"2023-12-14T18:26:48.563228+0000","last_clean":"2023-12-14T18:26:48.563228+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:48.563228+0000","last_undegraded":"2023-12-14T18:26:48.563228+0000","last_fullsized":"2023-12-14T18:26:48.563228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:41:45.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566971+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:48.566971+0000","last_peered":"2023-12-14T18:26:48.566971+0000","last_clean":"2023-12-14T18:26:48.566971+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:48.566971+0000","last_undegraded":"2023-12-14T18:26:48.566971+0000","last_fullsized":"2023-12-14T18:26:48.566971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:54.638247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:34.717332+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:27:34.717332+0000","last_peered":"2023-12-14T18:27:34.717332+0000","last_clean":"2023-12-14T18:27:34.717332+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:27:34.717332+0000","last_undegraded":"2023-12-14T18:27:34.717332+0000","last_fullsized":"2023-12-14T18:27:34.717332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:07:36.295981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562086+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:48.562086+0000","last_peered":"2023-12-14T18:26:48.562086+0000","last_clean":"2023-12-14T18:26:48.562086+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:48.562086+0000","last_undegraded":"2023-12-14T18:26:48.562086+0000","last_fullsized":"2023-12-14T18:26:48.562086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:00.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559419+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:48.559419+0000","last_peered":"2023-12-14T18:26:48.559419+0000","last_clean":"2023-12-14T18:26:48.559419+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:48.559419+0000","last_undegraded":"2023-12-14T18:26:48.559419+0000","last_fullsized":"2023-12-14T18:26:48.559419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:01.825894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564209+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:48.564209+0000","last_peered":"2023-12-14T18:26:48.564209+0000","last_clean":"2023-12-14T18:26:48.564209+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:48.564209+0000","last_undegraded":"2023-12-14T18:26:48.564209+0000","last_fullsized":"2023-12-14T18:26:48.564209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:27:58.179129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561521+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:48.561521+0000","last_peered":"2023-12-14T18:26:48.561521+0000","last_clean":"2023-12-14T18:26:48.561521+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:48.561521+0000","last_undegraded":"2023-12-14T18:26:48.561521+0000","last_fullsized":"2023-12-14T18:26:48.561521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:22.740418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566535+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:48.566535+0000","last_peered":"2023-12-14T18:26:48.566535+0000","last_clean":"2023-12-14T18:26:48.566535+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:48.566535+0000","last_undegraded":"2023-12-14T18:26:48.566535+0000","last_fullsized":"2023-12-14T18:26:48.566535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:54:48.048416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562125+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:48.562125+0000","last_peered":"2023-12-14T18:26:48.562125+0000","last_clean":"2023-12-14T18:26:48.562125+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:48.562125+0000","last_undegraded":"2023-12-14T18:26:48.562125+0000","last_fullsized":"2023-12-14T18:26:48.562125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:36:49.964293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561575+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:48.561575+0000","last_peered":"2023-12-14T18:26:48.561575+0000","last_clean":"2023-12-14T18:26:48.561575+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:48.561575+0000","last_undegraded":"2023-12-14T18:26:48.561575+0000","last_fullsized":"2023-12-14T18:26:48.561575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:20:52.709127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562632+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:48.562632+0000","last_peered":"2023-12-14T18:26:48.562632+0000","last_clean":"2023-12-14T18:26:48.562632+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:48.562632+0000","last_undegraded":"2023-12-14T18:26:48.562632+0000","last_fullsized":"2023-12-14T18:26:48.562632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:26.536039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568593+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:48.568593+0000","last_peered":"2023-12-14T18:26:48.568593+0000","last_clean":"2023-12-14T18:26:48.568593+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:48.568593+0000","last_undegraded":"2023-12-14T18:26:48.568593+0000","last_fullsized":"2023-12-14T18:26:48.568593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:59:51.600710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564002+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:48.564002+0000","last_peered":"2023-12-14T18:26:48.564002+0000","last_clean":"2023-12-14T18:26:48.564002+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:48.564002+0000","last_undegraded":"2023-12-14T18:26:48.564002+0000","last_fullsized":"2023-12-14T18:26:48.564002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:06.768114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559014+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:48.559014+0000","last_peered":"2023-12-14T18:26:48.559014+0000","last_clean":"2023-12-14T18:26:48.559014+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:48.559014+0000","last_undegraded":"2023-12-14T18:26:48.559014+0000","last_fullsized":"2023-12-14T18:26:48.559014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:32.285691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.577550+0000","last_change":"2023-12-14T18:26:48.577550+0000","last_active":"2023-12-14T18:26:48.577550+0000","last_peered":"2023-12-14T18:26:48.577550+0000","last_clean":"2023-12-14T18:26:48.577550+0000","last_became_active":"2023-12-14T18:26:48.569607+0000","last_became_peered":"2023-12-14T18:26:48.569607+0000","last_unstale":"2023-12-14T18:26:48.577550+0000","last_undegraded":"2023-12-14T18:26:48.577550+0000","last_fullsized":"2023-12-14T18:26:48.577550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:23.766724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559798+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:48.559798+0000","last_peered":"2023-12-14T18:26:48.559798+0000","last_clean":"2023-12-14T18:26:48.559798+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:48.559798+0000","last_undegraded":"2023-12-14T18:26:48.559798+0000","last_fullsized":"2023-12-14T18:26:48.559798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:29.491959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563921+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:26:48.563921+0000","last_peered":"2023-12-14T18:26:48.563921+0000","last_clean":"2023-12-14T18:26:48.563921+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:26:48.563921+0000","last_undegraded":"2023-12-14T18:26:48.563921+0000","last_fullsized":"2023-12-14T18:26:48.563921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:12.984642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563541+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:48.563541+0000","last_peered":"2023-12-14T18:26:48.563541+0000","last_clean":"2023-12-14T18:26:48.563541+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:48.563541+0000","last_undegraded":"2023-12-14T18:26:48.563541+0000","last_fullsized":"2023-12-14T18:26:48.563541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:56.278549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557904+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:48.557904+0000","last_peered":"2023-12-14T18:26:48.557904+0000","last_clean":"2023-12-14T18:26:48.557904+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:48.557904+0000","last_undegraded":"2023-12-14T18:26:48.557904+0000","last_fullsized":"2023-12-14T18:26:48.557904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:36:53.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563087+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:48.563087+0000","last_peered":"2023-12-14T18:26:48.563087+0000","last_clean":"2023-12-14T18:26:48.563087+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:48.563087+0000","last_undegraded":"2023-12-14T18:26:48.563087+0000","last_fullsized":"2023-12-14T18:26:48.563087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:19.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559959+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:48.559959+0000","last_peered":"2023-12-14T18:26:48.559959+0000","last_clean":"2023-12-14T18:26:48.559959+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:48.559959+0000","last_undegraded":"2023-12-14T18:26:48.559959+0000","last_fullsized":"2023-12-14T18:26:48.559959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:05.103613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566175+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:48.566175+0000","last_peered":"2023-12-14T18:26:48.566175+0000","last_clean":"2023-12-14T18:26:48.566175+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:48.566175+0000","last_undegraded":"2023-12-14T18:26:48.566175+0000","last_fullsized":"2023-12-14T18:26:48.566175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:21.311660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562945+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:26:48.562945+0000","last_peered":"2023-12-14T18:26:48.562945+0000","last_clean":"2023-12-14T18:26:48.562945+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:26:48.562945+0000","last_undegraded":"2023-12-14T18:26:48.562945+0000","last_fullsized":"2023-12-14T18:26:48.562945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:04:53.461295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558908+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:48.558908+0000","last_peered":"2023-12-14T18:26:48.558908+0000","last_clean":"2023-12-14T18:26:48.558908+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:48.558908+0000","last_undegraded":"2023-12-14T18:26:48.558908+0000","last_fullsized":"2023-12-14T18:26:48.558908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:04:46.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561591+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:48.561591+0000","last_peered":"2023-12-14T18:26:48.561591+0000","last_clean":"2023-12-14T18:26:48.561591+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:48.561591+0000","last_undegraded":"2023-12-14T18:26:48.561591+0000","last_fullsized":"2023-12-14T18:26:48.561591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:52:50.350437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562857+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:48.562857+0000","last_peered":"2023-12-14T18:26:48.562857+0000","last_clean":"2023-12-14T18:26:48.562857+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:48.562857+0000","last_undegraded":"2023-12-14T18:26:48.562857+0000","last_fullsized":"2023-12-14T18:26:48.562857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:22.082703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565524+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:26:48.565524+0000","last_peered":"2023-12-14T18:26:48.565524+0000","last_clean":"2023-12-14T18:26:48.565524+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:26:48.565524+0000","last_undegraded":"2023-12-14T18:26:48.565524+0000","last_fullsized":"2023-12-14T18:26:48.565524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:51:55.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559293+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:48.559293+0000","last_peered":"2023-12-14T18:26:48.559293+0000","last_clean":"2023-12-14T18:26:48.559293+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:48.559293+0000","last_undegraded":"2023-12-14T18:26:48.559293+0000","last_fullsized":"2023-12-14T18:26:48.559293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:17:28.674237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563263+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:48.563263+0000","last_peered":"2023-12-14T18:26:48.563263+0000","last_clean":"2023-12-14T18:26:48.563263+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:48.563263+0000","last_undegraded":"2023-12-14T18:26:48.563263+0000","last_fullsized":"2023-12-14T18:26:48.563263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:08.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.578387+0000","last_change":"2023-12-14T18:26:48.577964+0000","last_active":"2023-12-14T18:26:48.578387+0000","last_peered":"2023-12-14T18:26:48.578387+0000","last_clean":"2023-12-14T18:26:48.578387+0000","last_became_active":"2023-12-14T18:26:48.569637+0000","last_became_peered":"2023-12-14T18:26:48.569637+0000","last_unstale":"2023-12-14T18:26:48.578387+0000","last_undegraded":"2023-12-14T18:26:48.578387+0000","last_fullsized":"2023-12-14T18:26:48.578387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:31:31.614815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562048+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:48.562048+0000","last_peered":"2023-12-14T18:26:48.562048+0000","last_clean":"2023-12-14T18:26:48.562048+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:48.562048+0000","last_undegraded":"2023-12-14T18:26:48.562048+0000","last_fullsized":"2023-12-14T18:26:48.562048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:43.573208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559867+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:48.559867+0000","last_peered":"2023-12-14T18:26:48.559867+0000","last_clean":"2023-12-14T18:26:48.559867+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:48.559867+0000","last_undegraded":"2023-12-14T18:26:48.559867+0000","last_fullsized":"2023-12-14T18:26:48.559867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:15:04.165007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.574468+0000","last_change":"2023-12-14T18:26:48.574083+0000","last_active":"2023-12-14T18:26:48.574468+0000","last_peered":"2023-12-14T18:26:48.574468+0000","last_clean":"2023-12-14T18:26:48.574468+0000","last_became_active":"2023-12-14T18:26:48.565205+0000","last_became_peered":"2023-12-14T18:26:48.565205+0000","last_unstale":"2023-12-14T18:26:48.574468+0000","last_undegraded":"2023-12-14T18:26:48.574468+0000","last_fullsized":"2023-12-14T18:26:48.574468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:19.409191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562445+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:48.562445+0000","last_peered":"2023-12-14T18:26:48.562445+0000","last_clean":"2023-12-14T18:26:48.562445+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:48.562445+0000","last_undegraded":"2023-12-14T18:26:48.562445+0000","last_fullsized":"2023-12-14T18:26:48.562445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:37.001669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557585+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:48.557585+0000","last_peered":"2023-12-14T18:26:48.557585+0000","last_clean":"2023-12-14T18:26:48.557585+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:48.557585+0000","last_undegraded":"2023-12-14T18:26:48.557585+0000","last_fullsized":"2023-12-14T18:26:48.557585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:54.017877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564140+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:48.564140+0000","last_peered":"2023-12-14T18:26:48.564140+0000","last_clean":"2023-12-14T18:26:48.564140+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:48.564140+0000","last_undegraded":"2023-12-14T18:26:48.564140+0000","last_fullsized":"2023-12-14T18:26:48.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:46:47.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562523+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:48.562523+0000","last_peered":"2023-12-14T18:26:48.562523+0000","last_clean":"2023-12-14T18:26:48.562523+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:48.562523+0000","last_undegraded":"2023-12-14T18:26:48.562523+0000","last_fullsized":"2023-12-14T18:26:48.562523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:52:04.129230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566378+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:48.566378+0000","last_peered":"2023-12-14T18:26:48.566378+0000","last_clean":"2023-12-14T18:26:48.566378+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:48.566378+0000","last_undegraded":"2023-12-14T18:26:48.566378+0000","last_fullsized":"2023-12-14T18:26:48.566378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:08:14.934634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564930+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:48.564930+0000","last_peered":"2023-12-14T18:26:48.564930+0000","last_clean":"2023-12-14T18:26:48.564930+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:48.564930+0000","last_undegraded":"2023-12-14T18:26:48.564930+0000","last_fullsized":"2023-12-14T18:26:48.564930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:28.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562150+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:48.562150+0000","last_peered":"2023-12-14T18:26:48.562150+0000","last_clean":"2023-12-14T18:26:48.562150+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:48.562150+0000","last_undegraded":"2023-12-14T18:26:48.562150+0000","last_fullsized":"2023-12-14T18:26:48.562150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:43:39.281554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562009+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:48.562009+0000","last_peered":"2023-12-14T18:26:48.562009+0000","last_clean":"2023-12-14T18:26:48.562009+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:48.562009+0000","last_undegraded":"2023-12-14T18:26:48.562009+0000","last_fullsized":"2023-12-14T18:26:48.562009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:49:45.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564454+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:26:48.564454+0000","last_peered":"2023-12-14T18:26:48.564454+0000","last_clean":"2023-12-14T18:26:48.564454+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:26:48.564454+0000","last_undegraded":"2023-12-14T18:26:48.564454+0000","last_fullsized":"2023-12-14T18:26:48.564454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:16.913094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563425+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:26:48.563425+0000","last_peered":"2023-12-14T18:26:48.563425+0000","last_clean":"2023-12-14T18:26:48.563425+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:26:48.563425+0000","last_undegraded":"2023-12-14T18:26:48.563425+0000","last_fullsized":"2023-12-14T18:26:48.563425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:54.437297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557742+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:48.557742+0000","last_peered":"2023-12-14T18:26:48.557742+0000","last_clean":"2023-12-14T18:26:48.557742+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:48.557742+0000","last_undegraded":"2023-12-14T18:26:48.557742+0000","last_fullsized":"2023-12-14T18:26:48.557742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:35.196672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564064+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:48.564064+0000","last_peered":"2023-12-14T18:26:48.564064+0000","last_clean":"2023-12-14T18:26:48.564064+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:48.564064+0000","last_undegraded":"2023-12-14T18:26:48.564064+0000","last_fullsized":"2023-12-14T18:26:48.564064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:27.906838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562103+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:48.562103+0000","last_peered":"2023-12-14T18:26:48.562103+0000","last_clean":"2023-12-14T18:26:48.562103+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:48.562103+0000","last_undegraded":"2023-12-14T18:26:48.562103+0000","last_fullsized":"2023-12-14T18:26:48.562103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:44:51.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566964+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:26:48.566964+0000","last_peered":"2023-12-14T18:26:48.566964+0000","last_clean":"2023-12-14T18:26:48.566964+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:26:48.566964+0000","last_undegraded":"2023-12-14T18:26:48.566964+0000","last_fullsized":"2023-12-14T18:26:48.566964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:07:32.112350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559871+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:48.559871+0000","last_peered":"2023-12-14T18:26:48.559871+0000","last_clean":"2023-12-14T18:26:48.559871+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:48.559871+0000","last_undegraded":"2023-12-14T18:26:48.559871+0000","last_fullsized":"2023-12-14T18:26:48.559871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:36:21.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566829+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:48.566829+0000","last_peered":"2023-12-14T18:26:48.566829+0000","last_clean":"2023-12-14T18:26:48.566829+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:48.566829+0000","last_undegraded":"2023-12-14T18:26:48.566829+0000","last_fullsized":"2023-12-14T18:26:48.566829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:17.885326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559062+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:48.559062+0000","last_peered":"2023-12-14T18:26:48.559062+0000","last_clean":"2023-12-14T18:26:48.559062+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:48.559062+0000","last_undegraded":"2023-12-14T18:26:48.559062+0000","last_fullsized":"2023-12-14T18:26:48.559062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:28:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560244+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:48.560244+0000","last_peered":"2023-12-14T18:26:48.560244+0000","last_clean":"2023-12-14T18:26:48.560244+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:48.560244+0000","last_undegraded":"2023-12-14T18:26:48.560244+0000","last_fullsized":"2023-12-14T18:26:48.560244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:54.715861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562903+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:48.562903+0000","last_peered":"2023-12-14T18:26:48.562903+0000","last_clean":"2023-12-14T18:26:48.562903+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:48.562903+0000","last_undegraded":"2023-12-14T18:26:48.562903+0000","last_fullsized":"2023-12-14T18:26:48.562903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:54:48.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562082+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:48.562082+0000","last_peered":"2023-12-14T18:26:48.562082+0000","last_clean":"2023-12-14T18:26:48.562082+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:48.562082+0000","last_undegraded":"2023-12-14T18:26:48.562082+0000","last_fullsized":"2023-12-14T18:26:48.562082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:40:01.866310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68351,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":469,"num_read_kb":305,"num_write":160,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514527,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739028,"statfs":{"total":95995035648,"available":95988764672,"internally_reserved":0,"allocated":954368,"data_stored":484012,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68400000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.318,"5min":0.311,"15min":0.311},"max":{"1min":0.69399999999999995,"5min":0.751,"15min":0.751},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":1.268,"15min":1.268},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90000000000000002,"5min":1.097,"15min":1.097},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.50800000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.79500000000000004,"5min":1.2,"15min":1.2},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.878,"15min":0.878},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.88200000000000001,"5min":1.075,"15min":1.075},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.05,"15min":1.05},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625}]},{"osd":9,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.86199999999999999,"5min":1.115,"15min":1.115},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.68000000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0009999999999999,"5min":1.389,"15min":1.389},"last":0.76300000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.628,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98599999999999999,"5min":1.147,"15min":1.147},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92800000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.752}]}]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6584,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738568,"statfs":{"total":95995035648,"available":95988293632,"internally_reserved":0,"allocated":1097728,"data_stored":1135398,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45800000000000002,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.748,"5min":0.749,"15min":0.749},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.76000000000000001,"15min":0.78900000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78500000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.80300000000000005,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.628},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.56200000000000006}]},{"osd":9,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":1.252,"15min":1.252},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.93600000000000005,"15min":0.97999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85699999999999998,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Dec 14 18:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88500000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90400000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68500000000000005}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":974848,"data_stored":488344,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.437}]},{"osd":1,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.504},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38,"15min":0.38},"max":{"1min":1.01,"5min":1.028,"15min":1.028},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.081,"15min":1.081},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.036,"15min":1.036},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1200000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0189999999999999,"5min":1.123,"15min":1.123},"last":0.79800000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.81799999999999995,"15min":0.83099999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0840000000000001,"5min":1.105,"15min":1.105},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84299999999999997,"15min":0.85699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0329999999999999,"5min":1.121,"15min":1.121},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6460,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738692,"statfs":{"total":95995035648,"available":95988420608,"internally_reserved":0,"allocated":970752,"data_stored":484168,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78400000000000003,"5min":0.875,"15min":0.875},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.48799999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.83799999999999997,"5min":0.879,"15min":0.879},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.871},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78800000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71399999999999997,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95899999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.66900000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.90400000000000003}]},{"osd":9,"last update":"Thu Dec 14 18:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.628,"5min":0.502,"15min":0.502},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.96899999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.68500000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.004}]}]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":1079376,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.34899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72799999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.53000000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73399999999999999,"5min":1.8,"15min":1.8},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.626}]},{"osd":5,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78900000000000003,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.503,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.58199999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83799999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84099999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82299999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85799999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593777,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6440,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738712,"statfs":{"total":95995035648,"available":95988441088,"internally_reserved":0,"allocated":950272,"data_stored":479424,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.34599999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.435}]},{"osd":1,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64200000000000002,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.56100000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.88200000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.57899999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.127,"5min":1.133,"15min":1.133},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":539450,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79000000000000004,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":5.0800000000000001,"15min":5.0800000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.53000000000000003,"15min":0.47999999999999998},"min":{"1min":0.40200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82799999999999996,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.55600000000000005},"min":{"1min":0.46200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82899999999999996,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45200000000000001,"15min":0.41099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44}]},{"osd":5,"last update":"Thu Dec 14 18:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61599999999999999,"15min":0.58199999999999996},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82899999999999996,"5min":1.657,"15min":1.657},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.58599999999999997},"min":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.872,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48399999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.628},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92500000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71799999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.59299999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.70899999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.78800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":480812,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.622,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94199999999999995,"5min":1.091,"15min":1.091},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.83499999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.079,"15min":1.079},"last":0.877}]},{"osd":9,"last update":"Thu Dec 14 18:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.82499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":0.72399999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.55300000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.258,"15min":1.258},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.077,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86599999999999999,"15min":0.871},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.125,"5min":1.2,"15min":1.2},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88200000000000001,"15min":0.877},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.748}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7540,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737612,"statfs":{"total":95995035648,"available":95987314688,"internally_reserved":0,"allocated":1028096,"data_stored":539607,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.484,"15min":1.484},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.52300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84899999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.47799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.83999999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.443},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81000000000000005,"5min":1.514,"15min":1.514},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78300000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57099999999999995},"min":{"1min":0.44400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.88700000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60299999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.67400000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.54700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79700000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":958464,"data_stored":479345,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.43099999999999999,"15min":0.39100000000000001},"min":{"1min":0.29699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.41899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.63600000000000001,"15min":0.55800000000000005},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.5800000000000001,"5min":6.5800000000000001,"15min":6.5800000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61699999999999999,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.67200000000000004,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.5419999999999998,"5min":6.5419999999999998,"15min":6.5419999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:27:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64600000000000002,"15min":0.61299999999999999},"min":{"1min":0.50600000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.621}]},{"osd":5,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.68200000000000005,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.79600000000000004,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.617,"5min":6.617,"15min":6.617},"last":0.73699999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":6.7249999999999996,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.753},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85799999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.72799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986232,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":484990,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.379,"15min":0.34499999999999997},"min":{"1min":0.29699999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66400000000000003,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.315},{"interface":"front","average":{"1min":0.44,"5min":0.39200000000000002,"15min":0.36399999999999999},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64500000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.42299999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78900000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.442},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":1.121,"15min":1.121},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.376},{"interface":"front","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85999999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.879,"15min":0.879},"last":0.40600000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":1.161,"15min":1.161},"last":0.59899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.56999999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85699999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53300000000000003}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58799999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822709,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":1024000,"data_stored":1075285,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63800000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66600000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.314,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":4.7290000000000001,"5min":4.7290000000000001,"15min":4.7290000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.621,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.391,"5min":6.391,"15min":6.391},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.66800000000000004,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.3300000000000001,"5min":6.3300000000000001,"15min":6.3300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.66600000000000004,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Dec 14 18:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81200000000000006,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.751}]},{"osd":8,"last update":"Thu Dec 14 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.84599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.51200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58999999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.71099999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.58599999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:27:40.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:27:40.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 -- 172.21.15.31:0/3042046490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0103660 msgr2=0x7f79e0105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/3042046490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0103660 0x7f79e0105a40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009990 tx=0x7f79d0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 -- 172.21.15.31:0/3042046490 shutdown_connections 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/3042046490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e0105f80 0x7f79e0108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/3042046490 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0100d40 0x7f79e0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/3042046490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0103660 0x7f79e0105a40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 -- 172.21.15.31:0/3042046490 >> 172.21.15.31:0/3042046490 conn(0x7f79e00fa780 msgr2=0x7f79e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 -- 172.21.15.31:0/3042046490 shutdown_connections 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.171+0000 7f79e7cf4700 1 -- 172.21.15.31:0/3042046490 wait complete. 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 Processor -- start 2023-12-14T18:27:40.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 -- start start 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e0100d40 0x7f79e01006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0103660 0x7f79e00fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 0x7f79e00ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00ff9d0 con 0x7f79e0105f80 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00ffb40 con 0x7f79e0103660 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e7cf4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e01a0030 con 0x7f79e0100d40 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e6291700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 0x7f79e00ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e6291700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 0x7f79e00ff230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53146/0 (socket says 172.21.15.31:53146) 2023-12-14T18:27:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.172+0000 7f79e6291700 1 -- 172.21.15.31:0/1926424659 learned_addr learned my addr 172.21.15.31:0/1926424659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e5a90700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e0100d40 0x7f79e01006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e528f700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0103660 0x7f79e00fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e6291700 1 -- 172.21.15.31:0/1926424659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e0100d40 msgr2=0x7f79e01006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e6291700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e0100d40 0x7f79e01006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e6291700 1 -- 172.21.15.31:0/1926424659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0103660 msgr2=0x7f79e00fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e6291700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0103660 0x7f79e00fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e6291700 1 -- 172.21.15.31:0/1926424659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d00095f0 con 0x7f79e0105f80 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e528f700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0103660 0x7f79e00fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e6291700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 0x7f79e00ff230 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f79d800ef30 tx=0x7f79d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8018070 con 0x7f79e0105f80 2023-12-14T18:27:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79d800f040 con 0x7f79e0105f80 2023-12-14T18:27:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e010bc10 con 0x7f79e0105f80 2023-12-14T18:27:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800cdb0 con 0x7f79e0105f80 2023-12-14T18:27:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.173+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e010ba80 con 0x7f79e0105f80 2023-12-14T18:27:40.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.174+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e00fc2f0 con 0x7f79e0105f80 2023-12-14T18:27:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.176+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f79d8003a00 con 0x7f79e0105f80 2023-12-14T18:27:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.177+0000 7f79d6ffd700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f79cc0a11e0 0x7f79cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.177+0000 7f79e5a90700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f79cc0a11e0 0x7f79cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.177+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f79d80cd2b0 con 0x7f79e0105f80 2023-12-14T18:27:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.177+0000 7f79e5a90700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f79cc0a11e0 0x7f79cc0a3690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005950 tx=0x7f79dc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.180+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f79d80966c0 con 0x7f79e0105f80 2023-12-14T18:27:40.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.386+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79e0061c30 con 0x7f79cc0a11e0 2023-12-14T18:27:40.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.399+0000 7f79d6ffd700 1 -- 172.21.15.31:0/1926424659 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427137 (secure 0 0 0) 0x7f79e0061c30 con 0x7f79cc0a11e0 2023-12-14T18:27:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f79cc0a11e0 msgr2=0x7f79cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f79cc0a11e0 0x7f79cc0a3690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f79dc005950 tx=0x7f79dc00b410 comp rx=0 tx=0).stop 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 msgr2=0x7f79e00ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 0x7f79e00ff230 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f79d800ef30 tx=0x7f79d800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 shutdown_connections 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79e0105f80 0x7f79e00ff230 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f79cc0a11e0 0x7f79cc0a3690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79e0100d40 0x7f79e01006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 --2- 172.21.15.31:0/1926424659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f79e0103660 0x7f79e00fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 >> 172.21.15.31:0/1926424659 conn(0x7f79e00fa780 msgr2=0x7f79e01019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 shutdown_connections 2023-12-14T18:27:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.404+0000 7f79e7cf4700 1 -- 172.21.15.31:0/1926424659 wait complete. 2023-12-14T18:27:40.415 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:27:40.420 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-12-14T18:27:39.499954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658965,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":605,"num_read_kb":424,"num_write":343,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1182282,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80804,"kb_used_data":11652,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124861020,"statfs":{"total":1151940427776,"available":1151857684480,"internally_reserved":0,"allocated":11931648,"data_stored":7750211,"data_compressed":319020,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4640,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":41,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003489"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562794+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:48.562794+0000","last_peered":"2023-12-14T18:26:48.562794+0000","last_clean":"2023-12-14T18:26:48.562794+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:48.562794+0000","last_undegraded":"2023-12-14T18:26:48.562794+0000","last_fullsized":"2023-12-14T18:26:48.562794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:33:46.076524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557880+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:48.557880+0000","last_peered":"2023-12-14T18:26:48.557880+0000","last_clean":"2023-12-14T18:26:48.557880+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:48.557880+0000","last_undegraded":"2023-12-14T18:26:48.557880+0000","last_fullsized":"2023-12-14T18:26:48.557880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:10.369082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557355+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:48.557355+0000","last_peered":"2023-12-14T18:26:48.557355+0000","last_clean":"2023-12-14T18:26:48.557355+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:48.557355+0000","last_undegraded":"2023-12-14T18:26:48.557355+0000","last_fullsized":"2023-12-14T18:26:48.557355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:23.331965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563177+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:48.563177+0000","last_peered":"2023-12-14T18:26:48.563177+0000","last_clean":"2023-12-14T18:26:48.563177+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:48.563177+0000","last_undegraded":"2023-12-14T18:26:48.563177+0000","last_fullsized":"2023-12-14T18:26:48.563177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:01.529418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567124+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:48.567124+0000","last_peered":"2023-12-14T18:26:48.567124+0000","last_clean":"2023-12-14T18:26:48.567124+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:48.567124+0000","last_undegraded":"2023-12-14T18:26:48.567124+0000","last_fullsized":"2023-12-14T18:26:48.567124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:53:32.988966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647501+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:26:48.647501+0000","last_peered":"2023-12-14T18:26:48.647501+0000","last_clean":"2023-12-14T18:26:48.647501+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:26:48.647501+0000","last_undegraded":"2023-12-14T18:26:48.647501+0000","last_fullsized":"2023-12-14T18:26:48.647501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:00:59.230921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561700+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:48.561700+0000","last_peered":"2023-12-14T18:26:48.561700+0000","last_clean":"2023-12-14T18:26:48.561700+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:48.561700+0000","last_undegraded":"2023-12-14T18:26:48.561700+0000","last_fullsized":"2023-12-14T18:26:48.561700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:47.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559287+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:48.559287+0000","last_peered":"2023-12-14T18:26:48.559287+0000","last_clean":"2023-12-14T18:26:48.559287+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:48.559287+0000","last_undegraded":"2023-12-14T18:26:48.559287+0000","last_fullsized":"2023-12-14T18:26:48.559287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:13:56.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560906+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:48.560906+0000","last_peered":"2023-12-14T18:26:48.560906+0000","last_clean":"2023-12-14T18:26:48.560906+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:48.560906+0000","last_undegraded":"2023-12-14T18:26:48.560906+0000","last_fullsized":"2023-12-14T18:26:48.560906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:38:42.914156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647960+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:26:48.647960+0000","last_peered":"2023-12-14T18:26:48.647960+0000","last_clean":"2023-12-14T18:26:48.647960+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:26:48.647960+0000","last_undegraded":"2023-12-14T18:26:48.647960+0000","last_fullsized":"2023-12-14T18:26:48.647960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:09:41.346067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562267+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:48.562267+0000","last_peered":"2023-12-14T18:26:48.562267+0000","last_clean":"2023-12-14T18:26:48.562267+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:48.562267+0000","last_undegraded":"2023-12-14T18:26:48.562267+0000","last_fullsized":"2023-12-14T18:26:48.562267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:26.374795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563022+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:48.563022+0000","last_peered":"2023-12-14T18:26:48.563022+0000","last_clean":"2023-12-14T18:26:48.563022+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:48.563022+0000","last_undegraded":"2023-12-14T18:26:48.563022+0000","last_fullsized":"2023-12-14T18:26:48.563022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:38.171386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563106+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:48.563106+0000","last_peered":"2023-12-14T18:26:48.563106+0000","last_clean":"2023-12-14T18:26:48.563106+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:48.563106+0000","last_undegraded":"2023-12-14T18:26:48.563106+0000","last_fullsized":"2023-12-14T18:26:48.563106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:18:44.617868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557419+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:48.557419+0000","last_peered":"2023-12-14T18:26:48.557419+0000","last_clean":"2023-12-14T18:26:48.557419+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:48.557419+0000","last_undegraded":"2023-12-14T18:26:48.557419+0000","last_fullsized":"2023-12-14T18:26:48.557419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:04:10.141803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558264+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:48.558264+0000","last_peered":"2023-12-14T18:26:48.558264+0000","last_clean":"2023-12-14T18:26:48.558264+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:48.558264+0000","last_undegraded":"2023-12-14T18:26:48.558264+0000","last_fullsized":"2023-12-14T18:26:48.558264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:53.022658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559788+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:48.559788+0000","last_peered":"2023-12-14T18:26:48.559788+0000","last_clean":"2023-12-14T18:26:48.559788+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:48.559788+0000","last_undegraded":"2023-12-14T18:26:48.559788+0000","last_fullsized":"2023-12-14T18:26:48.559788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:18.996023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562659+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:48.562659+0000","last_peered":"2023-12-14T18:26:48.562659+0000","last_clean":"2023-12-14T18:26:48.562659+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:48.562659+0000","last_undegraded":"2023-12-14T18:26:48.562659+0000","last_fullsized":"2023-12-14T18:26:48.562659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:00:27.549342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562608+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:26:48.562608+0000","last_peered":"2023-12-14T18:26:48.562608+0000","last_clean":"2023-12-14T18:26:48.562608+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:26:48.562608+0000","last_undegraded":"2023-12-14T18:26:48.562608+0000","last_fullsized":"2023-12-14T18:26:48.562608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:06:20.688646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566623+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:26:48.566623+0000","last_peered":"2023-12-14T18:26:48.566623+0000","last_clean":"2023-12-14T18:26:48.566623+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:26:48.566623+0000","last_undegraded":"2023-12-14T18:26:48.566623+0000","last_fullsized":"2023-12-14T18:26:48.566623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:58.296365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564244+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:26:48.564244+0000","last_peered":"2023-12-14T18:26:48.564244+0000","last_clean":"2023-12-14T18:26:48.564244+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:26:48.564244+0000","last_undegraded":"2023-12-14T18:26:48.564244+0000","last_fullsized":"2023-12-14T18:26:48.564244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:23.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563082+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:26:48.563082+0000","last_peered":"2023-12-14T18:26:48.563082+0000","last_clean":"2023-12-14T18:26:48.563082+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:26:48.563082+0000","last_undegraded":"2023-12-14T18:26:48.563082+0000","last_fullsized":"2023-12-14T18:26:48.563082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:35:18.652626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559422+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:48.559422+0000","last_peered":"2023-12-14T18:26:48.559422+0000","last_clean":"2023-12-14T18:26:48.559422+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:48.559422+0000","last_undegraded":"2023-12-14T18:26:48.559422+0000","last_fullsized":"2023-12-14T18:26:48.559422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:31.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562609+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:48.562609+0000","last_peered":"2023-12-14T18:26:48.562609+0000","last_clean":"2023-12-14T18:26:48.562609+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:48.562609+0000","last_undegraded":"2023-12-14T18:26:48.562609+0000","last_fullsized":"2023-12-14T18:26:48.562609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:43.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559749+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:48.559749+0000","last_peered":"2023-12-14T18:26:48.559749+0000","last_clean":"2023-12-14T18:26:48.559749+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:48.559749+0000","last_undegraded":"2023-12-14T18:26:48.559749+0000","last_fullsized":"2023-12-14T18:26:48.559749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:43.650498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558888+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:48.558888+0000","last_peered":"2023-12-14T18:26:48.558888+0000","last_clean":"2023-12-14T18:26:48.558888+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:48.558888+0000","last_undegraded":"2023-12-14T18:26:48.558888+0000","last_fullsized":"2023-12-14T18:26:48.558888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:50.284685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564072+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:48.564072+0000","last_peered":"2023-12-14T18:26:48.564072+0000","last_clean":"2023-12-14T18:26:48.564072+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:48.564072+0000","last_undegraded":"2023-12-14T18:26:48.564072+0000","last_fullsized":"2023-12-14T18:26:48.564072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:29:21.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568778+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:48.568778+0000","last_peered":"2023-12-14T18:26:48.568778+0000","last_clean":"2023-12-14T18:26:48.568778+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:48.568778+0000","last_undegraded":"2023-12-14T18:26:48.568778+0000","last_fullsized":"2023-12-14T18:26:48.568778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:38.658671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560627+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:48.560627+0000","last_peered":"2023-12-14T18:26:48.560627+0000","last_clean":"2023-12-14T18:26:48.560627+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:48.560627+0000","last_undegraded":"2023-12-14T18:26:48.560627+0000","last_fullsized":"2023-12-14T18:26:48.560627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:54:13.436594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562615+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:48.562615+0000","last_peered":"2023-12-14T18:26:48.562615+0000","last_clean":"2023-12-14T18:26:48.562615+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:48.562615+0000","last_undegraded":"2023-12-14T18:26:48.562615+0000","last_fullsized":"2023-12-14T18:26:48.562615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:22:13.915160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568436+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:48.568436+0000","last_peered":"2023-12-14T18:26:48.568436+0000","last_clean":"2023-12-14T18:26:48.568436+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:48.568436+0000","last_undegraded":"2023-12-14T18:26:48.568436+0000","last_fullsized":"2023-12-14T18:26:48.568436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:51:57.429285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563999+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:48.563999+0000","last_peered":"2023-12-14T18:26:48.563999+0000","last_clean":"2023-12-14T18:26:48.563999+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:48.563999+0000","last_undegraded":"2023-12-14T18:26:48.563999+0000","last_fullsized":"2023-12-14T18:26:48.563999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:21:53.454334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564084+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:26:48.564084+0000","last_peered":"2023-12-14T18:26:48.564084+0000","last_clean":"2023-12-14T18:26:48.564084+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:26:48.564084+0000","last_undegraded":"2023-12-14T18:26:48.564084+0000","last_fullsized":"2023-12-14T18:26:48.564084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:33.154296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:35.636522+0000","last_change":"2023-12-14T18:26:48.574808+0000","last_active":"2023-12-14T18:27:35.636522+0000","last_peered":"2023-12-14T18:27:35.636522+0000","last_clean":"2023-12-14T18:27:35.636522+0000","last_became_active":"2023-12-14T18:26:48.568640+0000","last_became_peered":"2023-12-14T18:26:48.568640+0000","last_unstale":"2023-12-14T18:27:35.636522+0000","last_undegraded":"2023-12-14T18:27:35.636522+0000","last_fullsized":"2023-12-14T18:27:35.636522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:16.685666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566784+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:48.566784+0000","last_peered":"2023-12-14T18:26:48.566784+0000","last_clean":"2023-12-14T18:26:48.566784+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:48.566784+0000","last_undegraded":"2023-12-14T18:26:48.566784+0000","last_fullsized":"2023-12-14T18:26:48.566784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.326220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.573798+0000","last_change":"2023-12-14T18:26:48.573798+0000","last_active":"2023-12-14T18:26:48.573798+0000","last_peered":"2023-12-14T18:26:48.573798+0000","last_clean":"2023-12-14T18:26:48.573798+0000","last_became_active":"2023-12-14T18:26:48.573308+0000","last_became_peered":"2023-12-14T18:26:48.573308+0000","last_unstale":"2023-12-14T18:26:48.573798+0000","last_undegraded":"2023-12-14T18:26:48.573798+0000","last_fullsized":"2023-12-14T18:26:48.573798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:28:46.273622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647410+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:48.647410+0000","last_peered":"2023-12-14T18:26:48.647410+0000","last_clean":"2023-12-14T18:26:48.647410+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:48.647410+0000","last_undegraded":"2023-12-14T18:26:48.647410+0000","last_fullsized":"2023-12-14T18:26:48.647410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:05:15.020350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564341+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:48.564341+0000","last_peered":"2023-12-14T18:26:48.564341+0000","last_clean":"2023-12-14T18:26:48.564341+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:48.564341+0000","last_undegraded":"2023-12-14T18:26:48.564341+0000","last_fullsized":"2023-12-14T18:26:48.564341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:43.990428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570306+0000","last_change":"2023-12-14T18:26:48.570306+0000","last_active":"2023-12-14T18:26:48.570306+0000","last_peered":"2023-12-14T18:26:48.570306+0000","last_clean":"2023-12-14T18:26:48.570306+0000","last_became_active":"2023-12-14T18:26:48.569803+0000","last_became_peered":"2023-12-14T18:26:48.569803+0000","last_unstale":"2023-12-14T18:26:48.570306+0000","last_undegraded":"2023-12-14T18:26:48.570306+0000","last_fullsized":"2023-12-14T18:26:48.570306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:33.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562436+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:48.562436+0000","last_peered":"2023-12-14T18:26:48.562436+0000","last_clean":"2023-12-14T18:26:48.562436+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:48.562436+0000","last_undegraded":"2023-12-14T18:26:48.562436+0000","last_fullsized":"2023-12-14T18:26:48.562436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:53.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559154+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:48.559154+0000","last_peered":"2023-12-14T18:26:48.559154+0000","last_clean":"2023-12-14T18:26:48.559154+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:48.559154+0000","last_undegraded":"2023-12-14T18:26:48.559154+0000","last_fullsized":"2023-12-14T18:26:48.559154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:49.740844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557724+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:48.557724+0000","last_peered":"2023-12-14T18:26:48.557724+0000","last_clean":"2023-12-14T18:26:48.557724+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:48.557724+0000","last_undegraded":"2023-12-14T18:26:48.557724+0000","last_fullsized":"2023-12-14T18:26:48.557724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:44:17.333118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567219+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:48.567219+0000","last_peered":"2023-12-14T18:26:48.567219+0000","last_clean":"2023-12-14T18:26:48.567219+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:48.567219+0000","last_undegraded":"2023-12-14T18:26:48.567219+0000","last_fullsized":"2023-12-14T18:26:48.567219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:45:09.792477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561943+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:26:48.561943+0000","last_peered":"2023-12-14T18:26:48.561943+0000","last_clean":"2023-12-14T18:26:48.561943+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:26:48.561943+0000","last_undegraded":"2023-12-14T18:26:48.561943+0000","last_fullsized":"2023-12-14T18:26:48.561943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:21:43.141339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559411+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:48.559411+0000","last_peered":"2023-12-14T18:26:48.559411+0000","last_clean":"2023-12-14T18:26:48.559411+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:48.559411+0000","last_undegraded":"2023-12-14T18:26:48.559411+0000","last_fullsized":"2023-12-14T18:26:48.559411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:36.307493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:37.638765+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:27:37.638765+0000","last_peered":"2023-12-14T18:27:37.638765+0000","last_clean":"2023-12-14T18:27:37.638765+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:27:37.638765+0000","last_undegraded":"2023-12-14T18:27:37.638765+0000","last_fullsized":"2023-12-14T18:27:37.638765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:20:23.673760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560703+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:48.560703+0000","last_peered":"2023-12-14T18:26:48.560703+0000","last_clean":"2023-12-14T18:26:48.560703+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:48.560703+0000","last_undegraded":"2023-12-14T18:26:48.560703+0000","last_fullsized":"2023-12-14T18:26:48.560703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:33:22.836099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561628+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:26:48.561628+0000","last_peered":"2023-12-14T18:26:48.561628+0000","last_clean":"2023-12-14T18:26:48.561628+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:26:48.561628+0000","last_undegraded":"2023-12-14T18:26:48.561628+0000","last_fullsized":"2023-12-14T18:26:48.561628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:47.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561906+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:48.561906+0000","last_peered":"2023-12-14T18:26:48.561906+0000","last_clean":"2023-12-14T18:26:48.561906+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:48.561906+0000","last_undegraded":"2023-12-14T18:26:48.561906+0000","last_fullsized":"2023-12-14T18:26:48.561906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:00.359853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558750+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:48.558750+0000","last_peered":"2023-12-14T18:26:48.558750+0000","last_clean":"2023-12-14T18:26:48.558750+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:48.558750+0000","last_undegraded":"2023-12-14T18:26:48.558750+0000","last_fullsized":"2023-12-14T18:26:48.558750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:49.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568565+0000","last_change":"2023-12-14T18:26:48.568565+0000","last_active":"2023-12-14T18:26:48.568565+0000","last_peered":"2023-12-14T18:26:48.568565+0000","last_clean":"2023-12-14T18:26:48.568565+0000","last_became_active":"2023-12-14T18:26:48.568090+0000","last_became_peered":"2023-12-14T18:26:48.568090+0000","last_unstale":"2023-12-14T18:26:48.568565+0000","last_undegraded":"2023-12-14T18:26:48.568565+0000","last_fullsized":"2023-12-14T18:26:48.568565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:14:38.160855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563856+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:48.563856+0000","last_peered":"2023-12-14T18:26:48.563856+0000","last_clean":"2023-12-14T18:26:48.563856+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:48.563856+0000","last_undegraded":"2023-12-14T18:26:48.563856+0000","last_fullsized":"2023-12-14T18:26:48.563856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:12.189176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560853+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:48.560853+0000","last_peered":"2023-12-14T18:26:48.560853+0000","last_clean":"2023-12-14T18:26:48.560853+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:48.560853+0000","last_undegraded":"2023-12-14T18:26:48.560853+0000","last_fullsized":"2023-12-14T18:26:48.560853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:55:18.680896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562308+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:26:48.562308+0000","last_peered":"2023-12-14T18:26:48.562308+0000","last_clean":"2023-12-14T18:26:48.562308+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:26:48.562308+0000","last_undegraded":"2023-12-14T18:26:48.562308+0000","last_fullsized":"2023-12-14T18:26:48.562308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:05:51.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":223,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:38.641353+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:27:38.641353+0000","last_peered":"2023-12-14T18:27:38.641353+0000","last_clean":"2023-12-14T18:27:38.641353+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:27:38.641353+0000","last_undegraded":"2023-12-14T18:27:38.641353+0000","last_fullsized":"2023-12-14T18:27:38.641353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:33.081169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64447,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":139,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568542+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:48.568542+0000","last_peered":"2023-12-14T18:26:48.568542+0000","last_clean":"2023-12-14T18:26:48.568542+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:48.568542+0000","last_undegraded":"2023-12-14T18:26:48.568542+0000","last_fullsized":"2023-12-14T18:26:48.568542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:30.839862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564457+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:48.564457+0000","last_peered":"2023-12-14T18:26:48.564457+0000","last_clean":"2023-12-14T18:26:48.564457+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:48.564457+0000","last_undegraded":"2023-12-14T18:26:48.564457+0000","last_fullsized":"2023-12-14T18:26:48.564457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:27.839169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565422+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:48.565422+0000","last_peered":"2023-12-14T18:26:48.565422+0000","last_clean":"2023-12-14T18:26:48.565422+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:48.565422+0000","last_undegraded":"2023-12-14T18:26:48.565422+0000","last_fullsized":"2023-12-14T18:26:48.565422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:34.931404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562476+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:48.562476+0000","last_peered":"2023-12-14T18:26:48.562476+0000","last_clean":"2023-12-14T18:26:48.562476+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:48.562476+0000","last_undegraded":"2023-12-14T18:26:48.562476+0000","last_fullsized":"2023-12-14T18:26:48.562476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:45.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561959+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:48.561959+0000","last_peered":"2023-12-14T18:26:48.561959+0000","last_clean":"2023-12-14T18:26:48.561959+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:48.561959+0000","last_undegraded":"2023-12-14T18:26:48.561959+0000","last_fullsized":"2023-12-14T18:26:48.561959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:54.542913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565425+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:48.565425+0000","last_peered":"2023-12-14T18:26:48.565425+0000","last_clean":"2023-12-14T18:26:48.565425+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:48.565425+0000","last_undegraded":"2023-12-14T18:26:48.565425+0000","last_fullsized":"2023-12-14T18:26:48.565425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:46.938078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566091+0000","last_change":"2023-12-14T18:26:48.566091+0000","last_active":"2023-12-14T18:26:48.566091+0000","last_peered":"2023-12-14T18:26:48.566091+0000","last_clean":"2023-12-14T18:26:48.566091+0000","last_became_active":"2023-12-14T18:26:48.565714+0000","last_became_peered":"2023-12-14T18:26:48.565714+0000","last_unstale":"2023-12-14T18:26:48.566091+0000","last_undegraded":"2023-12-14T18:26:48.566091+0000","last_fullsized":"2023-12-14T18:26:48.566091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:48.478859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562658+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:26:48.562658+0000","last_peered":"2023-12-14T18:26:48.562658+0000","last_clean":"2023-12-14T18:26:48.562658+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:26:48.562658+0000","last_undegraded":"2023-12-14T18:26:48.562658+0000","last_fullsized":"2023-12-14T18:26:48.562658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:22.379528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:32.714881+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:27:32.714881+0000","last_peered":"2023-12-14T18:27:32.714881+0000","last_clean":"2023-12-14T18:27:32.714881+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:27:32.714881+0000","last_undegraded":"2023-12-14T18:27:32.714881+0000","last_fullsized":"2023-12-14T18:27:32.714881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:26:13.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561690+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:48.561690+0000","last_peered":"2023-12-14T18:26:48.561690+0000","last_clean":"2023-12-14T18:26:48.561690+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:48.561690+0000","last_undegraded":"2023-12-14T18:26:48.561690+0000","last_fullsized":"2023-12-14T18:26:48.561690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:10:17.702084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.649376+0000","last_change":"2023-12-14T18:26:48.649376+0000","last_active":"2023-12-14T18:26:48.649376+0000","last_peered":"2023-12-14T18:26:48.649376+0000","last_clean":"2023-12-14T18:26:48.649376+0000","last_became_active":"2023-12-14T18:26:48.648882+0000","last_became_peered":"2023-12-14T18:26:48.648882+0000","last_unstale":"2023-12-14T18:26:48.649376+0000","last_undegraded":"2023-12-14T18:26:48.649376+0000","last_fullsized":"2023-12-14T18:26:48.649376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:12.045140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566931+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:48.566931+0000","last_peered":"2023-12-14T18:26:48.566931+0000","last_clean":"2023-12-14T18:26:48.566931+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:48.566931+0000","last_undegraded":"2023-12-14T18:26:48.566931+0000","last_fullsized":"2023-12-14T18:26:48.566931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:47.382142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.572300+0000","last_change":"2023-12-14T18:26:48.572300+0000","last_active":"2023-12-14T18:26:48.572300+0000","last_peered":"2023-12-14T18:26:48.572300+0000","last_clean":"2023-12-14T18:26:48.572300+0000","last_became_active":"2023-12-14T18:26:48.571895+0000","last_became_peered":"2023-12-14T18:26:48.571895+0000","last_unstale":"2023-12-14T18:26:48.572300+0000","last_undegraded":"2023-12-14T18:26:48.572300+0000","last_fullsized":"2023-12-14T18:26:48.572300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:40:54.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566446+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:48.566446+0000","last_peered":"2023-12-14T18:26:48.566446+0000","last_clean":"2023-12-14T18:26:48.566446+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:48.566446+0000","last_undegraded":"2023-12-14T18:26:48.566446+0000","last_fullsized":"2023-12-14T18:26:48.566446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:44.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570871+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:48.570871+0000","last_peered":"2023-12-14T18:26:48.570871+0000","last_clean":"2023-12-14T18:26:48.570871+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:48.570871+0000","last_undegraded":"2023-12-14T18:26:48.570871+0000","last_fullsized":"2023-12-14T18:26:48.570871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:28.018194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561975+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:48.561975+0000","last_peered":"2023-12-14T18:26:48.561975+0000","last_clean":"2023-12-14T18:26:48.561975+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:48.561975+0000","last_undegraded":"2023-12-14T18:26:48.561975+0000","last_fullsized":"2023-12-14T18:26:48.561975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:37.576788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566649+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:48.566649+0000","last_peered":"2023-12-14T18:26:48.566649+0000","last_clean":"2023-12-14T18:26:48.566649+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:48.566649+0000","last_undegraded":"2023-12-14T18:26:48.566649+0000","last_fullsized":"2023-12-14T18:26:48.566649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:37.853928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562990+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:26:48.562990+0000","last_peered":"2023-12-14T18:26:48.562990+0000","last_clean":"2023-12-14T18:26:48.562990+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:26:48.562990+0000","last_undegraded":"2023-12-14T18:26:48.562990+0000","last_fullsized":"2023-12-14T18:26:48.562990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:32.215966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647772+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:26:48.647772+0000","last_peered":"2023-12-14T18:26:48.647772+0000","last_clean":"2023-12-14T18:26:48.647772+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:26:48.647772+0000","last_undegraded":"2023-12-14T18:26:48.647772+0000","last_fullsized":"2023-12-14T18:26:48.647772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:51:12.578633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563700+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:48.563700+0000","last_peered":"2023-12-14T18:26:48.563700+0000","last_clean":"2023-12-14T18:26:48.563700+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:48.563700+0000","last_undegraded":"2023-12-14T18:26:48.563700+0000","last_fullsized":"2023-12-14T18:26:48.563700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:58.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561937+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:48.561937+0000","last_peered":"2023-12-14T18:26:48.561937+0000","last_clean":"2023-12-14T18:26:48.561937+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:48.561937+0000","last_undegraded":"2023-12-14T18:26:48.561937+0000","last_fullsized":"2023-12-14T18:26:48.561937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:21.783482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647707+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:26:48.647707+0000","last_peered":"2023-12-14T18:26:48.647707+0000","last_clean":"2023-12-14T18:26:48.647707+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:26:48.647707+0000","last_undegraded":"2023-12-14T18:26:48.647707+0000","last_fullsized":"2023-12-14T18:26:48.647707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:26:41.629317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563228+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:48.563228+0000","last_peered":"2023-12-14T18:26:48.563228+0000","last_clean":"2023-12-14T18:26:48.563228+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:48.563228+0000","last_undegraded":"2023-12-14T18:26:48.563228+0000","last_fullsized":"2023-12-14T18:26:48.563228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:41:45.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566971+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:48.566971+0000","last_peered":"2023-12-14T18:26:48.566971+0000","last_clean":"2023-12-14T18:26:48.566971+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:48.566971+0000","last_undegraded":"2023-12-14T18:26:48.566971+0000","last_fullsized":"2023-12-14T18:26:48.566971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:54.638247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:34.717332+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:27:34.717332+0000","last_peered":"2023-12-14T18:27:34.717332+0000","last_clean":"2023-12-14T18:27:34.717332+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:27:34.717332+0000","last_undegraded":"2023-12-14T18:27:34.717332+0000","last_fullsized":"2023-12-14T18:27:34.717332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:07:36.295981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562086+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:48.562086+0000","last_peered":"2023-12-14T18:26:48.562086+0000","last_clean":"2023-12-14T18:26:48.562086+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:48.562086+0000","last_undegraded":"2023-12-14T18:26:48.562086+0000","last_fullsized":"2023-12-14T18:26:48.562086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:00.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559419+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:48.559419+0000","last_peered":"2023-12-14T18:26:48.559419+0000","last_clean":"2023-12-14T18:26:48.559419+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:48.559419+0000","last_undegraded":"2023-12-14T18:26:48.559419+0000","last_fullsized":"2023-12-14T18:26:48.559419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:01.825894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564209+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:48.564209+0000","last_peered":"2023-12-14T18:26:48.564209+0000","last_clean":"2023-12-14T18:26:48.564209+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:48.564209+0000","last_undegraded":"2023-12-14T18:26:48.564209+0000","last_fullsized":"2023-12-14T18:26:48.564209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:27:58.179129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561521+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:48.561521+0000","last_peered":"2023-12-14T18:26:48.561521+0000","last_clean":"2023-12-14T18:26:48.561521+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:48.561521+0000","last_undegraded":"2023-12-14T18:26:48.561521+0000","last_fullsized":"2023-12-14T18:26:48.561521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:22.740418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566535+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:48.566535+0000","last_peered":"2023-12-14T18:26:48.566535+0000","last_clean":"2023-12-14T18:26:48.566535+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:48.566535+0000","last_undegraded":"2023-12-14T18:26:48.566535+0000","last_fullsized":"2023-12-14T18:26:48.566535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:54:48.048416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562125+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:48.562125+0000","last_peered":"2023-12-14T18:26:48.562125+0000","last_clean":"2023-12-14T18:26:48.562125+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:48.562125+0000","last_undegraded":"2023-12-14T18:26:48.562125+0000","last_fullsized":"2023-12-14T18:26:48.562125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:36:49.964293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561575+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:48.561575+0000","last_peered":"2023-12-14T18:26:48.561575+0000","last_clean":"2023-12-14T18:26:48.561575+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:48.561575+0000","last_undegraded":"2023-12-14T18:26:48.561575+0000","last_fullsized":"2023-12-14T18:26:48.561575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:20:52.709127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562632+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:48.562632+0000","last_peered":"2023-12-14T18:26:48.562632+0000","last_clean":"2023-12-14T18:26:48.562632+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:48.562632+0000","last_undegraded":"2023-12-14T18:26:48.562632+0000","last_fullsized":"2023-12-14T18:26:48.562632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:26.536039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568593+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:48.568593+0000","last_peered":"2023-12-14T18:26:48.568593+0000","last_clean":"2023-12-14T18:26:48.568593+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:48.568593+0000","last_undegraded":"2023-12-14T18:26:48.568593+0000","last_fullsized":"2023-12-14T18:26:48.568593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:59:51.600710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564002+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:48.564002+0000","last_peered":"2023-12-14T18:26:48.564002+0000","last_clean":"2023-12-14T18:26:48.564002+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:48.564002+0000","last_undegraded":"2023-12-14T18:26:48.564002+0000","last_fullsized":"2023-12-14T18:26:48.564002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:06.768114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559014+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:48.559014+0000","last_peered":"2023-12-14T18:26:48.559014+0000","last_clean":"2023-12-14T18:26:48.559014+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:48.559014+0000","last_undegraded":"2023-12-14T18:26:48.559014+0000","last_fullsized":"2023-12-14T18:26:48.559014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:32.285691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.577550+0000","last_change":"2023-12-14T18:26:48.577550+0000","last_active":"2023-12-14T18:26:48.577550+0000","last_peered":"2023-12-14T18:26:48.577550+0000","last_clean":"2023-12-14T18:26:48.577550+0000","last_became_active":"2023-12-14T18:26:48.569607+0000","last_became_peered":"2023-12-14T18:26:48.569607+0000","last_unstale":"2023-12-14T18:26:48.577550+0000","last_undegraded":"2023-12-14T18:26:48.577550+0000","last_fullsized":"2023-12-14T18:26:48.577550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:23.766724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559798+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:48.559798+0000","last_peered":"2023-12-14T18:26:48.559798+0000","last_clean":"2023-12-14T18:26:48.559798+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:48.559798+0000","last_undegraded":"2023-12-14T18:26:48.559798+0000","last_fullsized":"2023-12-14T18:26:48.559798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:29.491959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563921+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:26:48.563921+0000","last_peered":"2023-12-14T18:26:48.563921+0000","last_clean":"2023-12-14T18:26:48.563921+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:26:48.563921+0000","last_undegraded":"2023-12-14T18:26:48.563921+0000","last_fullsized":"2023-12-14T18:26:48.563921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:12.984642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563541+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:48.563541+0000","last_peered":"2023-12-14T18:26:48.563541+0000","last_clean":"2023-12-14T18:26:48.563541+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:48.563541+0000","last_undegraded":"2023-12-14T18:26:48.563541+0000","last_fullsized":"2023-12-14T18:26:48.563541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:56.278549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557904+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:48.557904+0000","last_peered":"2023-12-14T18:26:48.557904+0000","last_clean":"2023-12-14T18:26:48.557904+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:48.557904+0000","last_undegraded":"2023-12-14T18:26:48.557904+0000","last_fullsized":"2023-12-14T18:26:48.557904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:36:53.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563087+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:48.563087+0000","last_peered":"2023-12-14T18:26:48.563087+0000","last_clean":"2023-12-14T18:26:48.563087+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:48.563087+0000","last_undegraded":"2023-12-14T18:26:48.563087+0000","last_fullsized":"2023-12-14T18:26:48.563087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:19.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559959+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:48.559959+0000","last_peered":"2023-12-14T18:26:48.559959+0000","last_clean":"2023-12-14T18:26:48.559959+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:48.559959+0000","last_undegraded":"2023-12-14T18:26:48.559959+0000","last_fullsized":"2023-12-14T18:26:48.559959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:05.103613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566175+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:48.566175+0000","last_peered":"2023-12-14T18:26:48.566175+0000","last_clean":"2023-12-14T18:26:48.566175+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:48.566175+0000","last_undegraded":"2023-12-14T18:26:48.566175+0000","last_fullsized":"2023-12-14T18:26:48.566175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:21.311660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562945+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:26:48.562945+0000","last_peered":"2023-12-14T18:26:48.562945+0000","last_clean":"2023-12-14T18:26:48.562945+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:26:48.562945+0000","last_undegraded":"2023-12-14T18:26:48.562945+0000","last_fullsized":"2023-12-14T18:26:48.562945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:04:53.461295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558908+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:48.558908+0000","last_peered":"2023-12-14T18:26:48.558908+0000","last_clean":"2023-12-14T18:26:48.558908+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:48.558908+0000","last_undegraded":"2023-12-14T18:26:48.558908+0000","last_fullsized":"2023-12-14T18:26:48.558908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:04:46.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561591+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:48.561591+0000","last_peered":"2023-12-14T18:26:48.561591+0000","last_clean":"2023-12-14T18:26:48.561591+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:48.561591+0000","last_undegraded":"2023-12-14T18:26:48.561591+0000","last_fullsized":"2023-12-14T18:26:48.561591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:52:50.350437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562857+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:48.562857+0000","last_peered":"2023-12-14T18:26:48.562857+0000","last_clean":"2023-12-14T18:26:48.562857+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:48.562857+0000","last_undegraded":"2023-12-14T18:26:48.562857+0000","last_fullsized":"2023-12-14T18:26:48.562857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:22.082703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565524+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:26:48.565524+0000","last_peered":"2023-12-14T18:26:48.565524+0000","last_clean":"2023-12-14T18:26:48.565524+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:26:48.565524+0000","last_undegraded":"2023-12-14T18:26:48.565524+0000","last_fullsized":"2023-12-14T18:26:48.565524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:51:55.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559293+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:48.559293+0000","last_peered":"2023-12-14T18:26:48.559293+0000","last_clean":"2023-12-14T18:26:48.559293+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:48.559293+0000","last_undegraded":"2023-12-14T18:26:48.559293+0000","last_fullsized":"2023-12-14T18:26:48.559293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:17:28.674237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563263+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:48.563263+0000","last_peered":"2023-12-14T18:26:48.563263+0000","last_clean":"2023-12-14T18:26:48.563263+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:48.563263+0000","last_undegraded":"2023-12-14T18:26:48.563263+0000","last_fullsized":"2023-12-14T18:26:48.563263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:08.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.578387+0000","last_change":"2023-12-14T18:26:48.577964+0000","last_active":"2023-12-14T18:26:48.578387+0000","last_peered":"2023-12-14T18:26:48.578387+0000","last_clean":"2023-12-14T18:26:48.578387+0000","last_became_active":"2023-12-14T18:26:48.569637+0000","last_became_peered":"2023-12-14T18:26:48.569637+0000","last_unstale":"2023-12-14T18:26:48.578387+0000","last_undegraded":"2023-12-14T18:26:48.578387+0000","last_fullsized":"2023-12-14T18:26:48.578387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:31:31.614815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562048+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:48.562048+0000","last_peered":"2023-12-14T18:26:48.562048+0000","last_clean":"2023-12-14T18:26:48.562048+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:48.562048+0000","last_undegraded":"2023-12-14T18:26:48.562048+0000","last_fullsized":"2023-12-14T18:26:48.562048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:43.573208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559867+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:48.559867+0000","last_peered":"2023-12-14T18:26:48.559867+0000","last_clean":"2023-12-14T18:26:48.559867+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:48.559867+0000","last_undegraded":"2023-12-14T18:26:48.559867+0000","last_fullsized":"2023-12-14T18:26:48.559867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:15:04.165007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.574468+0000","last_change":"2023-12-14T18:26:48.574083+0000","last_active":"2023-12-14T18:26:48.574468+0000","last_peered":"2023-12-14T18:26:48.574468+0000","last_clean":"2023-12-14T18:26:48.574468+0000","last_became_active":"2023-12-14T18:26:48.565205+0000","last_became_peered":"2023-12-14T18:26:48.565205+0000","last_unstale":"2023-12-14T18:26:48.574468+0000","last_undegraded":"2023-12-14T18:26:48.574468+0000","last_fullsized":"2023-12-14T18:26:48.574468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:19.409191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562445+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:48.562445+0000","last_peered":"2023-12-14T18:26:48.562445+0000","last_clean":"2023-12-14T18:26:48.562445+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:48.562445+0000","last_undegraded":"2023-12-14T18:26:48.562445+0000","last_fullsized":"2023-12-14T18:26:48.562445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:37.001669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557585+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:48.557585+0000","last_peered":"2023-12-14T18:26:48.557585+0000","last_clean":"2023-12-14T18:26:48.557585+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:48.557585+0000","last_undegraded":"2023-12-14T18:26:48.557585+0000","last_fullsized":"2023-12-14T18:26:48.557585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:54.017877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564140+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:48.564140+0000","last_peered":"2023-12-14T18:26:48.564140+0000","last_clean":"2023-12-14T18:26:48.564140+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:48.564140+0000","last_undegraded":"2023-12-14T18:26:48.564140+0000","last_fullsized":"2023-12-14T18:26:48.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:46:47.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562523+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:48.562523+0000","last_peered":"2023-12-14T18:26:48.562523+0000","last_clean":"2023-12-14T18:26:48.562523+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:48.562523+0000","last_undegraded":"2023-12-14T18:26:48.562523+0000","last_fullsized":"2023-12-14T18:26:48.562523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:52:04.129230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566378+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:48.566378+0000","last_peered":"2023-12-14T18:26:48.566378+0000","last_clean":"2023-12-14T18:26:48.566378+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:48.566378+0000","last_undegraded":"2023-12-14T18:26:48.566378+0000","last_fullsized":"2023-12-14T18:26:48.566378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:08:14.934634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564930+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:48.564930+0000","last_peered":"2023-12-14T18:26:48.564930+0000","last_clean":"2023-12-14T18:26:48.564930+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:48.564930+0000","last_undegraded":"2023-12-14T18:26:48.564930+0000","last_fullsized":"2023-12-14T18:26:48.564930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:28.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562150+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:48.562150+0000","last_peered":"2023-12-14T18:26:48.562150+0000","last_clean":"2023-12-14T18:26:48.562150+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:48.562150+0000","last_undegraded":"2023-12-14T18:26:48.562150+0000","last_fullsized":"2023-12-14T18:26:48.562150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:43:39.281554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562009+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:48.562009+0000","last_peered":"2023-12-14T18:26:48.562009+0000","last_clean":"2023-12-14T18:26:48.562009+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:48.562009+0000","last_undegraded":"2023-12-14T18:26:48.562009+0000","last_fullsized":"2023-12-14T18:26:48.562009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:49:45.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564454+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:26:48.564454+0000","last_peered":"2023-12-14T18:26:48.564454+0000","last_clean":"2023-12-14T18:26:48.564454+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:26:48.564454+0000","last_undegraded":"2023-12-14T18:26:48.564454+0000","last_fullsized":"2023-12-14T18:26:48.564454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:16.913094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563425+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:26:48.563425+0000","last_peered":"2023-12-14T18:26:48.563425+0000","last_clean":"2023-12-14T18:26:48.563425+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:26:48.563425+0000","last_undegraded":"2023-12-14T18:26:48.563425+0000","last_fullsized":"2023-12-14T18:26:48.563425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:54.437297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557742+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:48.557742+0000","last_peered":"2023-12-14T18:26:48.557742+0000","last_clean":"2023-12-14T18:26:48.557742+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:48.557742+0000","last_undegraded":"2023-12-14T18:26:48.557742+0000","last_fullsized":"2023-12-14T18:26:48.557742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:35.196672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564064+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:48.564064+0000","last_peered":"2023-12-14T18:26:48.564064+0000","last_clean":"2023-12-14T18:26:48.564064+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:48.564064+0000","last_undegraded":"2023-12-14T18:26:48.564064+0000","last_fullsized":"2023-12-14T18:26:48.564064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:27.906838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562103+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:48.562103+0000","last_peered":"2023-12-14T18:26:48.562103+0000","last_clean":"2023-12-14T18:26:48.562103+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:48.562103+0000","last_undegraded":"2023-12-14T18:26:48.562103+0000","last_fullsized":"2023-12-14T18:26:48.562103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:44:51.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566964+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:26:48.566964+0000","last_peered":"2023-12-14T18:26:48.566964+0000","last_clean":"2023-12-14T18:26:48.566964+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:26:48.566964+0000","last_undegraded":"2023-12-14T18:26:48.566964+0000","last_fullsized":"2023-12-14T18:26:48.566964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:07:32.112350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559871+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:48.559871+0000","last_peered":"2023-12-14T18:26:48.559871+0000","last_clean":"2023-12-14T18:26:48.559871+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:48.559871+0000","last_undegraded":"2023-12-14T18:26:48.559871+0000","last_fullsized":"2023-12-14T18:26:48.559871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:36:21.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566829+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:48.566829+0000","last_peered":"2023-12-14T18:26:48.566829+0000","last_clean":"2023-12-14T18:26:48.566829+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:48.566829+0000","last_undegraded":"2023-12-14T18:26:48.566829+0000","last_fullsized":"2023-12-14T18:26:48.566829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:17.885326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559062+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:48.559062+0000","last_peered":"2023-12-14T18:26:48.559062+0000","last_clean":"2023-12-14T18:26:48.559062+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:48.559062+0000","last_undegraded":"2023-12-14T18:26:48.559062+0000","last_fullsized":"2023-12-14T18:26:48.559062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:28:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560244+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:48.560244+0000","last_peered":"2023-12-14T18:26:48.560244+0000","last_clean":"2023-12-14T18:26:48.560244+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:48.560244+0000","last_undegraded":"2023-12-14T18:26:48.560244+0000","last_fullsized":"2023-12-14T18:26:48.560244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:54.715861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562903+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:48.562903+0000","last_peered":"2023-12-14T18:26:48.562903+0000","last_clean":"2023-12-14T18:26:48.562903+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:48.562903+0000","last_undegraded":"2023-12-14T18:26:48.562903+0000","last_fullsized":"2023-12-14T18:26:48.562903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:54:48.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562082+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:48.562082+0000","last_peered":"2023-12-14T18:26:48.562082+0000","last_clean":"2023-12-14T18:26:48.562082+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:48.562082+0000","last_undegraded":"2023-12-14T18:26:48.562082+0000","last_fullsized":"2023-12-14T18:26:48.562082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:40:01.866310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68351,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":469,"num_read_kb":305,"num_write":160,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514527,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739028,"statfs":{"total":95995035648,"available":95988764672,"internally_reserved":0,"allocated":954368,"data_stored":484012,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68400000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.318,"5min":0.311,"15min":0.311},"max":{"1min":0.69399999999999995,"5min":0.751,"15min":0.751},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":1.268,"15min":1.268},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90000000000000002,"5min":1.097,"15min":1.097},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.50800000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.79500000000000004,"5min":1.2,"15min":1.2},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.878,"15min":0.878},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.88200000000000001,"5min":1.075,"15min":1.075},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.05,"15min":1.05},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625}]},{"osd":9,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.86199999999999999,"5min":1.115,"15min":1.115},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.68000000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0009999999999999,"5min":1.389,"15min":1.389},"last":0.76300000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.628,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98599999999999999,"5min":1.147,"15min":1.147},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92800000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.752}]}]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6584,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738568,"statfs":{"total":95995035648,"available":95988293632,"internally_reserved":0,"allocated":1097728,"data_stored":1135398,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45800000000000002,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.748,"5min":0.749,"15min":0.749},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.76000000000000001,"15min":0.78900000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78500000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.80300000000000005,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.628},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.56200000000000006}]},{"osd":9,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":1.252,"15min":1.252},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.93600000000000005,"15min":0.97999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85699999999999998,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Dec 14 18:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88500000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90400000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68500000000000005}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":974848,"data_stored":488344,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.437}]},{"osd":1,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.504},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38,"15min":0.38},"max":{"1min":1.01,"5min":1.028,"15min":1.028},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.081,"15min":1.081},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.036,"15min":1.036},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1200000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0189999999999999,"5min":1.123,"15min":1.123},"last":0.79800000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.81799999999999995,"15min":0.83099999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0840000000000001,"5min":1.105,"15min":1.105},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84299999999999997,"15min":0.85699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0329999999999999,"5min":1.121,"15min":1.121},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6460,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738692,"statfs":{"total":95995035648,"available":95988420608,"internally_reserved":0,"allocated":970752,"data_stored":484168,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78400000000000003,"5min":0.875,"15min":0.875},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.48799999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.83799999999999997,"5min":0.879,"15min":0.879},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.871},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78800000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71399999999999997,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95899999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.66900000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.90400000000000003}]},{"osd":9,"last update":"Thu Dec 14 18:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.628,"5min":0.502,"15min":0.502},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.96899999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.68500000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.004}]}]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":1079376,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.34899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72799999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.53000000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73399999999999999,"5min":1.8,"15min":1.8},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.626}]},{"osd":5,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78900000000000003,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.503,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.58199999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83799999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84099999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82299999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85799999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593777,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6440,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738712,"statfs":{"total":95995035648,"available":95988441088,"internally_reserved":0,"allocated":950272,"data_stored":479424,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.34599999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.435}]},{"osd":1,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64200000000000002,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.56100000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.88200000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.57899999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.127,"5min":1.133,"15min":1.133},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":539450,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79000000000000004,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":5.0800000000000001,"15min":5.0800000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.53000000000000003,"15min":0.47999999999999998},"min":{"1min":0.40200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82799999999999996,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.55600000000000005},"min":{"1min":0.46200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82899999999999996,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45200000000000001,"15min":0.41099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44}]},{"osd":5,"last update":"Thu Dec 14 18:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61599999999999999,"15min":0.58199999999999996},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82899999999999996,"5min":1.657,"15min":1.657},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.58599999999999997},"min":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.872,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48399999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.628},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92500000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71799999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.59299999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.70899999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.78800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":480812,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.622,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94199999999999995,"5min":1.091,"15min":1.091},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.83499999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.079,"15min":1.079},"last":0.877}]},{"osd":9,"last update":"Thu Dec 14 18:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.82499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":0.72399999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.55300000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.258,"15min":1.258},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.077,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86599999999999999,"15min":0.871},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.125,"5min":1.2,"15min":1.2},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88200000000000001,"15min":0.877},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.748}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7540,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737612,"statfs":{"total":95995035648,"available":95987314688,"internally_reserved":0,"allocated":1028096,"data_stored":539607,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.484,"15min":1.484},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.52300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84899999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.47799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.83999999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.443},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81000000000000005,"5min":1.514,"15min":1.514},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78300000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57099999999999995},"min":{"1min":0.44400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.88700000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60299999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.67400000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.54700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79700000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":958464,"data_stored":479345,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.43099999999999999,"15min":0.39100000000000001},"min":{"1min":0.29699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.41899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.63600000000000001,"15min":0.55800000000000005},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.5800000000000001,"5min":6.5800000000000001,"15min":6.5800000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61699999999999999,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.67200000000000004,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.5419999999999998,"5min":6.5419999999999998,"15min":6.5419999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:27:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64600000000000002,"15min":0.61299999999999999},"min":{"1min":0.50600000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.621}]},{"osd":5,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.68200000000000005,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.79600000000000004,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.617,"5min":6.617,"15min":6.617},"last":0.73699999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":6.7249999999999996,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.753},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85799999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.72799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986232,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":484990,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.379,"15min":0.34499999999999997},"min":{"1min":0.29699999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66400000000000003,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.315},{"interface":"front","average":{"1min":0.44,"5min":0.39200000000000002,"15min":0.36399999999999999},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64500000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.42299999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78900000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.442},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":1.121,"15min":1.121},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.376},{"interface":"front","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85999999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.879,"15min":0.879},"last":0.40600000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":1.161,"15min":1.161},"last":0.59899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.56999999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85699999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53300000000000003}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58799999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822709,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":1024000,"data_stored":1075285,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63800000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66600000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.314,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":4.7290000000000001,"5min":4.7290000000000001,"15min":4.7290000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.621,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.391,"5min":6.391,"15min":6.391},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.66800000000000004,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.3300000000000001,"5min":6.3300000000000001,"15min":6.3300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.66600000000000004,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Dec 14 18:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81200000000000006,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.751}]},{"osd":8,"last update":"Thu Dec 14 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.84599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.51200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58999999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.71099999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.58599999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:27:40.428 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-12-14T18:27:40.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 -- 172.21.15.31:0/1823980323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500ff110 msgr2=0x7fba500ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 --2- 172.21.15.31:0/1823980323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500ff110 0x7fba500ff520 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fba40009990 tx=0x7fba40009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 -- 172.21.15.31:0/1823980323 shutdown_connections 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 --2- 172.21.15.31:0/1823980323 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba50100c90 0x7fba50107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 --2- 172.21.15.31:0/1823980323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba500ff110 0x7fba500ff520 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 --2- 172.21.15.31:0/1823980323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 0x7fba50100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.600+0000 7fba56207700 1 -- 172.21.15.31:0/1823980323 >> 172.21.15.31:0/1823980323 conn(0x7fba500fa6f0 msgr2=0x7fba500fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.601+0000 7fba56207700 1 -- 172.21.15.31:0/1823980323 shutdown_connections 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.601+0000 7fba56207700 1 -- 172.21.15.31:0/1823980323 wait complete. 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.601+0000 7fba56207700 1 Processor -- start 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.601+0000 7fba56207700 1 -- start start 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba56207700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba500ff110 0x7fba500689a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba56207700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 0x7fba5019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba56207700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 0x7fba5019d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba56207700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5019db20 con 0x7fba50100c90 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba56207700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5019dc90 con 0x7fba500ff110 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba56207700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5019de50 con 0x7fba50100300 2023-12-14T18:27:40.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 0x7fba5019d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 0x7fba5019d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53162/0 (socket says 172.21.15.31:53162) 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 -- 172.21.15.31:0/2371216749 learned_addr learned my addr 172.21.15.31:0/2371216749 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4f7fe700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba500ff110 0x7fba500689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4effd700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 0x7fba5019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 -- 172.21.15.31:0/2371216749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 msgr2=0x7fba5019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 0x7fba5019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 -- 172.21.15.31:0/2371216749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba500ff110 msgr2=0x7fba500689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba500ff110 0x7fba500689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 -- 172.21.15.31:0/2371216749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba400095f0 con 0x7fba50100c90 2023-12-14T18:27:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4effd700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 0x7fba5019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4f7fe700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba500ff110 0x7fba500689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.602+0000 7fba4ffff700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 0x7fba5019d380 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fba4800ef30 tx=0x7fba4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.603+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48018070 con 0x7fba50100c90 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.603+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba4800f040 con 0x7fba50100c90 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.603+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5010aad0 con 0x7fba50100c90 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.603+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4800cdb0 con 0x7fba50100c90 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.603+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5010a940 con 0x7fba50100c90 2023-12-14T18:27:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.604+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba50106090 con 0x7fba50100c90 2023-12-14T18:27:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.605+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fba48003a00 con 0x7fba50100c90 2023-12-14T18:27:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.606+0000 7fba4cff9700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba380a11e0 0x7fba380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.606+0000 7fba4f7fe700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba380a11e0 0x7fba380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.606+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba480cd2b0 con 0x7fba50100c90 2023-12-14T18:27:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.606+0000 7fba4f7fe700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba380a11e0 0x7fba380a3690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b2e0 tx=0x7fba40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.609+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba480966c0 con 0x7fba50100c90 2023-12-14T18:27:40.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:40 smithi031 ceph-mon[42085]: pgmap v283: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:27:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:40 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2761074991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:40.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:40 smithi031 ceph-mon[42085]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:40.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.816+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba50061aa0 con 0x7fba380a11e0 2023-12-14T18:27:40.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.829+0000 7fba4cff9700 1 -- 172.21.15.31:0/2371216749 <== mgr.14152 v2:172.21.15.31:6800/2688246057 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427137 (secure 0 0 0) 0x7fba50061aa0 con 0x7fba380a11e0 2023-12-14T18:27:40.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba380a11e0 msgr2=0x7fba380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba380a11e0 0x7fba380a3690 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fba4000b2e0 tx=0x7fba40005fb0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 msgr2=0x7fba5019d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 0x7fba5019d380 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fba4800ef30 tx=0x7fba4800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 shutdown_connections 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba50100c90 0x7fba5019d380 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fba500ff110 0x7fba500689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fba380a11e0 0x7fba380a3690 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 --2- 172.21.15.31:0/2371216749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba50100300 0x7fba5019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 >> 172.21.15.31:0/2371216749 conn(0x7fba500fa6f0 msgr2=0x7fba500fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 shutdown_connections 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:40.833+0000 7fba56207700 1 -- 172.21.15.31:0/2371216749 wait complete. 2023-12-14T18:27:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-12-14T18:27:40.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:40 smithi204 ceph-mon[43866]: pgmap v283: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:27:40.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:40 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2761074991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:40.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:40 smithi204 ceph-mon[43866]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:40.849 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2023-12-14T18:27:39.499954+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658965,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":605,"num_read_kb":424,"num_write":343,"num_write_kb":3059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1182282,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80804,"kb_used_data":11652,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124861020,"statfs":{"total":1151940427776,"available":1151857684480,"internally_reserved":0,"allocated":11931648,"data_stored":7750211,"data_compressed":319020,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4640,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":41,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003489"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562794+0000","last_change":"2023-12-14T18:25:55.308261+0000","last_active":"2023-12-14T18:26:48.562794+0000","last_peered":"2023-12-14T18:26:48.562794+0000","last_clean":"2023-12-14T18:26:48.562794+0000","last_became_active":"2023-12-14T18:25:55.307704+0000","last_became_peered":"2023-12-14T18:25:55.307704+0000","last_unstale":"2023-12-14T18:26:48.562794+0000","last_undegraded":"2023-12-14T18:26:48.562794+0000","last_fullsized":"2023-12-14T18:26:48.562794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:33:46.076524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557880+0000","last_change":"2023-12-14T18:25:55.315031+0000","last_active":"2023-12-14T18:26:48.557880+0000","last_peered":"2023-12-14T18:26:48.557880+0000","last_clean":"2023-12-14T18:26:48.557880+0000","last_became_active":"2023-12-14T18:25:55.314756+0000","last_became_peered":"2023-12-14T18:25:55.314756+0000","last_unstale":"2023-12-14T18:26:48.557880+0000","last_undegraded":"2023-12-14T18:26:48.557880+0000","last_fullsized":"2023-12-14T18:26:48.557880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:10.369082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557355+0000","last_change":"2023-12-14T18:25:55.324556+0000","last_active":"2023-12-14T18:26:48.557355+0000","last_peered":"2023-12-14T18:26:48.557355+0000","last_clean":"2023-12-14T18:26:48.557355+0000","last_became_active":"2023-12-14T18:25:55.324212+0000","last_became_peered":"2023-12-14T18:25:55.324212+0000","last_unstale":"2023-12-14T18:26:48.557355+0000","last_undegraded":"2023-12-14T18:26:48.557355+0000","last_fullsized":"2023-12-14T18:26:48.557355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:23.331965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563177+0000","last_change":"2023-12-14T18:25:55.315470+0000","last_active":"2023-12-14T18:26:48.563177+0000","last_peered":"2023-12-14T18:26:48.563177+0000","last_clean":"2023-12-14T18:26:48.563177+0000","last_became_active":"2023-12-14T18:25:55.315119+0000","last_became_peered":"2023-12-14T18:25:55.315119+0000","last_unstale":"2023-12-14T18:26:48.563177+0000","last_undegraded":"2023-12-14T18:26:48.563177+0000","last_fullsized":"2023-12-14T18:26:48.563177+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:01.529418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567124+0000","last_change":"2023-12-14T18:25:55.316513+0000","last_active":"2023-12-14T18:26:48.567124+0000","last_peered":"2023-12-14T18:26:48.567124+0000","last_clean":"2023-12-14T18:26:48.567124+0000","last_became_active":"2023-12-14T18:25:55.316183+0000","last_became_peered":"2023-12-14T18:25:55.316183+0000","last_unstale":"2023-12-14T18:26:48.567124+0000","last_undegraded":"2023-12-14T18:26:48.567124+0000","last_fullsized":"2023-12-14T18:26:48.567124+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:53:32.988966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647501+0000","last_change":"2023-12-14T18:25:55.317844+0000","last_active":"2023-12-14T18:26:48.647501+0000","last_peered":"2023-12-14T18:26:48.647501+0000","last_clean":"2023-12-14T18:26:48.647501+0000","last_became_active":"2023-12-14T18:25:55.317278+0000","last_became_peered":"2023-12-14T18:25:55.317278+0000","last_unstale":"2023-12-14T18:26:48.647501+0000","last_undegraded":"2023-12-14T18:26:48.647501+0000","last_fullsized":"2023-12-14T18:26:48.647501+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:00:59.230921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561700+0000","last_change":"2023-12-14T18:25:55.312655+0000","last_active":"2023-12-14T18:26:48.561700+0000","last_peered":"2023-12-14T18:26:48.561700+0000","last_clean":"2023-12-14T18:26:48.561700+0000","last_became_active":"2023-12-14T18:25:55.312353+0000","last_became_peered":"2023-12-14T18:25:55.312353+0000","last_unstale":"2023-12-14T18:26:48.561700+0000","last_undegraded":"2023-12-14T18:26:48.561700+0000","last_fullsized":"2023-12-14T18:26:48.561700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:47.143609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559287+0000","last_change":"2023-12-14T18:25:55.320832+0000","last_active":"2023-12-14T18:26:48.559287+0000","last_peered":"2023-12-14T18:26:48.559287+0000","last_clean":"2023-12-14T18:26:48.559287+0000","last_became_active":"2023-12-14T18:25:55.320031+0000","last_became_peered":"2023-12-14T18:25:55.320031+0000","last_unstale":"2023-12-14T18:26:48.559287+0000","last_undegraded":"2023-12-14T18:26:48.559287+0000","last_fullsized":"2023-12-14T18:26:48.559287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:13:56.460051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560906+0000","last_change":"2023-12-14T18:25:55.306801+0000","last_active":"2023-12-14T18:26:48.560906+0000","last_peered":"2023-12-14T18:26:48.560906+0000","last_clean":"2023-12-14T18:26:48.560906+0000","last_became_active":"2023-12-14T18:25:55.306373+0000","last_became_peered":"2023-12-14T18:25:55.306373+0000","last_unstale":"2023-12-14T18:26:48.560906+0000","last_undegraded":"2023-12-14T18:26:48.560906+0000","last_fullsized":"2023-12-14T18:26:48.560906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:38:42.914156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647960+0000","last_change":"2023-12-14T18:25:55.317508+0000","last_active":"2023-12-14T18:26:48.647960+0000","last_peered":"2023-12-14T18:26:48.647960+0000","last_clean":"2023-12-14T18:26:48.647960+0000","last_became_active":"2023-12-14T18:25:55.317176+0000","last_became_peered":"2023-12-14T18:25:55.317176+0000","last_unstale":"2023-12-14T18:26:48.647960+0000","last_undegraded":"2023-12-14T18:26:48.647960+0000","last_fullsized":"2023-12-14T18:26:48.647960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:09:41.346067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562267+0000","last_change":"2023-12-14T18:25:55.323246+0000","last_active":"2023-12-14T18:26:48.562267+0000","last_peered":"2023-12-14T18:26:48.562267+0000","last_clean":"2023-12-14T18:26:48.562267+0000","last_became_active":"2023-12-14T18:25:55.322869+0000","last_became_peered":"2023-12-14T18:25:55.322869+0000","last_unstale":"2023-12-14T18:26:48.562267+0000","last_undegraded":"2023-12-14T18:26:48.562267+0000","last_fullsized":"2023-12-14T18:26:48.562267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:26.374795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563022+0000","last_change":"2023-12-14T18:25:55.317319+0000","last_active":"2023-12-14T18:26:48.563022+0000","last_peered":"2023-12-14T18:26:48.563022+0000","last_clean":"2023-12-14T18:26:48.563022+0000","last_became_active":"2023-12-14T18:25:55.316740+0000","last_became_peered":"2023-12-14T18:25:55.316740+0000","last_unstale":"2023-12-14T18:26:48.563022+0000","last_undegraded":"2023-12-14T18:26:48.563022+0000","last_fullsized":"2023-12-14T18:26:48.563022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:44:38.171386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563106+0000","last_change":"2023-12-14T18:25:55.307441+0000","last_active":"2023-12-14T18:26:48.563106+0000","last_peered":"2023-12-14T18:26:48.563106+0000","last_clean":"2023-12-14T18:26:48.563106+0000","last_became_active":"2023-12-14T18:25:55.306714+0000","last_became_peered":"2023-12-14T18:25:55.306714+0000","last_unstale":"2023-12-14T18:26:48.563106+0000","last_undegraded":"2023-12-14T18:26:48.563106+0000","last_fullsized":"2023-12-14T18:26:48.563106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:18:44.617868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557419+0000","last_change":"2023-12-14T18:25:55.308428+0000","last_active":"2023-12-14T18:26:48.557419+0000","last_peered":"2023-12-14T18:26:48.557419+0000","last_clean":"2023-12-14T18:26:48.557419+0000","last_became_active":"2023-12-14T18:25:55.308080+0000","last_became_peered":"2023-12-14T18:25:55.308080+0000","last_unstale":"2023-12-14T18:26:48.557419+0000","last_undegraded":"2023-12-14T18:26:48.557419+0000","last_fullsized":"2023-12-14T18:26:48.557419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:04:10.141803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558264+0000","last_change":"2023-12-14T18:25:55.323512+0000","last_active":"2023-12-14T18:26:48.558264+0000","last_peered":"2023-12-14T18:26:48.558264+0000","last_clean":"2023-12-14T18:26:48.558264+0000","last_became_active":"2023-12-14T18:25:55.323159+0000","last_became_peered":"2023-12-14T18:25:55.323159+0000","last_unstale":"2023-12-14T18:26:48.558264+0000","last_undegraded":"2023-12-14T18:26:48.558264+0000","last_fullsized":"2023-12-14T18:26:48.558264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:53.022658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559788+0000","last_change":"2023-12-14T18:25:55.309903+0000","last_active":"2023-12-14T18:26:48.559788+0000","last_peered":"2023-12-14T18:26:48.559788+0000","last_clean":"2023-12-14T18:26:48.559788+0000","last_became_active":"2023-12-14T18:25:55.309293+0000","last_became_peered":"2023-12-14T18:25:55.309293+0000","last_unstale":"2023-12-14T18:26:48.559788+0000","last_undegraded":"2023-12-14T18:26:48.559788+0000","last_fullsized":"2023-12-14T18:26:48.559788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:18.996023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562659+0000","last_change":"2023-12-14T18:25:55.318643+0000","last_active":"2023-12-14T18:26:48.562659+0000","last_peered":"2023-12-14T18:26:48.562659+0000","last_clean":"2023-12-14T18:26:48.562659+0000","last_became_active":"2023-12-14T18:25:55.318309+0000","last_became_peered":"2023-12-14T18:25:55.318309+0000","last_unstale":"2023-12-14T18:26:48.562659+0000","last_undegraded":"2023-12-14T18:26:48.562659+0000","last_fullsized":"2023-12-14T18:26:48.562659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:00:27.549342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562608+0000","last_change":"2023-12-14T18:25:55.306836+0000","last_active":"2023-12-14T18:26:48.562608+0000","last_peered":"2023-12-14T18:26:48.562608+0000","last_clean":"2023-12-14T18:26:48.562608+0000","last_became_active":"2023-12-14T18:25:55.303751+0000","last_became_peered":"2023-12-14T18:25:55.303751+0000","last_unstale":"2023-12-14T18:26:48.562608+0000","last_undegraded":"2023-12-14T18:26:48.562608+0000","last_fullsized":"2023-12-14T18:26:48.562608+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:06:20.688646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566623+0000","last_change":"2023-12-14T18:25:55.313925+0000","last_active":"2023-12-14T18:26:48.566623+0000","last_peered":"2023-12-14T18:26:48.566623+0000","last_clean":"2023-12-14T18:26:48.566623+0000","last_became_active":"2023-12-14T18:25:55.313595+0000","last_became_peered":"2023-12-14T18:25:55.313595+0000","last_unstale":"2023-12-14T18:26:48.566623+0000","last_undegraded":"2023-12-14T18:26:48.566623+0000","last_fullsized":"2023-12-14T18:26:48.566623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:58.296365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564244+0000","last_change":"2023-12-14T18:25:59.339356+0000","last_active":"2023-12-14T18:26:48.564244+0000","last_peered":"2023-12-14T18:26:48.564244+0000","last_clean":"2023-12-14T18:26:48.564244+0000","last_became_active":"2023-12-14T18:25:59.338955+0000","last_became_peered":"2023-12-14T18:25:59.338955+0000","last_unstale":"2023-12-14T18:26:48.564244+0000","last_undegraded":"2023-12-14T18:26:48.564244+0000","last_fullsized":"2023-12-14T18:26:48.564244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:23.065151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563082+0000","last_change":"2023-12-14T18:25:54.855021+0000","last_active":"2023-12-14T18:26:48.563082+0000","last_peered":"2023-12-14T18:26:48.563082+0000","last_clean":"2023-12-14T18:26:48.563082+0000","last_became_active":"2023-12-14T18:25:54.854616+0000","last_became_peered":"2023-12-14T18:25:54.854616+0000","last_unstale":"2023-12-14T18:26:48.563082+0000","last_undegraded":"2023-12-14T18:26:48.563082+0000","last_fullsized":"2023-12-14T18:26:48.563082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:35:18.652626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559422+0000","last_change":"2023-12-14T18:25:55.325184+0000","last_active":"2023-12-14T18:26:48.559422+0000","last_peered":"2023-12-14T18:26:48.559422+0000","last_clean":"2023-12-14T18:26:48.559422+0000","last_became_active":"2023-12-14T18:25:55.324807+0000","last_became_peered":"2023-12-14T18:25:55.324807+0000","last_unstale":"2023-12-14T18:26:48.559422+0000","last_undegraded":"2023-12-14T18:26:48.559422+0000","last_fullsized":"2023-12-14T18:26:48.559422+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:50:31.758473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562609+0000","last_change":"2023-12-14T18:25:59.348347+0000","last_active":"2023-12-14T18:26:48.562609+0000","last_peered":"2023-12-14T18:26:48.562609+0000","last_clean":"2023-12-14T18:26:48.562609+0000","last_became_active":"2023-12-14T18:25:59.348021+0000","last_became_peered":"2023-12-14T18:25:59.348021+0000","last_unstale":"2023-12-14T18:26:48.562609+0000","last_undegraded":"2023-12-14T18:26:48.562609+0000","last_fullsized":"2023-12-14T18:26:48.562609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:43.233753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559749+0000","last_change":"2023-12-14T18:25:54.854063+0000","last_active":"2023-12-14T18:26:48.559749+0000","last_peered":"2023-12-14T18:26:48.559749+0000","last_clean":"2023-12-14T18:26:48.559749+0000","last_became_active":"2023-12-14T18:25:54.853718+0000","last_became_peered":"2023-12-14T18:25:54.853718+0000","last_unstale":"2023-12-14T18:26:48.559749+0000","last_undegraded":"2023-12-14T18:26:48.559749+0000","last_fullsized":"2023-12-14T18:26:48.559749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:43:43.650498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558888+0000","last_change":"2023-12-14T18:25:55.317270+0000","last_active":"2023-12-14T18:26:48.558888+0000","last_peered":"2023-12-14T18:26:48.558888+0000","last_clean":"2023-12-14T18:26:48.558888+0000","last_became_active":"2023-12-14T18:25:55.316679+0000","last_became_peered":"2023-12-14T18:25:55.316679+0000","last_unstale":"2023-12-14T18:26:48.558888+0000","last_undegraded":"2023-12-14T18:26:48.558888+0000","last_fullsized":"2023-12-14T18:26:48.558888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:50.284685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564072+0000","last_change":"2023-12-14T18:25:59.342832+0000","last_active":"2023-12-14T18:26:48.564072+0000","last_peered":"2023-12-14T18:26:48.564072+0000","last_clean":"2023-12-14T18:26:48.564072+0000","last_became_active":"2023-12-14T18:25:59.342397+0000","last_became_peered":"2023-12-14T18:25:59.342397+0000","last_unstale":"2023-12-14T18:26:48.564072+0000","last_undegraded":"2023-12-14T18:26:48.564072+0000","last_fullsized":"2023-12-14T18:26:48.564072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:29:21.615846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568778+0000","last_change":"2023-12-14T18:25:54.851881+0000","last_active":"2023-12-14T18:26:48.568778+0000","last_peered":"2023-12-14T18:26:48.568778+0000","last_clean":"2023-12-14T18:26:48.568778+0000","last_became_active":"2023-12-14T18:25:54.851514+0000","last_became_peered":"2023-12-14T18:25:54.851514+0000","last_unstale":"2023-12-14T18:26:48.568778+0000","last_undegraded":"2023-12-14T18:26:48.568778+0000","last_fullsized":"2023-12-14T18:26:48.568778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:38.658671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560627+0000","last_change":"2023-12-14T18:25:55.307376+0000","last_active":"2023-12-14T18:26:48.560627+0000","last_peered":"2023-12-14T18:26:48.560627+0000","last_clean":"2023-12-14T18:26:48.560627+0000","last_became_active":"2023-12-14T18:25:55.306859+0000","last_became_peered":"2023-12-14T18:25:55.306859+0000","last_unstale":"2023-12-14T18:26:48.560627+0000","last_undegraded":"2023-12-14T18:26:48.560627+0000","last_fullsized":"2023-12-14T18:26:48.560627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:54:13.436594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562615+0000","last_change":"2023-12-14T18:25:59.339195+0000","last_active":"2023-12-14T18:26:48.562615+0000","last_peered":"2023-12-14T18:26:48.562615+0000","last_clean":"2023-12-14T18:26:48.562615+0000","last_became_active":"2023-12-14T18:25:59.338845+0000","last_became_peered":"2023-12-14T18:25:59.338845+0000","last_unstale":"2023-12-14T18:26:48.562615+0000","last_undegraded":"2023-12-14T18:26:48.562615+0000","last_fullsized":"2023-12-14T18:26:48.562615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:22:13.915160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568436+0000","last_change":"2023-12-14T18:25:54.852429+0000","last_active":"2023-12-14T18:26:48.568436+0000","last_peered":"2023-12-14T18:26:48.568436+0000","last_clean":"2023-12-14T18:26:48.568436+0000","last_became_active":"2023-12-14T18:25:54.852027+0000","last_became_peered":"2023-12-14T18:25:54.852027+0000","last_unstale":"2023-12-14T18:26:48.568436+0000","last_undegraded":"2023-12-14T18:26:48.568436+0000","last_fullsized":"2023-12-14T18:26:48.568436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:51:57.429285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563999+0000","last_change":"2023-12-14T18:25:55.310545+0000","last_active":"2023-12-14T18:26:48.563999+0000","last_peered":"2023-12-14T18:26:48.563999+0000","last_clean":"2023-12-14T18:26:48.563999+0000","last_became_active":"2023-12-14T18:25:55.310137+0000","last_became_peered":"2023-12-14T18:25:55.310137+0000","last_unstale":"2023-12-14T18:26:48.563999+0000","last_undegraded":"2023-12-14T18:26:48.563999+0000","last_fullsized":"2023-12-14T18:26:48.563999+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:21:53.454334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564084+0000","last_change":"2023-12-14T18:25:59.334057+0000","last_active":"2023-12-14T18:26:48.564084+0000","last_peered":"2023-12-14T18:26:48.564084+0000","last_clean":"2023-12-14T18:26:48.564084+0000","last_became_active":"2023-12-14T18:25:59.333620+0000","last_became_peered":"2023-12-14T18:25:59.333620+0000","last_unstale":"2023-12-14T18:26:48.564084+0000","last_undegraded":"2023-12-14T18:26:48.564084+0000","last_fullsized":"2023-12-14T18:26:48.564084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:33.154296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:35.636522+0000","last_change":"2023-12-14T18:26:48.574808+0000","last_active":"2023-12-14T18:27:35.636522+0000","last_peered":"2023-12-14T18:27:35.636522+0000","last_clean":"2023-12-14T18:27:35.636522+0000","last_became_active":"2023-12-14T18:26:48.568640+0000","last_became_peered":"2023-12-14T18:26:48.568640+0000","last_unstale":"2023-12-14T18:27:35.636522+0000","last_undegraded":"2023-12-14T18:27:35.636522+0000","last_fullsized":"2023-12-14T18:27:35.636522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:16.685666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566784+0000","last_change":"2023-12-14T18:25:55.310678+0000","last_active":"2023-12-14T18:26:48.566784+0000","last_peered":"2023-12-14T18:26:48.566784+0000","last_clean":"2023-12-14T18:26:48.566784+0000","last_became_active":"2023-12-14T18:25:55.310088+0000","last_became_peered":"2023-12-14T18:25:55.310088+0000","last_unstale":"2023-12-14T18:26:48.566784+0000","last_undegraded":"2023-12-14T18:26:48.566784+0000","last_fullsized":"2023-12-14T18:26:48.566784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:02.326220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.573798+0000","last_change":"2023-12-14T18:26:48.573798+0000","last_active":"2023-12-14T18:26:48.573798+0000","last_peered":"2023-12-14T18:26:48.573798+0000","last_clean":"2023-12-14T18:26:48.573798+0000","last_became_active":"2023-12-14T18:26:48.573308+0000","last_became_peered":"2023-12-14T18:26:48.573308+0000","last_unstale":"2023-12-14T18:26:48.573798+0000","last_undegraded":"2023-12-14T18:26:48.573798+0000","last_fullsized":"2023-12-14T18:26:48.573798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:28:46.273622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647410+0000","last_change":"2023-12-14T18:25:54.855057+0000","last_active":"2023-12-14T18:26:48.647410+0000","last_peered":"2023-12-14T18:26:48.647410+0000","last_clean":"2023-12-14T18:26:48.647410+0000","last_became_active":"2023-12-14T18:25:54.853669+0000","last_became_peered":"2023-12-14T18:25:54.853669+0000","last_unstale":"2023-12-14T18:26:48.647410+0000","last_undegraded":"2023-12-14T18:26:48.647410+0000","last_fullsized":"2023-12-14T18:26:48.647410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:05:15.020350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564341+0000","last_change":"2023-12-14T18:25:55.320453+0000","last_active":"2023-12-14T18:26:48.564341+0000","last_peered":"2023-12-14T18:26:48.564341+0000","last_clean":"2023-12-14T18:26:48.564341+0000","last_became_active":"2023-12-14T18:25:55.320041+0000","last_became_peered":"2023-12-14T18:25:55.320041+0000","last_unstale":"2023-12-14T18:26:48.564341+0000","last_undegraded":"2023-12-14T18:26:48.564341+0000","last_fullsized":"2023-12-14T18:26:48.564341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:43.990428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570306+0000","last_change":"2023-12-14T18:26:48.570306+0000","last_active":"2023-12-14T18:26:48.570306+0000","last_peered":"2023-12-14T18:26:48.570306+0000","last_clean":"2023-12-14T18:26:48.570306+0000","last_became_active":"2023-12-14T18:26:48.569803+0000","last_became_peered":"2023-12-14T18:26:48.569803+0000","last_unstale":"2023-12-14T18:26:48.570306+0000","last_undegraded":"2023-12-14T18:26:48.570306+0000","last_fullsized":"2023-12-14T18:26:48.570306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:33.628855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562436+0000","last_change":"2023-12-14T18:25:54.855037+0000","last_active":"2023-12-14T18:26:48.562436+0000","last_peered":"2023-12-14T18:26:48.562436+0000","last_clean":"2023-12-14T18:26:48.562436+0000","last_became_active":"2023-12-14T18:25:54.854625+0000","last_became_peered":"2023-12-14T18:25:54.854625+0000","last_unstale":"2023-12-14T18:26:48.562436+0000","last_undegraded":"2023-12-14T18:26:48.562436+0000","last_fullsized":"2023-12-14T18:26:48.562436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:05:53.677809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559154+0000","last_change":"2023-12-14T18:25:55.320196+0000","last_active":"2023-12-14T18:26:48.559154+0000","last_peered":"2023-12-14T18:26:48.559154+0000","last_clean":"2023-12-14T18:26:48.559154+0000","last_became_active":"2023-12-14T18:25:55.319820+0000","last_became_peered":"2023-12-14T18:25:55.319820+0000","last_unstale":"2023-12-14T18:26:48.559154+0000","last_undegraded":"2023-12-14T18:26:48.559154+0000","last_fullsized":"2023-12-14T18:26:48.559154+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:49.740844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557724+0000","last_change":"2023-12-14T18:25:59.341103+0000","last_active":"2023-12-14T18:26:48.557724+0000","last_peered":"2023-12-14T18:26:48.557724+0000","last_clean":"2023-12-14T18:26:48.557724+0000","last_became_active":"2023-12-14T18:25:59.340578+0000","last_became_peered":"2023-12-14T18:25:59.340578+0000","last_unstale":"2023-12-14T18:26:48.557724+0000","last_undegraded":"2023-12-14T18:26:48.557724+0000","last_fullsized":"2023-12-14T18:26:48.557724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:44:17.333118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.567219+0000","last_change":"2023-12-14T18:25:54.854050+0000","last_active":"2023-12-14T18:26:48.567219+0000","last_peered":"2023-12-14T18:26:48.567219+0000","last_clean":"2023-12-14T18:26:48.567219+0000","last_became_active":"2023-12-14T18:25:54.853626+0000","last_became_peered":"2023-12-14T18:25:54.853626+0000","last_unstale":"2023-12-14T18:26:48.567219+0000","last_undegraded":"2023-12-14T18:26:48.567219+0000","last_fullsized":"2023-12-14T18:26:48.567219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:45:09.792477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561943+0000","last_change":"2023-12-14T18:25:55.310390+0000","last_active":"2023-12-14T18:26:48.561943+0000","last_peered":"2023-12-14T18:26:48.561943+0000","last_clean":"2023-12-14T18:26:48.561943+0000","last_became_active":"2023-12-14T18:25:55.310037+0000","last_became_peered":"2023-12-14T18:25:55.310037+0000","last_unstale":"2023-12-14T18:26:48.561943+0000","last_undegraded":"2023-12-14T18:26:48.561943+0000","last_fullsized":"2023-12-14T18:26:48.561943+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:21:43.141339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559411+0000","last_change":"2023-12-14T18:25:59.343494+0000","last_active":"2023-12-14T18:26:48.559411+0000","last_peered":"2023-12-14T18:26:48.559411+0000","last_clean":"2023-12-14T18:26:48.559411+0000","last_became_active":"2023-12-14T18:25:59.343095+0000","last_became_peered":"2023-12-14T18:25:59.343095+0000","last_unstale":"2023-12-14T18:26:48.559411+0000","last_undegraded":"2023-12-14T18:26:48.559411+0000","last_fullsized":"2023-12-14T18:26:48.559411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:41:36.307493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:37.638765+0000","last_change":"2023-12-14T18:25:54.851665+0000","last_active":"2023-12-14T18:27:37.638765+0000","last_peered":"2023-12-14T18:27:37.638765+0000","last_clean":"2023-12-14T18:27:37.638765+0000","last_became_active":"2023-12-14T18:25:54.851295+0000","last_became_peered":"2023-12-14T18:25:54.851295+0000","last_unstale":"2023-12-14T18:27:37.638765+0000","last_undegraded":"2023-12-14T18:27:37.638765+0000","last_fullsized":"2023-12-14T18:27:37.638765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:20:23.673760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560703+0000","last_change":"2023-12-14T18:25:55.307794+0000","last_active":"2023-12-14T18:26:48.560703+0000","last_peered":"2023-12-14T18:26:48.560703+0000","last_clean":"2023-12-14T18:26:48.560703+0000","last_became_active":"2023-12-14T18:25:55.307511+0000","last_became_peered":"2023-12-14T18:25:55.307511+0000","last_unstale":"2023-12-14T18:26:48.560703+0000","last_undegraded":"2023-12-14T18:26:48.560703+0000","last_fullsized":"2023-12-14T18:26:48.560703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:33:22.836099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561628+0000","last_change":"2023-12-14T18:25:59.339439+0000","last_active":"2023-12-14T18:26:48.561628+0000","last_peered":"2023-12-14T18:26:48.561628+0000","last_clean":"2023-12-14T18:26:48.561628+0000","last_became_active":"2023-12-14T18:25:59.338928+0000","last_became_peered":"2023-12-14T18:25:59.338928+0000","last_unstale":"2023-12-14T18:26:48.561628+0000","last_undegraded":"2023-12-14T18:26:48.561628+0000","last_fullsized":"2023-12-14T18:26:48.561628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:47.796981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561906+0000","last_change":"2023-12-14T18:25:54.851748+0000","last_active":"2023-12-14T18:26:48.561906+0000","last_peered":"2023-12-14T18:26:48.561906+0000","last_clean":"2023-12-14T18:26:48.561906+0000","last_became_active":"2023-12-14T18:25:54.851422+0000","last_became_peered":"2023-12-14T18:25:54.851422+0000","last_unstale":"2023-12-14T18:26:48.561906+0000","last_undegraded":"2023-12-14T18:26:48.561906+0000","last_fullsized":"2023-12-14T18:26:48.561906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:00.359853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558750+0000","last_change":"2023-12-14T18:25:55.321451+0000","last_active":"2023-12-14T18:26:48.558750+0000","last_peered":"2023-12-14T18:26:48.558750+0000","last_clean":"2023-12-14T18:26:48.558750+0000","last_became_active":"2023-12-14T18:25:55.317664+0000","last_became_peered":"2023-12-14T18:25:55.317664+0000","last_unstale":"2023-12-14T18:26:48.558750+0000","last_undegraded":"2023-12-14T18:26:48.558750+0000","last_fullsized":"2023-12-14T18:26:48.558750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:57:49.003816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568565+0000","last_change":"2023-12-14T18:26:48.568565+0000","last_active":"2023-12-14T18:26:48.568565+0000","last_peered":"2023-12-14T18:26:48.568565+0000","last_clean":"2023-12-14T18:26:48.568565+0000","last_became_active":"2023-12-14T18:26:48.568090+0000","last_became_peered":"2023-12-14T18:26:48.568090+0000","last_unstale":"2023-12-14T18:26:48.568565+0000","last_undegraded":"2023-12-14T18:26:48.568565+0000","last_fullsized":"2023-12-14T18:26:48.568565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:14:38.160855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563856+0000","last_change":"2023-12-14T18:25:54.851932+0000","last_active":"2023-12-14T18:26:48.563856+0000","last_peered":"2023-12-14T18:26:48.563856+0000","last_clean":"2023-12-14T18:26:48.563856+0000","last_became_active":"2023-12-14T18:25:54.851595+0000","last_became_peered":"2023-12-14T18:25:54.851595+0000","last_unstale":"2023-12-14T18:26:48.563856+0000","last_undegraded":"2023-12-14T18:26:48.563856+0000","last_fullsized":"2023-12-14T18:26:48.563856+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:12.189176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560853+0000","last_change":"2023-12-14T18:25:55.313751+0000","last_active":"2023-12-14T18:26:48.560853+0000","last_peered":"2023-12-14T18:26:48.560853+0000","last_clean":"2023-12-14T18:26:48.560853+0000","last_became_active":"2023-12-14T18:25:55.312245+0000","last_became_peered":"2023-12-14T18:25:55.312245+0000","last_unstale":"2023-12-14T18:26:48.560853+0000","last_undegraded":"2023-12-14T18:26:48.560853+0000","last_fullsized":"2023-12-14T18:26:48.560853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:55:18.680896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562308+0000","last_change":"2023-12-14T18:25:59.340153+0000","last_active":"2023-12-14T18:26:48.562308+0000","last_peered":"2023-12-14T18:26:48.562308+0000","last_clean":"2023-12-14T18:26:48.562308+0000","last_became_active":"2023-12-14T18:25:59.333543+0000","last_became_peered":"2023-12-14T18:25:59.333543+0000","last_unstale":"2023-12-14T18:26:48.562308+0000","last_undegraded":"2023-12-14T18:26:48.562308+0000","last_fullsized":"2023-12-14T18:26:48.562308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:05:51.668380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":223,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:38.641353+0000","last_change":"2023-12-14T18:25:54.850197+0000","last_active":"2023-12-14T18:27:38.641353+0000","last_peered":"2023-12-14T18:27:38.641353+0000","last_clean":"2023-12-14T18:27:38.641353+0000","last_became_active":"2023-12-14T18:25:54.849803+0000","last_became_peered":"2023-12-14T18:25:54.849803+0000","last_unstale":"2023-12-14T18:27:38.641353+0000","last_undegraded":"2023-12-14T18:27:38.641353+0000","last_fullsized":"2023-12-14T18:27:38.641353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:33.081169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64447,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":161,"num_read_kb":139,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568542+0000","last_change":"2023-12-14T18:25:55.320771+0000","last_active":"2023-12-14T18:26:48.568542+0000","last_peered":"2023-12-14T18:26:48.568542+0000","last_clean":"2023-12-14T18:26:48.568542+0000","last_became_active":"2023-12-14T18:25:55.320366+0000","last_became_peered":"2023-12-14T18:25:55.320366+0000","last_unstale":"2023-12-14T18:26:48.568542+0000","last_undegraded":"2023-12-14T18:26:48.568542+0000","last_fullsized":"2023-12-14T18:26:48.568542+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:30.839862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564457+0000","last_change":"2023-12-14T18:25:59.338314+0000","last_active":"2023-12-14T18:26:48.564457+0000","last_peered":"2023-12-14T18:26:48.564457+0000","last_clean":"2023-12-14T18:26:48.564457+0000","last_became_active":"2023-12-14T18:25:59.337903+0000","last_became_peered":"2023-12-14T18:25:59.337903+0000","last_unstale":"2023-12-14T18:26:48.564457+0000","last_undegraded":"2023-12-14T18:26:48.564457+0000","last_fullsized":"2023-12-14T18:26:48.564457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:48:27.839169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565422+0000","last_change":"2023-12-14T18:25:54.853586+0000","last_active":"2023-12-14T18:26:48.565422+0000","last_peered":"2023-12-14T18:26:48.565422+0000","last_clean":"2023-12-14T18:26:48.565422+0000","last_became_active":"2023-12-14T18:25:54.853207+0000","last_became_peered":"2023-12-14T18:25:54.853207+0000","last_unstale":"2023-12-14T18:26:48.565422+0000","last_undegraded":"2023-12-14T18:26:48.565422+0000","last_fullsized":"2023-12-14T18:26:48.565422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:27:34.931404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562476+0000","last_change":"2023-12-14T18:25:55.311906+0000","last_active":"2023-12-14T18:26:48.562476+0000","last_peered":"2023-12-14T18:26:48.562476+0000","last_clean":"2023-12-14T18:26:48.562476+0000","last_became_active":"2023-12-14T18:25:55.311512+0000","last_became_peered":"2023-12-14T18:25:55.311512+0000","last_unstale":"2023-12-14T18:26:48.562476+0000","last_undegraded":"2023-12-14T18:26:48.562476+0000","last_fullsized":"2023-12-14T18:26:48.562476+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:45.542772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561959+0000","last_change":"2023-12-14T18:25:59.337454+0000","last_active":"2023-12-14T18:26:48.561959+0000","last_peered":"2023-12-14T18:26:48.561959+0000","last_clean":"2023-12-14T18:26:48.561959+0000","last_became_active":"2023-12-14T18:25:59.337044+0000","last_became_peered":"2023-12-14T18:25:59.337044+0000","last_unstale":"2023-12-14T18:26:48.561959+0000","last_undegraded":"2023-12-14T18:26:48.561959+0000","last_fullsized":"2023-12-14T18:26:48.561959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:54.542913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565425+0000","last_change":"2023-12-14T18:25:54.853173+0000","last_active":"2023-12-14T18:26:48.565425+0000","last_peered":"2023-12-14T18:26:48.565425+0000","last_clean":"2023-12-14T18:26:48.565425+0000","last_became_active":"2023-12-14T18:25:54.852606+0000","last_became_peered":"2023-12-14T18:25:54.852606+0000","last_unstale":"2023-12-14T18:26:48.565425+0000","last_undegraded":"2023-12-14T18:26:48.565425+0000","last_fullsized":"2023-12-14T18:26:48.565425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:46.938078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566091+0000","last_change":"2023-12-14T18:26:48.566091+0000","last_active":"2023-12-14T18:26:48.566091+0000","last_peered":"2023-12-14T18:26:48.566091+0000","last_clean":"2023-12-14T18:26:48.566091+0000","last_became_active":"2023-12-14T18:26:48.565714+0000","last_became_peered":"2023-12-14T18:26:48.565714+0000","last_unstale":"2023-12-14T18:26:48.566091+0000","last_undegraded":"2023-12-14T18:26:48.566091+0000","last_fullsized":"2023-12-14T18:26:48.566091+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:44:48.478859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562658+0000","last_change":"2023-12-14T18:25:59.343195+0000","last_active":"2023-12-14T18:26:48.562658+0000","last_peered":"2023-12-14T18:26:48.562658+0000","last_clean":"2023-12-14T18:26:48.562658+0000","last_became_active":"2023-12-14T18:25:59.342964+0000","last_became_peered":"2023-12-14T18:25:59.342964+0000","last_unstale":"2023-12-14T18:26:48.562658+0000","last_undegraded":"2023-12-14T18:26:48.562658+0000","last_fullsized":"2023-12-14T18:26:48.562658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:07:22.379528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":101,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:32.714881+0000","last_change":"2023-12-14T18:25:54.849493+0000","last_active":"2023-12-14T18:27:32.714881+0000","last_peered":"2023-12-14T18:27:32.714881+0000","last_clean":"2023-12-14T18:27:32.714881+0000","last_became_active":"2023-12-14T18:25:54.849086+0000","last_became_peered":"2023-12-14T18:25:54.849086+0000","last_unstale":"2023-12-14T18:27:32.714881+0000","last_undegraded":"2023-12-14T18:27:32.714881+0000","last_fullsized":"2023-12-14T18:27:32.714881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:26:13.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561690+0000","last_change":"2023-12-14T18:25:55.307811+0000","last_active":"2023-12-14T18:26:48.561690+0000","last_peered":"2023-12-14T18:26:48.561690+0000","last_clean":"2023-12-14T18:26:48.561690+0000","last_became_active":"2023-12-14T18:25:55.307499+0000","last_became_peered":"2023-12-14T18:25:55.307499+0000","last_unstale":"2023-12-14T18:26:48.561690+0000","last_undegraded":"2023-12-14T18:26:48.561690+0000","last_fullsized":"2023-12-14T18:26:48.561690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:10:17.702084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.649376+0000","last_change":"2023-12-14T18:26:48.649376+0000","last_active":"2023-12-14T18:26:48.649376+0000","last_peered":"2023-12-14T18:26:48.649376+0000","last_clean":"2023-12-14T18:26:48.649376+0000","last_became_active":"2023-12-14T18:26:48.648882+0000","last_became_peered":"2023-12-14T18:26:48.648882+0000","last_unstale":"2023-12-14T18:26:48.649376+0000","last_undegraded":"2023-12-14T18:26:48.649376+0000","last_fullsized":"2023-12-14T18:26:48.649376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:12.045140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,10,11],"acting":[6,3,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566931+0000","last_change":"2023-12-14T18:25:54.851729+0000","last_active":"2023-12-14T18:26:48.566931+0000","last_peered":"2023-12-14T18:26:48.566931+0000","last_clean":"2023-12-14T18:26:48.566931+0000","last_became_active":"2023-12-14T18:25:54.851420+0000","last_became_peered":"2023-12-14T18:25:54.851420+0000","last_unstale":"2023-12-14T18:26:48.566931+0000","last_undegraded":"2023-12-14T18:26:48.566931+0000","last_fullsized":"2023-12-14T18:26:48.566931+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:28:47.382142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.572300+0000","last_change":"2023-12-14T18:26:48.572300+0000","last_active":"2023-12-14T18:26:48.572300+0000","last_peered":"2023-12-14T18:26:48.572300+0000","last_clean":"2023-12-14T18:26:48.572300+0000","last_became_active":"2023-12-14T18:26:48.571895+0000","last_became_peered":"2023-12-14T18:26:48.571895+0000","last_unstale":"2023-12-14T18:26:48.572300+0000","last_undegraded":"2023-12-14T18:26:48.572300+0000","last_fullsized":"2023-12-14T18:26:48.572300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:40:54.373914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566446+0000","last_change":"2023-12-14T18:25:59.339433+0000","last_active":"2023-12-14T18:26:48.566446+0000","last_peered":"2023-12-14T18:26:48.566446+0000","last_clean":"2023-12-14T18:26:48.566446+0000","last_became_active":"2023-12-14T18:25:59.339006+0000","last_became_peered":"2023-12-14T18:25:59.339006+0000","last_unstale":"2023-12-14T18:26:48.566446+0000","last_undegraded":"2023-12-14T18:26:48.566446+0000","last_fullsized":"2023-12-14T18:26:48.566446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:33:44.826472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.570871+0000","last_change":"2023-12-14T18:25:54.852299+0000","last_active":"2023-12-14T18:26:48.570871+0000","last_peered":"2023-12-14T18:26:48.570871+0000","last_clean":"2023-12-14T18:26:48.570871+0000","last_became_active":"2023-12-14T18:25:54.851820+0000","last_became_peered":"2023-12-14T18:25:54.851820+0000","last_unstale":"2023-12-14T18:26:48.570871+0000","last_undegraded":"2023-12-14T18:26:48.570871+0000","last_fullsized":"2023-12-14T18:26:48.570871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:28.018194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561975+0000","last_change":"2023-12-14T18:25:55.311392+0000","last_active":"2023-12-14T18:26:48.561975+0000","last_peered":"2023-12-14T18:26:48.561975+0000","last_clean":"2023-12-14T18:26:48.561975+0000","last_became_active":"2023-12-14T18:25:55.311037+0000","last_became_peered":"2023-12-14T18:25:55.311037+0000","last_unstale":"2023-12-14T18:26:48.561975+0000","last_undegraded":"2023-12-14T18:26:48.561975+0000","last_fullsized":"2023-12-14T18:26:48.561975+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:13:37.576788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566649+0000","last_change":"2023-12-14T18:25:59.762771+0000","last_active":"2023-12-14T18:26:48.566649+0000","last_peered":"2023-12-14T18:26:48.566649+0000","last_clean":"2023-12-14T18:26:48.566649+0000","last_became_active":"2023-12-14T18:25:59.762220+0000","last_became_peered":"2023-12-14T18:25:59.762220+0000","last_unstale":"2023-12-14T18:26:48.566649+0000","last_undegraded":"2023-12-14T18:26:48.566649+0000","last_fullsized":"2023-12-14T18:26:48.566649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:36:37.853928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562990+0000","last_change":"2023-12-14T18:25:54.854108+0000","last_active":"2023-12-14T18:26:48.562990+0000","last_peered":"2023-12-14T18:26:48.562990+0000","last_clean":"2023-12-14T18:26:48.562990+0000","last_became_active":"2023-12-14T18:25:54.853728+0000","last_became_peered":"2023-12-14T18:25:54.853728+0000","last_unstale":"2023-12-14T18:26:48.562990+0000","last_undegraded":"2023-12-14T18:26:48.562990+0000","last_fullsized":"2023-12-14T18:26:48.562990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:32.215966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647772+0000","last_change":"2023-12-14T18:25:55.312673+0000","last_active":"2023-12-14T18:26:48.647772+0000","last_peered":"2023-12-14T18:26:48.647772+0000","last_clean":"2023-12-14T18:26:48.647772+0000","last_became_active":"2023-12-14T18:25:55.312360+0000","last_became_peered":"2023-12-14T18:25:55.312360+0000","last_unstale":"2023-12-14T18:26:48.647772+0000","last_undegraded":"2023-12-14T18:26:48.647772+0000","last_fullsized":"2023-12-14T18:26:48.647772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:51:12.578633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563700+0000","last_change":"2023-12-14T18:25:59.345435+0000","last_active":"2023-12-14T18:26:48.563700+0000","last_peered":"2023-12-14T18:26:48.563700+0000","last_clean":"2023-12-14T18:26:48.563700+0000","last_became_active":"2023-12-14T18:25:59.345029+0000","last_became_peered":"2023-12-14T18:25:59.345029+0000","last_unstale":"2023-12-14T18:26:48.563700+0000","last_undegraded":"2023-12-14T18:26:48.563700+0000","last_fullsized":"2023-12-14T18:26:48.563700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:58.698325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561937+0000","last_change":"2023-12-14T18:25:54.854649+0000","last_active":"2023-12-14T18:26:48.561937+0000","last_peered":"2023-12-14T18:26:48.561937+0000","last_clean":"2023-12-14T18:26:48.561937+0000","last_became_active":"2023-12-14T18:25:54.854150+0000","last_became_peered":"2023-12-14T18:25:54.854150+0000","last_unstale":"2023-12-14T18:26:48.561937+0000","last_undegraded":"2023-12-14T18:26:48.561937+0000","last_fullsized":"2023-12-14T18:26:48.561937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:26:21.783482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.647707+0000","last_change":"2023-12-14T18:25:55.317898+0000","last_active":"2023-12-14T18:26:48.647707+0000","last_peered":"2023-12-14T18:26:48.647707+0000","last_clean":"2023-12-14T18:26:48.647707+0000","last_became_active":"2023-12-14T18:25:55.317527+0000","last_became_peered":"2023-12-14T18:25:55.317527+0000","last_unstale":"2023-12-14T18:26:48.647707+0000","last_undegraded":"2023-12-14T18:26:48.647707+0000","last_fullsized":"2023-12-14T18:26:48.647707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:26:41.629317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563228+0000","last_change":"2023-12-14T18:25:59.762369+0000","last_active":"2023-12-14T18:26:48.563228+0000","last_peered":"2023-12-14T18:26:48.563228+0000","last_clean":"2023-12-14T18:26:48.563228+0000","last_became_active":"2023-12-14T18:25:59.761822+0000","last_became_peered":"2023-12-14T18:25:59.761822+0000","last_unstale":"2023-12-14T18:26:48.563228+0000","last_undegraded":"2023-12-14T18:26:48.563228+0000","last_fullsized":"2023-12-14T18:26:48.563228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:41:45.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566971+0000","last_change":"2023-12-14T18:25:55.307440+0000","last_active":"2023-12-14T18:26:48.566971+0000","last_peered":"2023-12-14T18:26:48.566971+0000","last_clean":"2023-12-14T18:26:48.566971+0000","last_became_active":"2023-12-14T18:25:55.307125+0000","last_became_peered":"2023-12-14T18:25:55.307125+0000","last_unstale":"2023-12-14T18:26:48.566971+0000","last_undegraded":"2023-12-14T18:26:48.566971+0000","last_fullsized":"2023-12-14T18:26:48.566971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:54.638247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:27:34.717332+0000","last_change":"2023-12-14T18:25:54.854094+0000","last_active":"2023-12-14T18:27:34.717332+0000","last_peered":"2023-12-14T18:27:34.717332+0000","last_clean":"2023-12-14T18:27:34.717332+0000","last_became_active":"2023-12-14T18:25:54.853553+0000","last_became_peered":"2023-12-14T18:25:54.853553+0000","last_unstale":"2023-12-14T18:27:34.717332+0000","last_undegraded":"2023-12-14T18:27:34.717332+0000","last_fullsized":"2023-12-14T18:27:34.717332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:07:36.295981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562086+0000","last_change":"2023-12-14T18:25:59.332256+0000","last_active":"2023-12-14T18:26:48.562086+0000","last_peered":"2023-12-14T18:26:48.562086+0000","last_clean":"2023-12-14T18:26:48.562086+0000","last_became_active":"2023-12-14T18:25:59.331835+0000","last_became_peered":"2023-12-14T18:25:59.331835+0000","last_unstale":"2023-12-14T18:26:48.562086+0000","last_undegraded":"2023-12-14T18:26:48.562086+0000","last_fullsized":"2023-12-14T18:26:48.562086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:00.261085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559419+0000","last_change":"2023-12-14T18:25:55.309207+0000","last_active":"2023-12-14T18:26:48.559419+0000","last_peered":"2023-12-14T18:26:48.559419+0000","last_clean":"2023-12-14T18:26:48.559419+0000","last_became_active":"2023-12-14T18:25:55.308493+0000","last_became_peered":"2023-12-14T18:25:55.308493+0000","last_unstale":"2023-12-14T18:26:48.559419+0000","last_undegraded":"2023-12-14T18:26:48.559419+0000","last_fullsized":"2023-12-14T18:26:48.559419+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:49:01.825894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564209+0000","last_change":"2023-12-14T18:25:54.847692+0000","last_active":"2023-12-14T18:26:48.564209+0000","last_peered":"2023-12-14T18:26:48.564209+0000","last_clean":"2023-12-14T18:26:48.564209+0000","last_became_active":"2023-12-14T18:25:54.846611+0000","last_became_peered":"2023-12-14T18:25:54.846611+0000","last_unstale":"2023-12-14T18:26:48.564209+0000","last_undegraded":"2023-12-14T18:26:48.564209+0000","last_fullsized":"2023-12-14T18:26:48.564209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:27:58.179129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561521+0000","last_change":"2023-12-14T18:25:06.215929+0000","last_active":"2023-12-14T18:26:48.561521+0000","last_peered":"2023-12-14T18:26:48.561521+0000","last_clean":"2023-12-14T18:26:48.561521+0000","last_became_active":"2023-12-14T18:25:06.215512+0000","last_became_peered":"2023-12-14T18:25:06.215512+0000","last_unstale":"2023-12-14T18:26:48.561521+0000","last_undegraded":"2023-12-14T18:26:48.561521+0000","last_fullsized":"2023-12-14T18:26:48.561521+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:22:37.960711+0000","last_clean_scrub_stamp":"2023-12-14T18:22:37.960711+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:22.740418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566535+0000","last_change":"2023-12-14T18:25:59.762611+0000","last_active":"2023-12-14T18:26:48.566535+0000","last_peered":"2023-12-14T18:26:48.566535+0000","last_clean":"2023-12-14T18:26:48.566535+0000","last_became_active":"2023-12-14T18:25:59.762097+0000","last_became_peered":"2023-12-14T18:25:59.762097+0000","last_unstale":"2023-12-14T18:26:48.566535+0000","last_undegraded":"2023-12-14T18:26:48.566535+0000","last_fullsized":"2023-12-14T18:26:48.566535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:54:48.048416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562125+0000","last_change":"2023-12-14T18:25:55.315046+0000","last_active":"2023-12-14T18:26:48.562125+0000","last_peered":"2023-12-14T18:26:48.562125+0000","last_clean":"2023-12-14T18:26:48.562125+0000","last_became_active":"2023-12-14T18:25:55.314699+0000","last_became_peered":"2023-12-14T18:25:55.314699+0000","last_unstale":"2023-12-14T18:26:48.562125+0000","last_undegraded":"2023-12-14T18:26:48.562125+0000","last_fullsized":"2023-12-14T18:26:48.562125+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:36:49.964293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561575+0000","last_change":"2023-12-14T18:25:54.852581+0000","last_active":"2023-12-14T18:26:48.561575+0000","last_peered":"2023-12-14T18:26:48.561575+0000","last_clean":"2023-12-14T18:26:48.561575+0000","last_became_active":"2023-12-14T18:25:54.852274+0000","last_became_peered":"2023-12-14T18:25:54.852274+0000","last_unstale":"2023-12-14T18:26:48.561575+0000","last_undegraded":"2023-12-14T18:26:48.561575+0000","last_fullsized":"2023-12-14T18:26:48.561575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:20:52.709127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562632+0000","last_change":"2023-12-14T18:25:59.337470+0000","last_active":"2023-12-14T18:26:48.562632+0000","last_peered":"2023-12-14T18:26:48.562632+0000","last_clean":"2023-12-14T18:26:48.562632+0000","last_became_active":"2023-12-14T18:25:59.336983+0000","last_became_peered":"2023-12-14T18:25:59.336983+0000","last_unstale":"2023-12-14T18:26:48.562632+0000","last_undegraded":"2023-12-14T18:26:48.562632+0000","last_fullsized":"2023-12-14T18:26:48.562632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:09:26.536039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.568593+0000","last_change":"2023-12-14T18:25:54.850088+0000","last_active":"2023-12-14T18:26:48.568593+0000","last_peered":"2023-12-14T18:26:48.568593+0000","last_clean":"2023-12-14T18:26:48.568593+0000","last_became_active":"2023-12-14T18:25:54.849671+0000","last_became_peered":"2023-12-14T18:25:54.849671+0000","last_unstale":"2023-12-14T18:26:48.568593+0000","last_undegraded":"2023-12-14T18:26:48.568593+0000","last_fullsized":"2023-12-14T18:26:48.568593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:59:51.600710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564002+0000","last_change":"2023-12-14T18:25:55.312876+0000","last_active":"2023-12-14T18:26:48.564002+0000","last_peered":"2023-12-14T18:26:48.564002+0000","last_clean":"2023-12-14T18:26:48.564002+0000","last_became_active":"2023-12-14T18:25:55.312309+0000","last_became_peered":"2023-12-14T18:25:55.312309+0000","last_unstale":"2023-12-14T18:26:48.564002+0000","last_undegraded":"2023-12-14T18:26:48.564002+0000","last_fullsized":"2023-12-14T18:26:48.564002+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:37:06.768114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559014+0000","last_change":"2023-12-14T18:25:59.762197+0000","last_active":"2023-12-14T18:26:48.559014+0000","last_peered":"2023-12-14T18:26:48.559014+0000","last_clean":"2023-12-14T18:26:48.559014+0000","last_became_active":"2023-12-14T18:25:59.761730+0000","last_became_peered":"2023-12-14T18:25:59.761730+0000","last_unstale":"2023-12-14T18:26:48.559014+0000","last_undegraded":"2023-12-14T18:26:48.559014+0000","last_fullsized":"2023-12-14T18:26:48.559014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:32.285691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.577550+0000","last_change":"2023-12-14T18:26:48.577550+0000","last_active":"2023-12-14T18:26:48.577550+0000","last_peered":"2023-12-14T18:26:48.577550+0000","last_clean":"2023-12-14T18:26:48.577550+0000","last_became_active":"2023-12-14T18:26:48.569607+0000","last_became_peered":"2023-12-14T18:26:48.569607+0000","last_unstale":"2023-12-14T18:26:48.577550+0000","last_undegraded":"2023-12-14T18:26:48.577550+0000","last_fullsized":"2023-12-14T18:26:48.577550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:23.766724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559798+0000","last_change":"2023-12-14T18:25:55.320601+0000","last_active":"2023-12-14T18:26:48.559798+0000","last_peered":"2023-12-14T18:26:48.559798+0000","last_clean":"2023-12-14T18:26:48.559798+0000","last_became_active":"2023-12-14T18:25:55.320018+0000","last_became_peered":"2023-12-14T18:25:55.320018+0000","last_unstale":"2023-12-14T18:26:48.559798+0000","last_undegraded":"2023-12-14T18:26:48.559798+0000","last_fullsized":"2023-12-14T18:26:48.559798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:29.491959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563921+0000","last_change":"2023-12-14T18:25:59.334015+0000","last_active":"2023-12-14T18:26:48.563921+0000","last_peered":"2023-12-14T18:26:48.563921+0000","last_clean":"2023-12-14T18:26:48.563921+0000","last_became_active":"2023-12-14T18:25:59.333568+0000","last_became_peered":"2023-12-14T18:25:59.333568+0000","last_unstale":"2023-12-14T18:26:48.563921+0000","last_undegraded":"2023-12-14T18:26:48.563921+0000","last_fullsized":"2023-12-14T18:26:48.563921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:12.984642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563541+0000","last_change":"2023-12-14T18:25:54.849384+0000","last_active":"2023-12-14T18:26:48.563541+0000","last_peered":"2023-12-14T18:26:48.563541+0000","last_clean":"2023-12-14T18:26:48.563541+0000","last_became_active":"2023-12-14T18:25:54.848927+0000","last_became_peered":"2023-12-14T18:25:54.848927+0000","last_unstale":"2023-12-14T18:26:48.563541+0000","last_undegraded":"2023-12-14T18:26:48.563541+0000","last_fullsized":"2023-12-14T18:26:48.563541+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:56.278549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557904+0000","last_change":"2023-12-14T18:25:55.324841+0000","last_active":"2023-12-14T18:26:48.557904+0000","last_peered":"2023-12-14T18:26:48.557904+0000","last_clean":"2023-12-14T18:26:48.557904+0000","last_became_active":"2023-12-14T18:25:55.322230+0000","last_became_peered":"2023-12-14T18:25:55.322230+0000","last_unstale":"2023-12-14T18:26:48.557904+0000","last_undegraded":"2023-12-14T18:26:48.557904+0000","last_fullsized":"2023-12-14T18:26:48.557904+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:36:53.003361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563087+0000","last_change":"2023-12-14T18:25:59.762640+0000","last_active":"2023-12-14T18:26:48.563087+0000","last_peered":"2023-12-14T18:26:48.563087+0000","last_clean":"2023-12-14T18:26:48.563087+0000","last_became_active":"2023-12-14T18:25:59.762175+0000","last_became_peered":"2023-12-14T18:25:59.762175+0000","last_unstale":"2023-12-14T18:26:48.563087+0000","last_undegraded":"2023-12-14T18:26:48.563087+0000","last_fullsized":"2023-12-14T18:26:48.563087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:19.429246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559959+0000","last_change":"2023-12-14T18:25:54.849440+0000","last_active":"2023-12-14T18:26:48.559959+0000","last_peered":"2023-12-14T18:26:48.559959+0000","last_clean":"2023-12-14T18:26:48.559959+0000","last_became_active":"2023-12-14T18:25:54.849023+0000","last_became_peered":"2023-12-14T18:25:54.849023+0000","last_unstale":"2023-12-14T18:26:48.559959+0000","last_undegraded":"2023-12-14T18:26:48.559959+0000","last_fullsized":"2023-12-14T18:26:48.559959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:05.103613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566175+0000","last_change":"2023-12-14T18:25:55.320024+0000","last_active":"2023-12-14T18:26:48.566175+0000","last_peered":"2023-12-14T18:26:48.566175+0000","last_clean":"2023-12-14T18:26:48.566175+0000","last_became_active":"2023-12-14T18:25:55.319708+0000","last_became_peered":"2023-12-14T18:25:55.319708+0000","last_unstale":"2023-12-14T18:26:48.566175+0000","last_undegraded":"2023-12-14T18:26:48.566175+0000","last_fullsized":"2023-12-14T18:26:48.566175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:54:21.311660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562945+0000","last_change":"2023-12-14T18:25:59.343155+0000","last_active":"2023-12-14T18:26:48.562945+0000","last_peered":"2023-12-14T18:26:48.562945+0000","last_clean":"2023-12-14T18:26:48.562945+0000","last_became_active":"2023-12-14T18:25:59.342738+0000","last_became_peered":"2023-12-14T18:25:59.342738+0000","last_unstale":"2023-12-14T18:26:48.562945+0000","last_undegraded":"2023-12-14T18:26:48.562945+0000","last_fullsized":"2023-12-14T18:26:48.562945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:04:53.461295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.558908+0000","last_change":"2023-12-14T18:25:54.851260+0000","last_active":"2023-12-14T18:26:48.558908+0000","last_peered":"2023-12-14T18:26:48.558908+0000","last_clean":"2023-12-14T18:26:48.558908+0000","last_became_active":"2023-12-14T18:25:54.850083+0000","last_became_peered":"2023-12-14T18:25:54.850083+0000","last_unstale":"2023-12-14T18:26:48.558908+0000","last_undegraded":"2023-12-14T18:26:48.558908+0000","last_fullsized":"2023-12-14T18:26:48.558908+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:04:46.069318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.561591+0000","last_change":"2023-12-14T18:25:55.312243+0000","last_active":"2023-12-14T18:26:48.561591+0000","last_peered":"2023-12-14T18:26:48.561591+0000","last_clean":"2023-12-14T18:26:48.561591+0000","last_became_active":"2023-12-14T18:25:55.311872+0000","last_became_peered":"2023-12-14T18:25:55.311872+0000","last_unstale":"2023-12-14T18:26:48.561591+0000","last_undegraded":"2023-12-14T18:26:48.561591+0000","last_fullsized":"2023-12-14T18:26:48.561591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:52:50.350437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562857+0000","last_change":"2023-12-14T18:25:59.343489+0000","last_active":"2023-12-14T18:26:48.562857+0000","last_peered":"2023-12-14T18:26:48.562857+0000","last_clean":"2023-12-14T18:26:48.562857+0000","last_became_active":"2023-12-14T18:25:59.343066+0000","last_became_peered":"2023-12-14T18:25:59.343066+0000","last_unstale":"2023-12-14T18:26:48.562857+0000","last_undegraded":"2023-12-14T18:26:48.562857+0000","last_fullsized":"2023-12-14T18:26:48.562857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:22.082703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.565524+0000","last_change":"2023-12-14T18:25:54.850481+0000","last_active":"2023-12-14T18:26:48.565524+0000","last_peered":"2023-12-14T18:26:48.565524+0000","last_clean":"2023-12-14T18:26:48.565524+0000","last_became_active":"2023-12-14T18:25:54.849996+0000","last_became_peered":"2023-12-14T18:25:54.849996+0000","last_unstale":"2023-12-14T18:26:48.565524+0000","last_undegraded":"2023-12-14T18:26:48.565524+0000","last_fullsized":"2023-12-14T18:26:48.565524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:51:55.387650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559293+0000","last_change":"2023-12-14T18:25:55.320413+0000","last_active":"2023-12-14T18:26:48.559293+0000","last_peered":"2023-12-14T18:26:48.559293+0000","last_clean":"2023-12-14T18:26:48.559293+0000","last_became_active":"2023-12-14T18:25:55.319858+0000","last_became_peered":"2023-12-14T18:25:55.319858+0000","last_unstale":"2023-12-14T18:26:48.559293+0000","last_undegraded":"2023-12-14T18:26:48.559293+0000","last_fullsized":"2023-12-14T18:26:48.559293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:17:28.674237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563263+0000","last_change":"2023-12-14T18:25:59.762684+0000","last_active":"2023-12-14T18:26:48.563263+0000","last_peered":"2023-12-14T18:26:48.563263+0000","last_clean":"2023-12-14T18:26:48.563263+0000","last_became_active":"2023-12-14T18:25:59.762053+0000","last_became_peered":"2023-12-14T18:25:59.762053+0000","last_unstale":"2023-12-14T18:26:48.563263+0000","last_undegraded":"2023-12-14T18:26:48.563263+0000","last_fullsized":"2023-12-14T18:26:48.563263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:08.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.578387+0000","last_change":"2023-12-14T18:26:48.577964+0000","last_active":"2023-12-14T18:26:48.578387+0000","last_peered":"2023-12-14T18:26:48.578387+0000","last_clean":"2023-12-14T18:26:48.578387+0000","last_became_active":"2023-12-14T18:26:48.569637+0000","last_became_peered":"2023-12-14T18:26:48.569637+0000","last_unstale":"2023-12-14T18:26:48.578387+0000","last_undegraded":"2023-12-14T18:26:48.578387+0000","last_fullsized":"2023-12-14T18:26:48.578387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:31:31.614815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562048+0000","last_change":"2023-12-14T18:25:55.308385+0000","last_active":"2023-12-14T18:26:48.562048+0000","last_peered":"2023-12-14T18:26:48.562048+0000","last_clean":"2023-12-14T18:26:48.562048+0000","last_became_active":"2023-12-14T18:25:55.307999+0000","last_became_peered":"2023-12-14T18:25:55.307999+0000","last_unstale":"2023-12-14T18:26:48.562048+0000","last_undegraded":"2023-12-14T18:26:48.562048+0000","last_fullsized":"2023-12-14T18:26:48.562048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:14:43.573208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559867+0000","last_change":"2023-12-14T18:25:59.336728+0000","last_active":"2023-12-14T18:26:48.559867+0000","last_peered":"2023-12-14T18:26:48.559867+0000","last_clean":"2023-12-14T18:26:48.559867+0000","last_became_active":"2023-12-14T18:25:59.336343+0000","last_became_peered":"2023-12-14T18:25:59.336343+0000","last_unstale":"2023-12-14T18:26:48.559867+0000","last_undegraded":"2023-12-14T18:26:48.559867+0000","last_fullsized":"2023-12-14T18:26:48.559867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:15:04.165007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.574468+0000","last_change":"2023-12-14T18:26:48.574083+0000","last_active":"2023-12-14T18:26:48.574468+0000","last_peered":"2023-12-14T18:26:48.574468+0000","last_clean":"2023-12-14T18:26:48.574468+0000","last_became_active":"2023-12-14T18:26:48.565205+0000","last_became_peered":"2023-12-14T18:26:48.565205+0000","last_unstale":"2023-12-14T18:26:48.574468+0000","last_undegraded":"2023-12-14T18:26:48.574468+0000","last_fullsized":"2023-12-14T18:26:48.574468+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:19.409191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562445+0000","last_change":"2023-12-14T18:25:55.314216+0000","last_active":"2023-12-14T18:26:48.562445+0000","last_peered":"2023-12-14T18:26:48.562445+0000","last_clean":"2023-12-14T18:26:48.562445+0000","last_became_active":"2023-12-14T18:25:55.313863+0000","last_became_peered":"2023-12-14T18:25:55.313863+0000","last_unstale":"2023-12-14T18:26:48.562445+0000","last_undegraded":"2023-12-14T18:26:48.562445+0000","last_fullsized":"2023-12-14T18:26:48.562445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:37.001669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557585+0000","last_change":"2023-12-14T18:25:59.345357+0000","last_active":"2023-12-14T18:26:48.557585+0000","last_peered":"2023-12-14T18:26:48.557585+0000","last_clean":"2023-12-14T18:26:48.557585+0000","last_became_active":"2023-12-14T18:25:59.344971+0000","last_became_peered":"2023-12-14T18:25:59.344971+0000","last_unstale":"2023-12-14T18:26:48.557585+0000","last_undegraded":"2023-12-14T18:26:48.557585+0000","last_fullsized":"2023-12-14T18:26:48.557585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:53:54.017877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564140+0000","last_change":"2023-12-14T18:25:54.854778+0000","last_active":"2023-12-14T18:26:48.564140+0000","last_peered":"2023-12-14T18:26:48.564140+0000","last_clean":"2023-12-14T18:26:48.564140+0000","last_became_active":"2023-12-14T18:25:54.854357+0000","last_became_peered":"2023-12-14T18:25:54.854357+0000","last_unstale":"2023-12-14T18:26:48.564140+0000","last_undegraded":"2023-12-14T18:26:48.564140+0000","last_fullsized":"2023-12-14T18:26:48.564140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:46:47.614505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562523+0000","last_change":"2023-12-14T18:25:55.311982+0000","last_active":"2023-12-14T18:26:48.562523+0000","last_peered":"2023-12-14T18:26:48.562523+0000","last_clean":"2023-12-14T18:26:48.562523+0000","last_became_active":"2023-12-14T18:25:55.311596+0000","last_became_peered":"2023-12-14T18:25:55.311596+0000","last_unstale":"2023-12-14T18:26:48.562523+0000","last_undegraded":"2023-12-14T18:26:48.562523+0000","last_fullsized":"2023-12-14T18:26:48.562523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:52:04.129230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566378+0000","last_change":"2023-12-14T18:25:59.332141+0000","last_active":"2023-12-14T18:26:48.566378+0000","last_peered":"2023-12-14T18:26:48.566378+0000","last_clean":"2023-12-14T18:26:48.566378+0000","last_became_active":"2023-12-14T18:25:59.331703+0000","last_became_peered":"2023-12-14T18:25:59.331703+0000","last_unstale":"2023-12-14T18:26:48.566378+0000","last_undegraded":"2023-12-14T18:26:48.566378+0000","last_fullsized":"2023-12-14T18:26:48.566378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:58.306864+0000","last_clean_scrub_stamp":"2023-12-14T18:25:58.306864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:08:14.934634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564930+0000","last_change":"2023-12-14T18:25:54.854357+0000","last_active":"2023-12-14T18:26:48.564930+0000","last_peered":"2023-12-14T18:26:48.564930+0000","last_clean":"2023-12-14T18:26:48.564930+0000","last_became_active":"2023-12-14T18:25:54.853953+0000","last_became_peered":"2023-12-14T18:25:54.853953+0000","last_unstale":"2023-12-14T18:26:48.564930+0000","last_undegraded":"2023-12-14T18:26:48.564930+0000","last_fullsized":"2023-12-14T18:26:48.564930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:53.825781+0000","last_clean_scrub_stamp":"2023-12-14T18:25:53.825781+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:28.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562150+0000","last_change":"2023-12-14T18:25:55.314370+0000","last_active":"2023-12-14T18:26:48.562150+0000","last_peered":"2023-12-14T18:26:48.562150+0000","last_clean":"2023-12-14T18:26:48.562150+0000","last_became_active":"2023-12-14T18:25:55.310717+0000","last_became_peered":"2023-12-14T18:25:55.310717+0000","last_unstale":"2023-12-14T18:26:48.562150+0000","last_undegraded":"2023-12-14T18:26:48.562150+0000","last_fullsized":"2023-12-14T18:26:48.562150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:43:39.281554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562009+0000","last_change":"2023-12-14T18:25:55.325015+0000","last_active":"2023-12-14T18:26:48.562009+0000","last_peered":"2023-12-14T18:26:48.562009+0000","last_clean":"2023-12-14T18:26:48.562009+0000","last_became_active":"2023-12-14T18:25:55.324747+0000","last_became_peered":"2023-12-14T18:25:55.324747+0000","last_unstale":"2023-12-14T18:26:48.562009+0000","last_undegraded":"2023-12-14T18:26:48.562009+0000","last_fullsized":"2023-12-14T18:26:48.562009+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:49:45.894102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564454+0000","last_change":"2023-12-14T18:25:55.315419+0000","last_active":"2023-12-14T18:26:48.564454+0000","last_peered":"2023-12-14T18:26:48.564454+0000","last_clean":"2023-12-14T18:26:48.564454+0000","last_became_active":"2023-12-14T18:25:55.315051+0000","last_became_peered":"2023-12-14T18:25:55.315051+0000","last_unstale":"2023-12-14T18:26:48.564454+0000","last_undegraded":"2023-12-14T18:26:48.564454+0000","last_fullsized":"2023-12-14T18:26:48.564454+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:16.913094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.563425+0000","last_change":"2023-12-14T18:25:55.312889+0000","last_active":"2023-12-14T18:26:48.563425+0000","last_peered":"2023-12-14T18:26:48.563425+0000","last_clean":"2023-12-14T18:26:48.563425+0000","last_became_active":"2023-12-14T18:25:55.312541+0000","last_became_peered":"2023-12-14T18:25:55.312541+0000","last_unstale":"2023-12-14T18:26:48.563425+0000","last_undegraded":"2023-12-14T18:26:48.563425+0000","last_fullsized":"2023-12-14T18:26:48.563425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:34:54.437297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.557742+0000","last_change":"2023-12-14T18:25:55.317740+0000","last_active":"2023-12-14T18:26:48.557742+0000","last_peered":"2023-12-14T18:26:48.557742+0000","last_clean":"2023-12-14T18:26:48.557742+0000","last_became_active":"2023-12-14T18:25:55.317373+0000","last_became_peered":"2023-12-14T18:25:55.317373+0000","last_unstale":"2023-12-14T18:26:48.557742+0000","last_undegraded":"2023-12-14T18:26:48.557742+0000","last_fullsized":"2023-12-14T18:26:48.557742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:35.196672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.564064+0000","last_change":"2023-12-14T18:25:55.312639+0000","last_active":"2023-12-14T18:26:48.564064+0000","last_peered":"2023-12-14T18:26:48.564064+0000","last_clean":"2023-12-14T18:26:48.564064+0000","last_became_active":"2023-12-14T18:25:55.311571+0000","last_became_peered":"2023-12-14T18:25:55.311571+0000","last_unstale":"2023-12-14T18:26:48.564064+0000","last_undegraded":"2023-12-14T18:26:48.564064+0000","last_fullsized":"2023-12-14T18:26:48.564064+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:57:27.906838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562103+0000","last_change":"2023-12-14T18:25:55.318935+0000","last_active":"2023-12-14T18:26:48.562103+0000","last_peered":"2023-12-14T18:26:48.562103+0000","last_clean":"2023-12-14T18:26:48.562103+0000","last_became_active":"2023-12-14T18:25:55.318605+0000","last_became_peered":"2023-12-14T18:25:55.318605+0000","last_unstale":"2023-12-14T18:26:48.562103+0000","last_undegraded":"2023-12-14T18:26:48.562103+0000","last_fullsized":"2023-12-14T18:26:48.562103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:44:51.531907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566964+0000","last_change":"2023-12-14T18:25:55.308072+0000","last_active":"2023-12-14T18:26:48.566964+0000","last_peered":"2023-12-14T18:26:48.566964+0000","last_clean":"2023-12-14T18:26:48.566964+0000","last_became_active":"2023-12-14T18:25:55.307736+0000","last_became_peered":"2023-12-14T18:25:55.307736+0000","last_unstale":"2023-12-14T18:26:48.566964+0000","last_undegraded":"2023-12-14T18:26:48.566964+0000","last_fullsized":"2023-12-14T18:26:48.566964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:07:32.112350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559871+0000","last_change":"2023-12-14T18:25:55.326285+0000","last_active":"2023-12-14T18:26:48.559871+0000","last_peered":"2023-12-14T18:26:48.559871+0000","last_clean":"2023-12-14T18:26:48.559871+0000","last_became_active":"2023-12-14T18:25:55.325689+0000","last_became_peered":"2023-12-14T18:25:55.325689+0000","last_unstale":"2023-12-14T18:26:48.559871+0000","last_undegraded":"2023-12-14T18:26:48.559871+0000","last_fullsized":"2023-12-14T18:26:48.559871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:36:21.644113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.566829+0000","last_change":"2023-12-14T18:25:55.312723+0000","last_active":"2023-12-14T18:26:48.566829+0000","last_peered":"2023-12-14T18:26:48.566829+0000","last_clean":"2023-12-14T18:26:48.566829+0000","last_became_active":"2023-12-14T18:25:55.312370+0000","last_became_peered":"2023-12-14T18:25:55.312370+0000","last_unstale":"2023-12-14T18:26:48.566829+0000","last_undegraded":"2023-12-14T18:26:48.566829+0000","last_fullsized":"2023-12-14T18:26:48.566829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:17.885326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.559062+0000","last_change":"2023-12-14T18:25:55.325241+0000","last_active":"2023-12-14T18:26:48.559062+0000","last_peered":"2023-12-14T18:26:48.559062+0000","last_clean":"2023-12-14T18:26:48.559062+0000","last_became_active":"2023-12-14T18:25:55.324903+0000","last_became_peered":"2023-12-14T18:25:55.324903+0000","last_unstale":"2023-12-14T18:26:48.559062+0000","last_undegraded":"2023-12-14T18:26:48.559062+0000","last_fullsized":"2023-12-14T18:26:48.559062+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:28:13.023125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.560244+0000","last_change":"2023-12-14T18:25:55.316672+0000","last_active":"2023-12-14T18:26:48.560244+0000","last_peered":"2023-12-14T18:26:48.560244+0000","last_clean":"2023-12-14T18:26:48.560244+0000","last_became_active":"2023-12-14T18:25:55.316149+0000","last_became_peered":"2023-12-14T18:25:55.316149+0000","last_unstale":"2023-12-14T18:26:48.560244+0000","last_undegraded":"2023-12-14T18:26:48.560244+0000","last_fullsized":"2023-12-14T18:26:48.560244+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:54.715861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562903+0000","last_change":"2023-12-14T18:25:55.307654+0000","last_active":"2023-12-14T18:26:48.562903+0000","last_peered":"2023-12-14T18:26:48.562903+0000","last_clean":"2023-12-14T18:26:48.562903+0000","last_became_active":"2023-12-14T18:25:55.307145+0000","last_became_peered":"2023-12-14T18:25:55.307145+0000","last_unstale":"2023-12-14T18:26:48.562903+0000","last_undegraded":"2023-12-14T18:26:48.562903+0000","last_fullsized":"2023-12-14T18:26:48.562903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:54:48.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-14T18:26:48.562082+0000","last_change":"2023-12-14T18:25:55.310273+0000","last_active":"2023-12-14T18:26:48.562082+0000","last_peered":"2023-12-14T18:26:48.562082+0000","last_clean":"2023-12-14T18:26:48.562082+0000","last_became_active":"2023-12-14T18:25:55.309966+0000","last_became_peered":"2023-12-14T18:25:55.309966+0000","last_unstale":"2023-12-14T18:26:48.562082+0000","last_undegraded":"2023-12-14T18:26:48.562082+0000","last_fullsized":"2023-12-14T18:26:48.562082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:25:54.828872+0000","last_clean_scrub_stamp":"2023-12-14T18:25:54.828872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:40:01.866310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":3542208,"data_compressed":209316,"data_compressed_allocated":270336,"data_compressed_original":3489792,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68351,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":469,"num_read_kb":305,"num_write":160,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1546,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":196413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514527,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739028,"statfs":{"total":95995035648,"available":95988764672,"internally_reserved":0,"allocated":954368,"data_stored":484012,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76900000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68400000000000005,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66800000000000004,"15min":0.67200000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84099999999999997,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.498,"15min":0.498},"min":{"1min":0.318,"5min":0.311,"15min":0.311},"max":{"1min":0.69399999999999995,"5min":0.751,"15min":0.751},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.70099999999999996,"15min":0.70199999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85299999999999998,"5min":1.268,"15min":1.268},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.65000000000000002,"15min":0.65300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90000000000000002,"5min":1.097,"15min":1.097},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70899999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67400000000000004,"15min":0.67500000000000004},"min":{"1min":0.50800000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.79500000000000004,"5min":1.2,"15min":1.2},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.61099999999999999},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79400000000000004,"5min":0.878,"15min":0.878},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51300000000000001,"5min":0.503,"15min":0.503},"max":{"1min":0.88200000000000001,"5min":1.075,"15min":1.075},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78800000000000003,"15min":0.79100000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.05,"15min":1.05},"last":0.72799999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.625}]},{"osd":9,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.59799999999999998,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.86199999999999999,"5min":1.115,"15min":1.115},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.68000000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.0009999999999999,"5min":1.389,"15min":1.389},"last":0.76300000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.628,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98599999999999999,"5min":1.147,"15min":1.147},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92800000000000005,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.752}]}]},{"osd":10,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6584,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738568,"statfs":{"total":95995035648,"available":95988293632,"internally_reserved":0,"allocated":1097728,"data_stored":1135398,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54300000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.45800000000000002,"15min":0.46000000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59899999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.748,"5min":0.749,"15min":0.749},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.437,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82899999999999996,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.76000000000000001,"15min":0.78900000000000003},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79300000000000004,"5min":3.4449999999999998,"15min":3.4449999999999998},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55000000000000004,"15min":0.55300000000000005},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.71299999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.501,"15min":0.50700000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.57899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.63},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78500000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.64700000000000002,"15min":0.65100000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.80300000000000005,"5min":3.3119999999999998,"15min":3.3119999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71899999999999997,"15min":0.72899999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.84199999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.74199999999999999,"15min":0.747},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.628},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.56200000000000006}]},{"osd":9,"last update":"Thu Dec 14 18:27:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":1.252,"15min":1.252},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.93600000000000005,"15min":0.97999999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.85699999999999998,"5min":5.4960000000000004,"15min":5.4960000000000004},"last":0.61799999999999999}]},{"osd":11,"last update":"Thu Dec 14 18:27:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78500000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88500000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.90400000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.68500000000000005}]}]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6336,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738816,"statfs":{"total":95995035648,"available":95988547584,"internally_reserved":0,"allocated":974848,"data_stored":488344,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.46400000000000002,"15min":0.47299999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72799999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.437}]},{"osd":1,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.504},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38,"15min":0.38},"max":{"1min":1.01,"5min":1.028,"15min":1.028},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.68500000000000005,"15min":0.69499999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76900000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75600000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.70999999999999996,"15min":0.71699999999999997},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.97799999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0509999999999999,"5min":1.081,"15min":1.081},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.036,"15min":1.036},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:25:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47099999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1200000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.081,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67000000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0189999999999999,"5min":1.123,"15min":1.123},"last":0.79800000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.81799999999999995,"15min":0.83099999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0840000000000001,"5min":1.105,"15min":1.105},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.84299999999999997,"15min":0.85699999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0329999999999999,"5min":1.121,"15min":1.121},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:27:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.74199999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6460,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738692,"statfs":{"total":95995035648,"available":95988420608,"internally_reserved":0,"allocated":970752,"data_stored":484168,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39800000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78400000000000003,"5min":0.875,"15min":0.875},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.44900000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.34200000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72499999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.49199999999999999,"15min":0.48799999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63400000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.32400000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.83799999999999997,"5min":0.879,"15min":0.879},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.45300000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76700000000000002,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.871},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63900000000000001,"15min":0.629},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82199999999999995,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54400000000000004},"min":{"1min":0.38700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78800000000000003,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89500000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71399999999999997,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.95899999999999996,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.75600000000000001,"15min":0.73199999999999998},"min":{"1min":0.66900000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.90400000000000003}]},{"osd":9,"last update":"Thu Dec 14 18:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.628,"5min":0.502,"15min":0.502},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.498,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.96899999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.753,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95599999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.78500000000000003},"min":{"1min":0.68500000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93200000000000005}]},{"osd":11,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.004}]}]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6564,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738588,"statfs":{"total":95995035648,"available":95988314112,"internally_reserved":0,"allocated":1011712,"data_stored":1079376,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.39600000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.40100000000000002},"min":{"1min":0.34899999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.40100000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72799999999999998,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.53000000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45900000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.48599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73399999999999999,"5min":1.8,"15min":1.8},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47999999999999998,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.30299999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.626}]},{"osd":5,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.80100000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60899999999999999,"15min":0.59099999999999997},"min":{"1min":0.50700000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.67100000000000004},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78900000000000003,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.503,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71599999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:27:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.58199999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.83799999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69099999999999995,"15min":0.67500000000000004},"min":{"1min":0.51900000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84099999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70899999999999996},"min":{"1min":0.55100000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.82299999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.75,"15min":0.753},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.85799999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593777,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6440,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738712,"statfs":{"total":95995035648,"available":95988441088,"internally_reserved":0,"allocated":950272,"data_stored":479424,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.34599999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.47699999999999998,"15min":0.45700000000000002},"min":{"1min":0.33800000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.435}]},{"osd":1,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61499999999999999,"15min":0.59199999999999997},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.86199999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.64200000000000002,"15min":0.61399999999999999},"min":{"1min":0.47799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51700000000000002,"15min":0.50700000000000001},"min":{"1min":0.36199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.56200000000000006},"min":{"1min":0.35599999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.63500000000000001,"15min":0.60099999999999998},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.988,"5min":1.988,"15min":1.988},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64500000000000002,"15min":0.63},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.70699999999999996,"15min":0.67700000000000005},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.72199999999999998,"15min":0.68999999999999995},"min":{"1min":0.51400000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.73699999999999999,"15min":0.71899999999999997},"min":{"1min":0.51400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.56100000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.88200000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:27:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76000000000000001,"15min":0.75900000000000001},"min":{"1min":0.60899999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79500000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.57899999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.127,"5min":1.133,"15min":1.133},"last":0.95899999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.59899999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.89300000000000002}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7464,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737688,"statfs":{"total":95995035648,"available":95987392512,"internally_reserved":0,"allocated":1015808,"data_stored":539450,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54000000000000004,"15min":0.51000000000000001},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79000000000000004,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57399999999999995,"15min":0.51400000000000001},"min":{"1min":0.36299999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.75600000000000001,"5min":5.0800000000000001,"15min":5.0800000000000001},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.52500000000000002,"15min":0.495},"min":{"1min":0.36899999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.53000000000000003,"15min":0.47999999999999998},"min":{"1min":0.40200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.55600000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82799999999999996,"5min":5.3150000000000004,"15min":5.3150000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64200000000000002,"15min":0.55600000000000005},"min":{"1min":0.46200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82899999999999996,"5min":5.3460000000000001,"15min":5.3460000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:27:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.25,"15min":0.25},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.45200000000000001,"15min":0.41099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.44}]},{"osd":5,"last update":"Thu Dec 14 18:26:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.83499999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.61599999999999999,"15min":0.58199999999999996},"min":{"1min":0.53500000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61799999999999999,"15min":0.58299999999999996},"min":{"1min":0.40799999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82899999999999996,"5min":1.657,"15min":1.657},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.58599999999999997},"min":{"1min":0.48699999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56899999999999995,"15min":0.54700000000000004},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.872,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48399999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.628},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72799999999999998,"15min":0.70699999999999996},"min":{"1min":0.55100000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.92500000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71799999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.59299999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1950000000000001,"5min":3.3570000000000002,"15min":3.3570000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.93100000000000005,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":5.1159999999999997,"15min":5.1159999999999997},"last":0.70899999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.78800000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.79000000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7360,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737792,"statfs":{"total":95995035648,"available":95987499008,"internally_reserved":0,"allocated":974848,"data_stored":480812,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.57799999999999996,"15min":0.51600000000000001},"min":{"1min":0.40600000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57599999999999996,"15min":0.51400000000000001},"min":{"1min":0.376,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58499999999999996,"15min":0.54500000000000004},"min":{"1min":0.42999999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90700000000000003,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.57299999999999995,"15min":0.54900000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.622,"15min":0.58799999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.94199999999999995,"5min":1.091,"15min":1.091},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64700000000000002,"15min":0.61099999999999999},"min":{"1min":0.48699999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.63400000000000001,"15min":0.60899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":2.6309999999999998,"5min":2.6309999999999998,"15min":2.6309999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.47899999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93600000000000005,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:27:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.83499999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:27:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.6640000000000001,"5min":2.6640000000000001,"15min":2.6640000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79700000000000004,"15min":0.80900000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.048,"5min":1.079,"15min":1.079},"last":0.877}]},{"osd":9,"last update":"Thu Dec 14 18:26:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.58399999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.82499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":0.72399999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.55300000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1850000000000001,"5min":1.258,"15min":1.258},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.077,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.80700000000000005}]},{"osd":11,"last update":"Thu Dec 14 18:27:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.86599999999999999,"15min":0.871},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.125,"5min":1.2,"15min":1.2},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.88200000000000001,"15min":0.877},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.694,"5min":2.694,"15min":2.694},"last":0.748}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7540,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737612,"statfs":{"total":95995035648,"available":95987314688,"internally_reserved":0,"allocated":1028096,"data_stored":539607,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86799999999999999,"5min":1.484,"15min":1.484},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.40300000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.56399999999999995,"15min":0.53400000000000003},"min":{"1min":0.52300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84899999999999998,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.47799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83499999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.83999999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.443},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81000000000000005,"5min":1.514,"15min":1.514},"last":0.69599999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89200000000000002,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66700000000000004,"15min":0.65100000000000002},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.78300000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.46100000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.57099999999999995},"min":{"1min":0.44400000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.88700000000000001,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60299999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.746,"15min":0.73199999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.67400000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.67000000000000004,"15min":0.65400000000000003},"min":{"1min":0.54700000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.82499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:26:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79700000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":958464,"data_stored":479345,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.43099999999999999,"15min":0.39100000000000001},"min":{"1min":0.29699999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.41899999999999998},"min":{"1min":0.30599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.63600000000000001,"15min":0.55800000000000005},"min":{"1min":0.39500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.5800000000000001,"5min":6.5800000000000001,"15min":6.5800000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.61699999999999999,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.67200000000000004,"15min":0.60799999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":6.5419999999999998,"5min":6.5419999999999998,"15min":6.5419999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:27:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.64600000000000002,"15min":0.61299999999999999},"min":{"1min":0.50600000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.68999999999999995,"15min":0.63500000000000001},"min":{"1min":0.44800000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":6.4740000000000002,"5min":6.4740000000000002,"15min":6.4740000000000002},"last":0.621}]},{"osd":5,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.66400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.38600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.68200000000000005,"15min":0.64000000000000001},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":6.6920000000000002,"15min":6.6920000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:27:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":0.79600000000000004,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":6.617,"5min":6.617,"15min":6.617},"last":0.73699999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.68600000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:26:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.83299999999999996,"15min":0.81399999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":6.7249999999999996,"5min":6.7249999999999996,"15min":6.7249999999999996},"last":0.77900000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:27:05 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.79400000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":0.753},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.85599999999999998,"15min":0.85799999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:26:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.6550000000000002,"5min":6.6550000000000002,"15min":6.6550000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.72799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986232,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6524,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738628,"statfs":{"total":95995035648,"available":95988355072,"internally_reserved":0,"allocated":970752,"data_stored":484990,"data_compressed":17659,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.36299999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.379,"15min":0.34499999999999997},"min":{"1min":0.29699999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.66400000000000003,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.315},{"interface":"front","average":{"1min":0.44,"5min":0.39200000000000002,"15min":0.36399999999999999},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64500000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.435,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.83499999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.58299999999999996,"15min":0.55100000000000005},"min":{"1min":0.42299999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.78000000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:27:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78900000000000003,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.442},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73499999999999999,"5min":1.121,"15min":1.121},"last":0.42899999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:26:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.872,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85999999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.51100000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:26:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.376},{"interface":"front","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.59499999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.85999999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.57599999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.41799999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.41099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.83599999999999997,"5min":0.879,"15min":0.879},"last":0.40600000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:26:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.67100000000000004,"15min":0.67600000000000005},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84899999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.60999999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52200000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":1.161,"15min":1.161},"last":0.59899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:27:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.754},"min":{"1min":0.56999999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.85699999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.71099999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53300000000000003}]},{"osd":11,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58799999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822709,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6704,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738448,"statfs":{"total":95995035648,"available":95988170752,"internally_reserved":0,"allocated":1024000,"data_stored":1075285,"data_compressed":53363,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.52400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63800000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66600000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52700000000000002,"15min":0.51800000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70999999999999996,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.192,"15min":0.192},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41099999999999998,"15min":0.40999999999999998},"min":{"1min":0.314,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.57499999999999996,"15min":0.55000000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":4.7290000000000001,"5min":4.7290000000000001,"15min":4.7290000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.621,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.391,"5min":6.391,"15min":6.391},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:26:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82899999999999996,"5min":0.874,"15min":0.874},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.66800000000000004,"15min":0.622},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.3300000000000001,"5min":6.3300000000000001,"15min":6.3300000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.66600000000000004,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":4.7130000000000001,"5min":4.7130000000000001,"15min":4.7130000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Dec 14 18:27:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81200000000000006,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.45800000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.751}]},{"osd":8,"last update":"Thu Dec 14 18:27:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.84599999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.627},"min":{"1min":0.51200000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80000000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58999999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:26:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.68400000000000005,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":4.7709999999999999,"5min":4.7709999999999999,"15min":4.7709999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":0.71099999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:27:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.67600000000000005,"15min":0.66700000000000004},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.84499999999999997,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66500000000000004,"15min":0.65900000000000003},"min":{"1min":0.58599999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34068,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35704,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:27:40.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:40 smithi176 ceph-mon[43506]: pgmap v283: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:27:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:40 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2761074991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:40.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:40 smithi176 ceph-mon[43506]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.027+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2606751514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c981035f0 msgr2=0x7f9c981059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.027+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2606751514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c981035f0 0x7f9c981059d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009990 tx=0x7f9c8c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2606751514 shutdown_connections 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2606751514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c98105f10 0x7f9c98108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2606751514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c98100cd0 0x7f9c981030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2606751514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c981035f0 0x7f9c981059d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2606751514 >> 172.21.15.31:0/2606751514 conn(0x7f9c980fa750 msgr2=0x7f9c980fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:41.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2606751514 shutdown_connections 2023-12-14T18:27:41.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2606751514 wait complete. 2023-12-14T18:27:41.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 Processor -- start 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.028+0000 7f9c9dc8c700 1 -- start start 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c9dc8c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c98100cd0 0x7f9c98100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c9dc8c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c981035f0 0x7f9c980fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c9dc8c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 0x7f9c980ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c9dc8c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980ff960 con 0x7f9c98105f10 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c9dc8c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980ffad0 con 0x7f9c98100cd0 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c9dc8c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819ffd0 con 0x7f9c981035f0 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 0x7f9c980ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 0x7f9c980ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53172/0 (socket says 172.21.15.31:53172) 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 -- 172.21.15.31:0/2377586573 learned_addr learned my addr 172.21.15.31:0/2377586573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:41.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c96ffd700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c981035f0 0x7f9c980fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c977fe700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c98100cd0 0x7f9c98100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 -- 172.21.15.31:0/2377586573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c981035f0 msgr2=0x7f9c980fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c981035f0 0x7f9c980fec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 -- 172.21.15.31:0/2377586573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c98100cd0 msgr2=0x7f9c98100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c98100cd0 0x7f9c98100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 -- 172.21.15.31:0/2377586573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0095f0 con 0x7f9c98105f10 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.029+0000 7f9c97fff700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 0x7f9c980ff1c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000eb80 tx=0x7f9c9000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.030+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90018070 con 0x7f9c98105f10 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.030+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c9000f040 con 0x7f9c98105f10 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.030+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9810bbd0 con 0x7f9c98105f10 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.030+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000ce80 con 0x7f9c98105f10 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.030+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9810ba40 con 0x7f9c98105f10 2023-12-14T18:27:41.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.031+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c980fc2c0 con 0x7f9c98105f10 2023-12-14T18:27:41.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.032+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9c90007500 con 0x7f9c98105f10 2023-12-14T18:27:41.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.032+0000 7f9c94ff9700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c800a1260 0x7f9c800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.033+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9c900cc890 con 0x7f9c98105f10 2023-12-14T18:27:41.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.033+0000 7f9c977fe700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c800a1260 0x7f9c800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.033+0000 7f9c977fe700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c800a1260 0x7f9c800a3710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800bbd0 tx=0x7f9c8800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:41.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.037+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9c90095c00 con 0x7f9c98105f10 2023-12-14T18:27:41.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.247+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c98061700 con 0x7f9c98105f10 2023-12-14T18:27:41.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.249+0000 7f9c94ff9700 1 -- 172.21.15.31:0/2377586573 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f9c900b7e60 con 0x7f9c98105f10 2023-12-14T18:27:41.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:41.251 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c800a1260 msgr2=0x7f9c800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c800a1260 0x7f9c800a3710 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800bbd0 tx=0x7f9c8800b5b0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 msgr2=0x7f9c980ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 0x7f9c980ff1c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000eb80 tx=0x7f9c9000ee90 comp rx=0 tx=0).stop 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 shutdown_connections 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c98105f10 0x7f9c980ff1c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c98100cd0 0x7f9c98100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f9c800a1260 0x7f9c800a3710 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 --2- 172.21.15.31:0/2377586573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9c981035f0 0x7f9c980fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 >> 172.21.15.31:0/2377586573 conn(0x7f9c980fa750 msgr2=0x7f9c98101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 shutdown_connections 2023-12-14T18:27:41.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.251+0000 7f9c9dc8c700 1 -- 172.21.15.31:0/2377586573 wait complete. 2023-12-14T18:27:41.267 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:27:41.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 -- 172.21.15.31:0/1510318427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 msgr2=0x7f2b8c10ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 --2- 172.21.15.31:0/1510318427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c10ebe0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b0a0 tx=0x7f2b8400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 -- 172.21.15.31:0/1510318427 shutdown_connections 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 --2- 172.21.15.31:0/1510318427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c10ebe0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 --2- 172.21.15.31:0/1510318427 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 0x7f2b8c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 --2- 172.21.15.31:0/1510318427 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b8c073550 0x7f2b8c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 -- 172.21.15.31:0/1510318427 >> 172.21.15.31:0/1510318427 conn(0x7f2b8c0fb850 msgr2=0x7f2b8c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 -- 172.21.15.31:0/1510318427 shutdown_connections 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 -- 172.21.15.31:0/1510318427 wait complete. 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.406+0000 7f2b93275700 1 Processor -- start 2023-12-14T18:27:41.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 -- start start 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b8c073550 0x7f2b8c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 0x7f2b8c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c1a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1a4200 con 0x7f2b8c10c800 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1a4340 con 0x7f2b8c074b30 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b93275700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1a44d0 con 0x7f2b8c073550 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c1a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c1a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53196/0 (socket says 172.21.15.31:53196) 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 -- 172.21.15.31:0/2826842120 learned_addr learned my addr 172.21.15.31:0/2826842120 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91011700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b8c073550 0x7f2b8c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b90810700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 0x7f2b8c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 -- 172.21.15.31:0/2826842120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b8c073550 msgr2=0x7f2b8c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b8c073550 0x7f2b8c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 -- 172.21.15.31:0/2826842120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 msgr2=0x7f2b8c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 0x7f2b8c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 -- 172.21.15.31:0/2826842120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84009d00 con 0x7f2b8c10c800 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b90810700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 0x7f2b8c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.407+0000 7f2b91812700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c1a3c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b780 tx=0x7f2b8400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.408+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8400e040 con 0x7f2b8c10c800 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.408+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b840094a0 con 0x7f2b8c10c800 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.408+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c1121c0 con 0x7f2b8c10c800 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.408+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84005330 con 0x7f2b8c10c800 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.408+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c112030 con 0x7f2b8c10c800 2023-12-14T18:27:41.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.409+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c102350 con 0x7f2b8c10c800 2023-12-14T18:27:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.410+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2b8401da30 con 0x7f2b8c10c800 2023-12-14T18:27:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.410+0000 7f2b827fc700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2b780a11e0 0x7f2b780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.411+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b840ce4a0 con 0x7f2b8c10c800 2023-12-14T18:27:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.411+0000 7f2b91011700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2b780a11e0 0x7f2b780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.411+0000 7f2b91011700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2b780a11e0 0x7f2b780a3690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c074f00 tx=0x7f2b88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.415+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2b840977b0 con 0x7f2b8c10c800 2023-12-14T18:27:41.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.625+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b8c0679f0 con 0x7f2b8c10c800 2023-12-14T18:27:41.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.626+0000 7f2b827fc700 1 -- 172.21.15.31:0/2826842120 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b8409add0 con 0x7f2b8c10c800 2023-12-14T18:27:41.627 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:27:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2b780a11e0 msgr2=0x7f2b780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2b780a11e0 0x7f2b780a3690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c074f00 tx=0x7f2b88009260 comp rx=0 tx=0).stop 2023-12-14T18:27:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 msgr2=0x7f2b8c1a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c1a3c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400b780 tx=0x7f2b8400bd60 comp rx=0 tx=0).stop 2023-12-14T18:27:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 shutdown_connections 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b8c10c800 0x7f2b8c1a3c30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f2b780a11e0 0x7f2b780a3690 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b8c073550 0x7f2b8c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 --2- 172.21.15.31:0/2826842120 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b8c074b30 0x7f2b8c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 >> 172.21.15.31:0/2826842120 conn(0x7f2b8c0fb850 msgr2=0x7f2b8c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 shutdown_connections 2023-12-14T18:27:41.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.628+0000 7f2b93275700 1 -- 172.21.15.31:0/2826842120 wait complete. 2023-12-14T18:27:41.644 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:27:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2546954983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8100cd0 msgr2=0x7f6dd81030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2546954983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8100cd0 0x7f6dd81030b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009990 tx=0x7f6dc8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2546954983 shutdown_connections 2023-12-14T18:27:41.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2546954983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dd8105f10 0x7f6dd8108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2546954983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8100cd0 0x7f6dd81030b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2546954983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd81035f0 0x7f6dd81059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2546954983 >> 172.21.15.31:0/2546954983 conn(0x7f6dd80fa750 msgr2=0x7f6dd80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2546954983 shutdown_connections 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.781+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2546954983 wait complete. 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.782+0000 7f6dde9e0700 1 Processor -- start 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.782+0000 7f6dde9e0700 1 -- start start 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.782+0000 7f6dde9e0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd8100cd0 0x7f6dd8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.782+0000 7f6dde9e0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dd81035f0 0x7f6dd81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.782+0000 7f6dde9e0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 0x7f6dd819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dde9e0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819db00 con 0x7f6dd8105f10 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dde9e0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819dc40 con 0x7f6dd8100cd0 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dde9e0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd819ddd0 con 0x7f6dd81035f0 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 0x7f6dd819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 0x7f6dd819d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53210/0 (socket says 172.21.15.31:53210) 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 -- 172.21.15.31:0/2650643005 learned_addr learned my addr 172.21.15.31:0/2650643005 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dd7fff700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd8100cd0 0x7f6dd8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dd77fe700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dd81035f0 0x7f6dd81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 -- 172.21.15.31:0/2650643005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dd81035f0 msgr2=0x7f6dd81964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dd81035f0 0x7f6dd81964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 -- 172.21.15.31:0/2650643005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd8100cd0 msgr2=0x7f6dd8195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd8100cd0 0x7f6dd8195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 -- 172.21.15.31:0/2650643005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80095f0 con 0x7f6dd8105f10 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6ddcf7d700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 0x7f6dd819d530 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000ada0 tx=0x7f6dd000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000f070 con 0x7f6dd8105f10 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd0007b30 con 0x7f6dd8105f10 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd810bbd0 con 0x7f6dd8105f10 2023-12-14T18:27:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.783+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00053e0 con 0x7f6dd8105f10 2023-12-14T18:27:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.784+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd810ba40 con 0x7f6dd8105f10 2023-12-14T18:27:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.784+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd80fc2c0 con 0x7f6dd8105f10 2023-12-14T18:27:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.786+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f6dd000c980 con 0x7f6dd8105f10 2023-12-14T18:27:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.787+0000 7f6dd57fa700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6dc00a11e0 0x7f6dc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:41.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.787+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6dd00cc970 con 0x7f6dd8105f10 2023-12-14T18:27:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.787+0000 7f6dd7fff700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6dc00a11e0 0x7f6dc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.788+0000 7f6dd7fff700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6dc00a11e0 0x7f6dc00a3690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b5c0 tx=0x7f6dc8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.790+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6dd0095ce0 con 0x7f6dd8105f10 2023-12-14T18:27:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:41 smithi204 ceph-mon[43866]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:41 smithi204 ceph-mon[43866]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:41 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2377586573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:41.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:41 smithi031 ceph-mon[42085]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:41 smithi031 ceph-mon[42085]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:41 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2377586573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:41 smithi176 ceph-mon[43506]: from='client.15477 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:41 smithi176 ceph-mon[43506]: from='client.15483 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:27:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:41 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2377586573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.998+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6dd8067160 con 0x7f6dd8105f10 2023-12-14T18:27:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:41.998+0000 7f6dd57fa700 1 -- 172.21.15.31:0/2650643005 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6dd0099300 con 0x7f6dd8105f10 2023-12-14T18:27:42.000 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6dc00a11e0 msgr2=0x7f6dc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6dc00a11e0 0x7f6dc00a3690 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800b5c0 tx=0x7f6dc8005fd0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 msgr2=0x7f6dd819d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 0x7f6dd819d530 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000ada0 tx=0x7f6dd000c330 comp rx=0 tx=0).stop 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 shutdown_connections 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd8105f10 0x7f6dd819d530 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6dd8100cd0 0x7f6dd8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f6dc00a11e0 0x7f6dc00a3690 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 --2- 172.21.15.31:0/2650643005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dd81035f0 0x7f6dd81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 >> 172.21.15.31:0/2650643005 conn(0x7f6dd80fa750 msgr2=0x7f6dd8101930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:42.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 shutdown_connections 2023-12-14T18:27:42.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.001+0000 7f6dde9e0700 1 -- 172.21.15.31:0/2650643005 wait complete. 2023-12-14T18:27:42.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:27:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 -- 172.21.15.31:0/1267845697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 msgr2=0x7efee4100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 --2- 172.21.15.31:0/1267845697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4100760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efed4009990 tx=0x7efed4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 -- 172.21.15.31:0/1267845697 shutdown_connections 2023-12-14T18:27:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 --2- 172.21.15.31:0/1267845697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee4100ca0 0x7efee4107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 --2- 172.21.15.31:0/1267845697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee40ff120 0x7efee40ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 --2- 172.21.15.31:0/1267845697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4100760 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 -- 172.21.15.31:0/1267845697 >> 172.21.15.31:0/1267845697 conn(0x7efee40fa740 msgr2=0x7efee40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 -- 172.21.15.31:0/1267845697 shutdown_connections 2023-12-14T18:27:42.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.156+0000 7efeec0d7700 1 -- 172.21.15.31:0/1267845697 wait complete. 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 Processor -- start 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 -- start start 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40ff120 0x7efee4195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee4100ca0 0x7efee419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419db40 con 0x7efee4100310 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419dc80 con 0x7efee4100ca0 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeec0d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419de10 con 0x7efee40ff120 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53236/0 (socket says 172.21.15.31:53236) 2023-12-14T18:27:42.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 -- 172.21.15.31:0/3566306821 learned_addr learned my addr 172.21.15.31:0/3566306821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9e73700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40ff120 0x7efee4195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeea674700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee4100ca0 0x7efee419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 -- 172.21.15.31:0/3566306821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40ff120 msgr2=0x7efee4195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40ff120 0x7efee4195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 -- 172.21.15.31:0/3566306821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee4100ca0 msgr2=0x7efee419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee4100ca0 0x7efee419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efee9672700 1 -- 172.21.15.31:0/3566306821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40095f0 con 0x7efee4100310 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.157+0000 7efeea674700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee4100ca0 0x7efee419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.158+0000 7efee9672700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4196500 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efed400b620 tx=0x7efed4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.158+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed401d070 con 0x7efee4100310 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.158+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed4005030 con 0x7efee4100310 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.158+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee410aa80 con 0x7efee4100310 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.158+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4004bf0 con 0x7efee4100310 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.158+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee410a8f0 con 0x7efee4100310 2023-12-14T18:27:42.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.159+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee419e3b0 con 0x7efee4100310 2023-12-14T18:27:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.161+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efed4004d50 con 0x7efee4100310 2023-12-14T18:27:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.161+0000 7efedaffd700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efed00a11e0 0x7efed00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.162+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efed40ce2b0 con 0x7efee4100310 2023-12-14T18:27:42.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.162+0000 7efee9e73700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efed00a11e0 0x7efed00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.162+0000 7efee9e73700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efed00a11e0 0x7efed00a3690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efee4100170 tx=0x7efee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:42.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.165+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efed40975c0 con 0x7efee4100310 2023-12-14T18:27:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.372+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efee419ea50 con 0x7efee4100310 2023-12-14T18:27:42.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.373+0000 7efedaffd700 1 -- 172.21.15.31:0/3566306821 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7efed409abe0 con 0x7efee4100310 2023-12-14T18:27:42.375 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:27:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efed00a11e0 msgr2=0x7efed00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efed00a11e0 0x7efed00a3690 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efee4100170 tx=0x7efee0009260 comp rx=0 tx=0).stop 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 msgr2=0x7efee4196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4196500 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efed400b620 tx=0x7efed4004730 comp rx=0 tx=0).stop 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 shutdown_connections 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efee4100ca0 0x7efee419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7efed00a11e0 0x7efed00a3690 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40ff120 0x7efee4195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 --2- 172.21.15.31:0/3566306821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efee4100310 0x7efee4196500 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 >> 172.21.15.31:0/3566306821 conn(0x7efee40fa740 msgr2=0x7efee4067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 shutdown_connections 2023-12-14T18:27:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.375+0000 7efeec0d7700 1 -- 172.21.15.31:0/3566306821 wait complete. 2023-12-14T18:27:42.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 -- 172.21.15.31:0/3034372240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100ca0 msgr2=0x7fafe0107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 --2- 172.21.15.31:0/3034372240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100ca0 0x7fafe0107560 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fafd800b0a0 tx=0x7fafd800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 -- 172.21.15.31:0/3034372240 shutdown_connections 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 --2- 172.21.15.31:0/3034372240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100ca0 0x7fafe0107560 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 --2- 172.21.15.31:0/3034372240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 0x7fafe00ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 --2- 172.21.15.31:0/3034372240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe0100310 0x7fafe0100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 -- 172.21.15.31:0/3034372240 >> 172.21.15.31:0/3034372240 conn(0x7fafe00fa740 msgr2=0x7fafe00fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:42.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 -- 172.21.15.31:0/3034372240 shutdown_connections 2023-12-14T18:27:42.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.531+0000 7fafe7266700 1 -- 172.21.15.31:0/3034372240 wait complete. 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 Processor -- start 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 -- start start 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 0x7fafe0196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 0x7fafe01965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe0100ca0 0x7fafe019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019dc10 con 0x7fafe0100310 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019dd50 con 0x7fafe00ff120 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe7266700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019dee0 con 0x7fafe0100ca0 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 0x7fafe01965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 0x7fafe01965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53258/0 (socket says 172.21.15.31:53258) 2023-12-14T18:27:42.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 -- 172.21.15.31:0/1617497897 learned_addr learned my addr 172.21.15.31:0/1617497897 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe5803700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe0100ca0 0x7fafe019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe5002700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 0x7fafe0196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 -- 172.21.15.31:0/1617497897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe0100ca0 msgr2=0x7fafe019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe0100ca0 0x7fafe019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 -- 172.21.15.31:0/1617497897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 msgr2=0x7fafe0196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 0x7fafe0196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe4801700 1 -- 172.21.15.31:0/1617497897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd8009d00 con 0x7fafe0100310 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.532+0000 7fafe5002700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 0x7fafe0196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.533+0000 7fafe4801700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 0x7fafe01965d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafd000e9a0 tx=0x7fafd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.533+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0009790 con 0x7fafe0100310 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.533+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafd000cbd0 con 0x7fafe0100310 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.533+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe010bc50 con 0x7fafe0100310 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.533+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd0018680 con 0x7fafe0100310 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.533+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe010bac0 con 0x7fafe0100310 2023-12-14T18:27:42.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.534+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe01065c0 con 0x7fafe0100310 2023-12-14T18:27:42.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.536+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fafd000cd40 con 0x7fafe0100310 2023-12-14T18:27:42.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.536+0000 7fafd67fc700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fafcc0a11e0 0x7fafcc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.537+0000 7fafe5002700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fafcc0a11e0 0x7fafcc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.537+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fafd0014070 con 0x7fafe0100310 2023-12-14T18:27:42.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.537+0000 7fafe5002700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fafcc0a11e0 0x7fafcc0a3690 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fafe0100170 tx=0x7fafdc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:42.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.540+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fafd00965c0 con 0x7fafe0100310 2023-12-14T18:27:42.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:42 smithi031 ceph-mon[42085]: pgmap v284: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:27:42.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:42 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2826842120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:42 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2650643005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:42 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3566306821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.746+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fafe0061960 con 0x7fafe0100310 2023-12-14T18:27:42.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.747+0000 7fafd67fc700 1 -- 172.21.15.31:0/1617497897 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fafd0099be0 con 0x7fafe0100310 2023-12-14T18:27:42.748 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fafcc0a11e0 msgr2=0x7fafcc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fafcc0a11e0 0x7fafcc0a3690 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fafe0100170 tx=0x7fafdc009260 comp rx=0 tx=0).stop 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 msgr2=0x7fafe01965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 0x7fafe01965d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafd000e9a0 tx=0x7fafd000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 shutdown_connections 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fafcc0a11e0 0x7fafcc0a3690 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fafe0100ca0 0x7fafe019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fafe00ff120 0x7fafe0196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 --2- 172.21.15.31:0/1617497897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fafe0100310 0x7fafe01965d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 >> 172.21.15.31:0/1617497897 conn(0x7fafe00fa740 msgr2=0x7fafe0103700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:42.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 shutdown_connections 2023-12-14T18:27:42.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.749+0000 7fafe7266700 1 -- 172.21.15.31:0/1617497897 wait complete. 2023-12-14T18:27:42.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:27:42.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:42 smithi204 ceph-mon[43866]: pgmap v284: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:27:42.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:42 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2826842120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:42 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2650643005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:42 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3566306821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 -- 172.21.15.31:0/3462168095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 msgr2=0x7f0024101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 --2- 172.21.15.31:0/3462168095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024101900 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0014009990 tx=0x7f0014009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 -- 172.21.15.31:0/3462168095 shutdown_connections 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 --2- 172.21.15.31:0/3462168095 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0024101e40 0x7f0024108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 --2- 172.21.15.31:0/3462168095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00240fe4e0 0x7f00240fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 --2- 172.21.15.31:0/3462168095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024101900 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 -- 172.21.15.31:0/3462168095 >> 172.21.15.31:0/3462168095 conn(0x7f00240f9e80 msgr2=0x7f00240fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:42.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 -- 172.21.15.31:0/3462168095 shutdown_connections 2023-12-14T18:27:42.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.900+0000 7f002b544700 1 -- 172.21.15.31:0/3462168095 wait complete. 2023-12-14T18:27:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 Processor -- start 2023-12-14T18:27:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 -- start start 2023-12-14T18:27:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00240fe4e0 0x7f00241981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0024101e40 0x7f002419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002419fd50 con 0x7f00241014b0 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002419fe90 con 0x7f00240fe4e0 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f002b544700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00241a0020 con 0x7f0024101e40 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53264/0 (socket says 172.21.15.31:53264) 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 -- 172.21.15.31:0/868933659 learned_addr learned my addr 172.21.15.31:0/868933659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f00292e0700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00240fe4e0 0x7f00241981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0029ae1700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0024101e40 0x7f002419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 -- 172.21.15.31:0/868933659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0024101e40 msgr2=0x7f002419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0024101e40 0x7f002419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 -- 172.21.15.31:0/868933659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00240fe4e0 msgr2=0x7f00241981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00240fe4e0 0x7f00241981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.901+0000 7f0028adf700 1 -- 172.21.15.31:0/868933659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00140095f0 con 0x7f00241014b0 2023-12-14T18:27:42.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.902+0000 7f0028adf700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024198710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0014004730 tx=0x7f0014004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.902+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001401d070 con 0x7f00241014b0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.902+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0014004c90 con 0x7f00241014b0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.902+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002410bbd0 con 0x7f00241014b0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.902+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00140054b0 con 0x7f00241014b0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.902+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002410ba40 con 0x7f00241014b0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.903+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00241a0620 con 0x7f00241014b0 2023-12-14T18:27:42.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.904+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f001400fbc0 con 0x7f00241014b0 2023-12-14T18:27:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.905+0000 7f001a7fc700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f00100a1260 0x7f00100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.905+0000 7f00292e0700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f00100a1260 0x7f00100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.905+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f00140ce470 con 0x7f00241014b0 2023-12-14T18:27:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.906+0000 7f00292e0700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f00100a1260 0x7f00100a3710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0024067bd0 tx=0x7f002000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:42.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:42.908+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0014097780 con 0x7f00241014b0 2023-12-14T18:27:42.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:42 smithi176 ceph-mon[43506]: pgmap v284: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:27:42.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:42 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2826842120' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:42 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2650643005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:27:42.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:42 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3566306821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:27:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.164+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00241a0bd0 con 0x7f00241014b0 2023-12-14T18:27:43.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.165+0000 7f001a7fc700 1 -- 172.21.15.31:0/868933659 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f001409ada0 con 0x7f00241014b0 2023-12-14T18:27:43.167 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:43.167 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:27:43.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f00100a1260 msgr2=0x7f00100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f00100a1260 0x7f00100a3710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0024067bd0 tx=0x7f002000b410 comp rx=0 tx=0).stop 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 msgr2=0x7f0024198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024198710 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0014004730 tx=0x7f0014004760 comp rx=0 tx=0).stop 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 shutdown_connections 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f00100a1260 0x7f00100a3710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0024101e40 0x7f002419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00240fe4e0 0x7f00241981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 --2- 172.21.15.31:0/868933659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00241014b0 0x7f0024198710 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 >> 172.21.15.31:0/868933659 conn(0x7f00240f9e80 msgr2=0x7f00240fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 shutdown_connections 2023-12-14T18:27:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.167+0000 7f002b544700 1 -- 172.21.15.31:0/868933659 wait complete. 2023-12-14T18:27:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:27:43.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:27:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 -- 172.21.15.31:0/3112168537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 msgr2=0x7f17fc0ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 --2- 172.21.15.31:0/3112168537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc0ff890 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009990 tx=0x7f17ec009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 -- 172.21.15.31:0/3112168537 shutdown_connections 2023-12-14T18:27:43.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 --2- 172.21.15.31:0/3112168537 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 0x7f17fc100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 --2- 172.21.15.31:0/3112168537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17fc0fead0 0x7f17fc0feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 --2- 172.21.15.31:0/3112168537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc0ff890 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 -- 172.21.15.31:0/3112168537 >> 172.21.15.31:0/3112168537 conn(0x7f17fc0fa740 msgr2=0x7f17fc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 -- 172.21.15.31:0/3112168537 shutdown_connections 2023-12-14T18:27:43.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.319+0000 7f180423a700 1 -- 172.21.15.31:0/3112168537 wait complete. 2023-12-14T18:27:43.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 Processor -- start 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 -- start start 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17fc0fead0 0x7f17fc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 0x7f17fc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc19fd70 con 0x7f17fc0ff420 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc19feb0 con 0x7f17fc0ffdd0 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.320+0000 7f180423a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc1a0040 con 0x7f17fc0fead0 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53272/0 (socket says 172.21.15.31:53272) 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 -- 172.21.15.31:0/2236833659 learned_addr learned my addr 172.21.15.31:0/2236833659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f1801fd6700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17fc0fead0 0x7f17fc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18027d7700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 0x7f17fc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 -- 172.21.15.31:0/2236833659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17fc0fead0 msgr2=0x7f17fc1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17fc0fead0 0x7f17fc1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 -- 172.21.15.31:0/2236833659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 msgr2=0x7f17fc19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 0x7f17fc19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 -- 172.21.15.31:0/2236833659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0095f0 con 0x7f17fc0ff420 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18027d7700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 0x7f17fc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f18017d5700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc198730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f17ec004740 tx=0x7f17ec009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec01d070 con 0x7f17fc0ff420 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17ec004cf0 con 0x7f17fc0ff420 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc10de00 con 0x7f17fc0ff420 2023-12-14T18:27:43.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0055a0 con 0x7f17fc0ff420 2023-12-14T18:27:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.321+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc10dc70 con 0x7f17fc0ff420 2023-12-14T18:27:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.322+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc1098b0 con 0x7f17fc0ff420 2023-12-14T18:27:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.324+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f17ec00fbc0 con 0x7f17fc0ff420 2023-12-14T18:27:43.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.324+0000 7f17f2ffd700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17e80a11e0 0x7f17e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.324+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f17ec0ce560 con 0x7f17fc0ff420 2023-12-14T18:27:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.324+0000 7f1801fd6700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17e80a11e0 0x7f17e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.325+0000 7f1801fd6700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17e80a11e0 0x7f17e80a3690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009bf0 tx=0x7f17f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.328+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f17ec0978d0 con 0x7f17fc0ff420 2023-12-14T18:27:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.540+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17fc02cfe0 con 0x7f17fc0ff420 2023-12-14T18:27:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.541+0000 7f17f2ffd700 1 -- 172.21.15.31:0/2236833659 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24511 (secure 0 0 0) 0x7f17ec09aef0 con 0x7f17fc0ff420 2023-12-14T18:27:43.544 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:27:43.544 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":87,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:26:48.546060+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:27:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17e80a11e0 msgr2=0x7f17e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17e80a11e0 0x7f17e80a3690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009bf0 tx=0x7f17f8009260 comp rx=0 tx=0).stop 2023-12-14T18:27:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 msgr2=0x7f17fc198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc198730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f17ec004740 tx=0x7f17ec009580 comp rx=0 tx=0).stop 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 shutdown_connections 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17fc0ffdd0 0x7f17fc19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f17e80a11e0 0x7f17e80a3690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17fc0fead0 0x7f17fc1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 --2- 172.21.15.31:0/2236833659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17fc0ff420 0x7f17fc198730 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 >> 172.21.15.31:0/2236833659 conn(0x7f17fc0fa740 msgr2=0x7f17fc0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 shutdown_connections 2023-12-14T18:27:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.544+0000 7f180423a700 1 -- 172.21.15.31:0/2236833659 wait complete. 2023-12-14T18:27:43.560 DEBUG:tasks.fwd_scrub:Yielding 2023-12-14T18:27:43.561 INFO:teuthology.run_tasks:Running task exec... 2023-12-14T18:27:43.570 INFO:teuthology.task.exec:Executing custom commands... 2023-12-14T18:27:43.570 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:27:43.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-12-14T18:27:43.570 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-12-14T18:27:43.571 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:27:43.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 -- 172.21.15.31:0/2930311983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec100330 msgr2=0x7f38ec100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 --2- 172.21.15.31:0/2930311983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec100330 0x7f38ec100780 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009990 tx=0x7f38e0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 -- 172.21.15.31:0/2930311983 shutdown_connections 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 --2- 172.21.15.31:0/2930311983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 0x7f38ec107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 --2- 172.21.15.31:0/2930311983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec0ff140 0x7f38ec0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 --2- 172.21.15.31:0/2930311983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec100330 0x7f38ec100780 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 -- 172.21.15.31:0/2930311983 >> 172.21.15.31:0/2930311983 conn(0x7f38ec0fa720 msgr2=0x7f38ec0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 -- 172.21.15.31:0/2930311983 shutdown_connections 2023-12-14T18:27:43.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.718+0000 7f38f0a08700 1 -- 172.21.15.31:0/2930311983 wait complete. 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.719+0000 7f38f0a08700 1 Processor -- start 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.719+0000 7f38f0a08700 1 -- start start 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38f0a08700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 0x7f38ec068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38f0a08700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec100330 0x7f38ec19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38f0a08700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 0x7f38ec19d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38f0a08700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec19dae0 con 0x7f38ec0ff140 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38f0a08700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec19dc50 con 0x7f38ec100cc0 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38f0a08700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec19de10 con 0x7f38ec100330 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 0x7f38ec068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ead9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 0x7f38ec19d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 0x7f38ec068bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53296/0 (socket says 172.21.15.31:53296) 2023-12-14T18:27:43.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 -- 172.21.15.31:0/1530300932 learned_addr learned my addr 172.21.15.31:0/1530300932 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38e9d9b700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec100330 0x7f38ec19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 -- 172.21.15.31:0/1530300932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec100330 msgr2=0x7f38ec19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec100330 0x7f38ec19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 -- 172.21.15.31:0/1530300932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 msgr2=0x7f38ec19d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 0x7f38ec19d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ea59c700 1 -- 172.21.15.31:0/1530300932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00095f0 con 0x7f38ec0ff140 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.720+0000 7f38ead9d700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 0x7f38ec19d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7f38ea59c700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 0x7f38ec068bd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e970 tx=0x7f38dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc009790 con 0x7f38ec0ff140 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38dc00c7e0 con 0x7f38ec0ff140 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc003c70 con 0x7f38ec0ff140 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec10aaa0 con 0x7f38ec0ff140 2023-12-14T18:27:43.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec10a910 con 0x7f38ec0ff140 2023-12-14T18:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7ff656915700 1 -- 172.21.15.31:0/2010840204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101440 msgr2=0x7ff650101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.721+0000 7ff656915700 1 --2- 172.21.15.31:0/2010840204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101440 0x7ff650101890 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff638009990 tx=0x7ff638009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 -- 172.21.15.31:0/2010840204 shutdown_connections 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 --2- 172.21.15.31:0/2010840204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff650101dd0 0x7ff650108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 --2- 172.21.15.31:0/2010840204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff650100250 0x7ff650100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 --2- 172.21.15.31:0/2010840204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101440 0x7ff650101890 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 -- 172.21.15.31:0/2010840204 >> 172.21.15.31:0/2010840204 conn(0x7ff6500fb890 msgr2=0x7ff6500fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 -- 172.21.15.31:0/2010840204 shutdown_connections 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7ff656915700 1 -- 172.21.15.31:0/2010840204 wait complete. 2023-12-14T18:27:43.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.722+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec1066d0 con 0x7f38ec0ff140 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 Processor -- start 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 -- start start 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff650100250 0x7ff650195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff650101440 0x7ff6501964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 0x7ff65019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65019db10 con 0x7ff650101dd0 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65019dc50 con 0x7ff650100250 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff656915700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65019dde0 con 0x7ff650101440 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 0x7ff65019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 0x7ff65019d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53306/0 (socket says 172.21.15.31:53306) 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 -- 172.21.15.31:0/1439014025 learned_addr learned my addr 172.21.15.31:0/1439014025 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff647fff700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff650101440 0x7ff6501964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff650100250 0x7ff650195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 -- 172.21.15.31:0/1439014025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff650101440 msgr2=0x7ff6501964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff650101440 0x7ff6501964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 -- 172.21.15.31:0/1439014025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff650100250 msgr2=0x7ff650195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff650100250 0x7ff650195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.723+0000 7ff654eb2700 1 -- 172.21.15.31:0/1439014025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380095f0 con 0x7ff650101dd0 2023-12-14T18:27:43.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.724+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 0x7ff65019d540 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff64800ef30 tx=0x7ff64800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.724+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f38dc00cc50 con 0x7f38ec0ff140 2023-12-14T18:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.724+0000 7f38db7fe700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38d40a11e0 0x7f38d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.725+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f38dc014070 con 0x7f38ec0ff140 2023-12-14T18:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.725+0000 7f38e9d9b700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38d40a11e0 0x7f38d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.725+0000 7f38e9d9b700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38d40a11e0 0x7f38d40a3690 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b2e0 tx=0x7f38e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.728+0000 7ff64dffb700 1 -- 172.21.15.31:0/1439014025 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648018070 con 0x7ff650101dd0 2023-12-14T18:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.728+0000 7ff64dffb700 1 -- 172.21.15.31:0/1439014025 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff64800f040 con 0x7ff650101dd0 2023-12-14T18:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.728+0000 7ff64dffb700 1 -- 172.21.15.31:0/1439014025 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64800cc80 con 0x7ff650101dd0 2023-12-14T18:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.728+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65010bc50 con 0x7ff650101dd0 2023-12-14T18:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.728+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f38dc0963a0 con 0x7f38ec0ff140 2023-12-14T18:27:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.728+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65010bac0 con 0x7ff650101dd0 2023-12-14T18:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.731+0000 7ff64dffb700 1 -- 172.21.15.31:0/1439014025 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff648003a00 con 0x7ff650101dd0 2023-12-14T18:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.731+0000 7ff64dffb700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff63c0a12a0 0x7ff63c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.731+0000 7ff64dffb700 1 -- 172.21.15.31:0/1439014025 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6480cc820 con 0x7ff650101dd0 2023-12-14T18:27:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.732+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff63c0a12a0 0x7ff63c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.732+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff63c0a12a0 0x7ff63c0a3750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff6501012a0 tx=0x7ff640009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.733+0000 7ff662aa3b80 1 Processor -- start 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.733+0000 7ff662aa3b80 1 -- start start 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 0x55ccec668990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668ed0 0x55ccec6697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec66a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec551660 con 0x55ccec668580 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec5c08f0 con 0x55ccec669d80 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec5b9d80 con 0x55ccec668ed0 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 0x55ccec668990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 0x55ccec668990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53310/0 (socket says 172.21.15.31:53310) 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 -- 172.21.15.31:0/2988919979 learned_addr learned my addr 172.21.15.31:0/2988919979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff654eb2700 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec66a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff647fff700 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668ed0 0x55ccec6697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 -- 172.21.15.31:0/2988919979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668ed0 msgr2=0x55ccec6697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668ed0 0x55ccec6697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 -- 172.21.15.31:0/2988919979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 msgr2=0x55ccec66a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec66a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 -- 172.21.15.31:0/2988919979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccec43b430 con 0x55ccec668580 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff654eb2700 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec66a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.734+0000 7ff64ffff700 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 0x55ccec668990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff640011d50 tx=0x7ff640014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1905162055cb6236 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff6467fc700 1 -- 172.21.15.31:0/2988919979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64000edd0 con 0x55ccec668580 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff6467fc700 1 -- 172.21.15.31:0/2988919979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff640021070 con 0x55ccec668580 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff6467fc700 1 -- 172.21.15.31:0/2988919979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64001d6e0 con 0x55ccec668580 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2988919979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 msgr2=0x55ccec668990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 0x55ccec668990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff640011d50 tx=0x7ff640014000 comp rx=0 tx=0).stop 2023-12-14T18:27:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2988919979 shutdown_connections 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec66a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668580 0x55ccec668990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.735+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2988919979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668ed0 0x55ccec6697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.736+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2988919979 >> 172.21.15.31:0/2988919979 conn(0x55ccec65dd20 msgr2=0x55ccec6634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.736+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2988919979 shutdown_connections 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.736+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2988919979 wait complete. 2023-12-14T18:27:43.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.736+0000 7ff662aa3b80 1 -- start start 2023-12-14T18:27:43.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.736+0000 7ff662aa3b80 1 Processor -- start 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec668580 0x55ccec672170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec669d80 0x55ccec6796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec556010 con 0x55ccec668ed0 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec555cb0 con 0x55ccec668580 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec43d720 con 0x55ccec669d80 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff662aa3b80 10 client.0 _ll_register_callbacks cb 0x55ccec579990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53318/0 (socket says 172.21.15.31:53318) 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff647fff700 1 -- 172.21.15.31:0/1849553400 learned_addr learned my addr 172.21.15.31:0/1849553400 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec669d80 0x55ccec6796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec668580 0x55ccec672170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 -- 172.21.15.31:0/1849553400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec668580 msgr2=0x55ccec672170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec668580 0x55ccec672170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 -- 172.21.15.31:0/1849553400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 msgr2=0x55ccec6726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 -- 172.21.15.31:0/1849553400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff640017070 con 0x55ccec669d80 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff647fff700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec668580 0x55ccec672170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff654eb2700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec669d80 0x55ccec6796f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff6480c9ec0 tx=0x7ff64800cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b30c108e6df8798a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.737+0000 7ff62f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff62f7fe700 1 -- 172.21.15.31:0/1849553400 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64809d880 con 0x55ccec669d80 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccec65b330 con 0x55ccec669d80 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff62f7fe700 1 -- 172.21.15.31:0/1849553400 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff6480a1c10 con 0x55ccec669d80 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff62f7fe700 1 -- 172.21.15.31:0/1849553400 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6480b9070 con 0x55ccec669d80 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 10 client.25405 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:43.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ccec5801f0 con 0x55ccec669d80 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff62f7fe700 1 -- 172.21.15.31:0/1849553400 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ccec65b330 con 0x55ccec669d80 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff62f7fe700 1 -- 172.21.15.31:0/1849553400 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff6480a1780 con 0x55ccec669d80 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 10 client.25405 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 20 client.25405 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec67ddc0 0x55ccec6801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 4 client.25405 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.738+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccec5b2d60 con 0x55ccec67ddc0 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.739+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec67ddc0 0x55ccec6801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.739+0000 7ff62f7fe700 10 client.25405 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.739+0000 7ff64ffff700 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec67ddc0 0x55ccec6801d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8711d79bac7f3b8a server_cookie=1acc94166eeaaf25 in_seq=0 out_seq=0 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff62f7fe700 1 -- 172.21.15.31:0/1849553400 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ccec5b2d60 con 0x55ccec67ddc0 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff62f7fe700 10 client.25405 handle_command_reply: tid=0 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 client.25405 shutdown 2023-12-14T18:27:43.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec669d80 msgr2=0x55ccec6796f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec669d80 0x55ccec6796f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff6480c9ec0 tx=0x7ff64800cee0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 shutdown_connections 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec67ddc0 0x55ccec6801d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec669d80 0x55ccec6796f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec668580 0x55ccec672170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/1849553400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 >> 172.21.15.31:0/1849553400 conn(0x55ccec65dd20 msgr2=0x55ccec661fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff62f7fe700 0 client.25405 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 shutdown_connections 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 1 -- 172.21.15.31:0/1849553400 wait complete. 2023-12-14T18:27:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.740+0000 7ff662aa3b80 20 client.25405 trim_cache size 0 max 16384 2023-12-14T18:27:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 Processor -- start 2023-12-14T18:27:43.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 -- start start 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec663300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6796b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec67fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec65d8a0 con 0x55ccec668ed0 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec669a00 con 0x55ccec669d80 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec668180 con 0x55ccec668580 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6796b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6796b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53334/0 (socket says 172.21.15.31:53334) 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.758+0000 7ff647fff700 1 -- 172.21.15.31:0/2604271738 learned_addr learned my addr 172.21.15.31:0/2604271738 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff64ffff700 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec663300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff654eb2700 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec67fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff647fff700 1 -- 172.21.15.31:0/2604271738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 msgr2=0x55ccec663300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff647fff700 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec663300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff647fff700 1 -- 172.21.15.31:0/2604271738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 msgr2=0x55ccec67fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff647fff700 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec67fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff647fff700 1 -- 172.21.15.31:0/2604271738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccec57ff40 con 0x55ccec668ed0 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff654eb2700 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec67fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff647fff700 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6796b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098e0 tx=0x7ff638005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304a93e91f3fb12f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff646ffd700 1 -- 172.21.15.31:0/2604271738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63801d070 con 0x55ccec668ed0 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff646ffd700 1 -- 172.21.15.31:0/2604271738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff63801e040 con 0x55ccec668ed0 2023-12-14T18:27:43.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.759+0000 7ff646ffd700 1 -- 172.21.15.31:0/2604271738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638004ba0 con 0x55ccec668ed0 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2604271738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 msgr2=0x55ccec6796b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6796b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098e0 tx=0x7ff638005420 comp rx=0 tx=0).stop 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2604271738 shutdown_connections 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec67fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec663300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/2604271738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec6796b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2604271738 >> 172.21.15.31:0/2604271738 conn(0x55ccec65dd20 msgr2=0x55ccec662a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2604271738 shutdown_connections 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.760+0000 7ff662aa3b80 1 -- 172.21.15.31:0/2604271738 wait complete. 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 -- start start 2023-12-14T18:27:43.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 Processor -- start 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec6793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec675f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec676480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec676a00 con 0x55ccec668ed0 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec676b40 con 0x55ccec669d80 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccec6d1c70 con 0x55ccec668580 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff662aa3b80 10 client.0 _ll_register_callbacks cb 0x7ff6480993d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec675f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff647fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec675f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53348/0 (socket says 172.21.15.31:53348) 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.761+0000 7ff647fff700 1 -- 172.21.15.31:0/207091836 learned_addr learned my addr 172.21.15.31:0/207091836 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff64ffff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec6793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff654eb2700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec676480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff647fff700 1 -- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 msgr2=0x55ccec6793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff647fff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec6793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff647fff700 1 -- 172.21.15.31:0/207091836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 msgr2=0x55ccec676480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff647fff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec676480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff647fff700 1 -- 172.21.15.31:0/207091836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccec5801f0 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff64ffff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec6793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff647fff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec675f40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff6380047b0 tx=0x7ff638005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d647b92e24817d95 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff62f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff62f7fe700 1 -- 172.21.15.31:0/207091836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63801d070 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff62f7fe700 1 -- 172.21.15.31:0/207091836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff63800b9f0 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff640021070 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff62f7fe700 1 -- 172.21.15.31:0/207091836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff63800bbb0 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff662aa3b80 10 client.15549 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.762+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7ff650108fc0 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff62f7fe700 1 -- 172.21.15.31:0/207091836 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff640021070 con 0x55ccec668ed0 2023-12-14T18:27:43.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff62f7fe700 1 -- 172.21.15.31:0/207091836 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff638025430 con 0x55ccec668ed0 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff662aa3b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff662aa3b80 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff662aa3b80 20 client.15549 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec6d3180 0x55ccec6d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff662aa3b80 4 client.15549 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ccec5b2a80 con 0x55ccec6d3180 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.763+0000 7ff64ffff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec6d3180 0x55ccec6d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.764+0000 7ff64ffff700 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec6d3180 0x55ccec6d5560 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95bac08dd8b1083 server_cookie=7b99df38f7b67881 in_seq=0 out_seq=0 2023-12-14T18:27:43.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.764+0000 7ff62f7fe700 10 client.15549 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.765+0000 7ff62f7fe700 1 -- 172.21.15.31:0/207091836 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ccec5b2a80 con 0x55ccec6d3180 2023-12-14T18:27:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.765+0000 7ff62f7fe700 10 client.15549 handle_command_reply: tid=0 2023-12-14T18:27:43.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.765+0000 7ff662aa3b80 1 client.15549 shutdown 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 msgr2=0x55ccec675f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec675f40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff6380047b0 tx=0x7ff638005e80 comp rx=0 tx=0).stop 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 shutdown_connections 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ccec6d3180 0x55ccec6d5560 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ccec669d80 0x55ccec676480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ccec668580 0x55ccec6793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 --2- 172.21.15.31:0/207091836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ccec668ed0 0x55ccec675f40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 >> 172.21.15.31:0/207091836 conn(0x55ccec65dd20 msgr2=0x55ccec67fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff62f7fe700 0 client.15549 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 shutdown_connections 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 1 -- 172.21.15.31:0/207091836 wait complete. 2023-12-14T18:27:43.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.766+0000 7ff662aa3b80 20 client.15549 trim_cache size 0 max 16384 2023-12-14T18:27:43.773 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:27:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "0e1834e8-a70d-486a-9c72-4070b15837be", 2023-12-14T18:27:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:27:43.773 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:43.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff63c0a12a0 msgr2=0x7ff63c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff63c0a12a0 0x7ff63c0a3750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff6501012a0 tx=0x7ff640009260 comp rx=0 tx=0).stop 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 msgr2=0x7ff65019d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 0x7ff65019d540 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff64800ef30 tx=0x7ff64800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 shutdown_connections 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff650101dd0 0x7ff65019d540 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff650100250 0x7ff650195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7ff63c0a12a0 0x7ff63c0a3750 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 --2- 172.21.15.31:0/1439014025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff650101440 0x7ff6501964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 >> 172.21.15.31:0/1439014025 conn(0x7ff6500fb890 msgr2=0x7ff650104890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 shutdown_connections 2023-12-14T18:27:43.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.767+0000 7ff656915700 1 -- 172.21.15.31:0/1439014025 wait complete. 2023-12-14T18:27:43.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:27:43.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:43 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1617497897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:43.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:43 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/868933659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:43.814 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:43 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2236833659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:43 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1617497897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:43 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/868933659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:43 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2236833659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 -- 172.21.15.31:0/2051457010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4105f60 msgr2=0x7fc8a4108360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 --2- 172.21.15.31:0/2051457010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4105f60 0x7fc8a4108360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0a0 tx=0x7fc89c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 -- 172.21.15.31:0/2051457010 shutdown_connections 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 --2- 172.21.15.31:0/2051457010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4105f60 0x7fc8a4108360 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 --2- 172.21.15.31:0/2051457010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8a4100d20 0x7fc8a4103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 --2- 172.21.15.31:0/2051457010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8a4103640 0x7fc8a4105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 -- 172.21.15.31:0/2051457010 >> 172.21.15.31:0/2051457010 conn(0x7fc8a40fa740 msgr2=0x7fc8a40fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 -- 172.21.15.31:0/2051457010 shutdown_connections 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.925+0000 7fc8aabad700 1 -- 172.21.15.31:0/2051457010 wait complete. 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 Processor -- start 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 -- start start 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8a4100d20 0x7fc8a41960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 0x7fc8a4196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8a4105f60 0x7fc8a419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a419dc70 con 0x7fc8a4103640 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a419ddb0 con 0x7fc8a4105f60 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8aabad700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a419df40 con 0x7fc8a4100d20 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 0x7fc8a4196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 0x7fc8a4196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53366/0 (socket says 172.21.15.31:53366) 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 -- 172.21.15.31:0/1500341515 learned_addr learned my addr 172.21.15.31:0/1500341515 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a914a700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8a4105f60 0x7fc8a419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a8949700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8a4100d20 0x7fc8a41960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 -- 172.21.15.31:0/1500341515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8a4100d20 msgr2=0x7fc8a41960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8a4100d20 0x7fc8a41960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 -- 172.21.15.31:0/1500341515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8a4105f60 msgr2=0x7fc8a419d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8a4105f60 0x7fc8a419d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.926+0000 7fc8a3fff700 1 -- 172.21.15.31:0/1500341515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c009d00 con 0x7fc8a4103640 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.927+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 0x7fc8a4196630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9a0 tx=0x7fc89000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.927+0000 7fc8a1ffb700 1 -- 172.21.15.31:0/1500341515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890009790 con 0x7fc8a4103640 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.927+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a410bbf0 con 0x7fc8a4103640 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.927+0000 7fc8a1ffb700 1 -- 172.21.15.31:0/1500341515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc89000cbd0 con 0x7fc8a4103640 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.927+0000 7fc8a1ffb700 1 -- 172.21.15.31:0/1500341515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890018680 con 0x7fc8a4103640 2023-12-14T18:27:43.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.927+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a410ba60 con 0x7fc8a4103640 2023-12-14T18:27:43.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.929+0000 7fc8a1ffb700 1 -- 172.21.15.31:0/1500341515 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc89000cd40 con 0x7fc8a4103640 2023-12-14T18:27:43.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.930+0000 7fc8a1ffb700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc8940a1260 0x7fc8940a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.930+0000 7fc8a1ffb700 1 -- 172.21.15.31:0/1500341515 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc890014070 con 0x7fc8a4103640 2023-12-14T18:27:43.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.930+0000 7fc8a8949700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc8940a1260 0x7fc8940a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8a8949700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc8940a1260 0x7fc8940a3710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc898005950 tx=0x7fc8980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 Processor -- start 2023-12-14T18:27:43.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 -- start start 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 0x5643f0693910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f0694770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0694d00 0x5643f0695170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f057c5e0 con 0x5643f0693500 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f05eb870 con 0x5643f0693e50 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f05e4d00 con 0x5643f0694d00 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8a8949700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 0x5643f0693910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.931+0000 7fc8a8949700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 0x5643f0693910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53382/0 (socket says 172.21.15.31:53382) 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 -- 172.21.15.31:0/706343022 learned_addr learned my addr 172.21.15.31:0/706343022 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f0694770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a914a700 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0694d00 0x5643f0695170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 -- 172.21.15.31:0/706343022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0694d00 msgr2=0x5643f0695170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0694d00 0x5643f0695170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 -- 172.21.15.31:0/706343022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 msgr2=0x5643f0694770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f0694770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 -- 172.21.15.31:0/706343022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643f04663b0 con 0x5643f0693500 2023-12-14T18:27:43.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f0694770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc8a8949700 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 0x5643f0693910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc89800fd70 tx=0x7fc898012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172c2a124c4e8ad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc88effd700 1 -- 172.21.15.31:0/706343022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898011bb0 con 0x5643f0693500 2023-12-14T18:27:43.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc88effd700 1 -- 172.21.15.31:0/706343022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc898011d10 con 0x5643f0693500 2023-12-14T18:27:43.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.932+0000 7fc88effd700 1 -- 172.21.15.31:0/706343022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89801d720 con 0x5643f0693500 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/706343022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 msgr2=0x5643f0693910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 0x5643f0693910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc89800fd70 tx=0x7fc898012000 comp rx=0 tx=0).stop 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/706343022 shutdown_connections 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0694d00 0x5643f0695170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693500 0x5643f0693910 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/706343022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f0694770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/706343022 >> 172.21.15.31:0/706343022 conn(0x5643f0688ca0 msgr2=0x5643f068e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/706343022 shutdown_connections 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/706343022 wait complete. 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.933+0000 7fc8b6d3bb80 1 -- start start 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 Processor -- start 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f069d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f0580f90 con 0x5643f0694d00 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f0580c30 con 0x5643f0693e50 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f04686a0 con 0x5643f0693500 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8b6d3bb80 10 client.0 _ll_register_callbacks cb 0x5643f05a4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f069d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8a914a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8a914a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53394/0 (socket says 172.21.15.31:53394) 2023-12-14T18:27:43.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.934+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f069d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:48146/0 (socket says 172.21.15.31:48146) 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a914a700 1 -- 172.21.15.31:0/3021219379 learned_addr learned my addr 172.21.15.31:0/3021219379 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a8949700 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a3fff700 1 -- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 msgr2=0x5643f069d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a3fff700 1 -- 172.21.15.31:0/3021219379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 msgr2=0x5643f06a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a3fff700 1 -- 172.21.15.31:0/3021219379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89800fa20 con 0x5643f0693e50 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f069d660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8900ca5d0 tx=0x7fc8900caec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d4acae0bd8b982 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc883fff700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc883fff700 1 -- 172.21.15.31:0/3021219379 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900b1070 con 0x5643f0693e50 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc883fff700 1 -- 172.21.15.31:0/3021219379 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc8900a2320 con 0x5643f0693e50 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.935+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643f06862b0 con 0x5643f0693e50 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc883fff700 1 -- 172.21.15.31:0/3021219379 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8900b8170 con 0x5643f0693e50 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 10 client.25424 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5643f05ab170 con 0x5643f0693e50 2023-12-14T18:27:43.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc883fff700 1 -- 172.21.15.31:0/3021219379 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8900c8080 con 0x5643f0693e50 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc883fff700 1 -- 172.21.15.31:0/3021219379 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8900a1e90 con 0x5643f0693e50 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 10 client.25424 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 10 client.25424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 20 client.25424 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f06b18d0 0x5643f06b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 4 client.25424 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643f05ddce0 con 0x5643f06b18d0 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.936+0000 7fc8a8949700 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f06b18d0 0x5643f06b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.937+0000 7fc883fff700 10 client.25424 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.937+0000 7fc8a8949700 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f06b18d0 0x5643f06b3cb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57fc39398066e9f server_cookie=a3520e55b31099ee in_seq=0 out_seq=0 2023-12-14T18:27:43.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc883fff700 1 -- 172.21.15.31:0/3021219379 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5643f05ddce0 con 0x5643f06b18d0 2023-12-14T18:27:43.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc883fff700 10 client.25424 handle_command_reply: tid=0 2023-12-14T18:27:43.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 client.25424 shutdown 2023-12-14T18:27:43.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 msgr2=0x5643f069d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f069d660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8900ca5d0 tx=0x7fc8900caec0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 shutdown_connections 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f06b18d0 0x5643f06b3cb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc883fff700 0 client.25424 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/3021219379 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f069d660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 >> 172.21.15.31:0/3021219379 conn(0x5643f0688ca0 msgr2=0x5643f068cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 shutdown_connections 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/3021219379 wait complete. 2023-12-14T18:27:43.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.938+0000 7fc8b6d3bb80 20 client.25424 trim_cache size 0 max 16384 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 Processor -- start 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 -- start start 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 0x5643f06a4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0694d00 0x5643f069b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f0688820 con 0x5643f0693e50 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f069ae60 con 0x5643f0694d00 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.956+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f0698750 con 0x5643f0693500 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 0x5643f06a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 0x5643f06a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53398/0 (socket says 172.21.15.31:53398) 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 -- 172.21.15.31:0/374587511 learned_addr learned my addr 172.21.15.31:0/374587511 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a8949700 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a914a700 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0694d00 0x5643f069b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 -- 172.21.15.31:0/374587511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 msgr2=0x5643f069bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 -- 172.21.15.31:0/374587511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0694d00 msgr2=0x5643f069b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0694d00 0x5643f069b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 -- 172.21.15.31:0/374587511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643f05aaec0 con 0x5643f0693e50 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a8949700 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 0x5643f06a4630 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc8900968f0 tx=0x7fc8900092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0542db9ebdb11b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.957+0000 7fc88f7fe700 1 -- 172.21.15.31:0/374587511 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900b1070 con 0x5643f0693e50 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc88f7fe700 1 -- 172.21.15.31:0/374587511 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643f06862b0 con 0x5643f0693e50 2023-12-14T18:27:43.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc88f7fe700 1 -- 172.21.15.31:0/374587511 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900ca310 con 0x5643f0693e50 2023-12-14T18:27:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/374587511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 msgr2=0x5643f06a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 0x5643f06a4630 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc8900968f0 tx=0x7fc8900092d0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/374587511 shutdown_connections 2023-12-14T18:27:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0694d00 0x5643f069b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f069bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/374587511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0693e50 0x5643f06a4630 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/374587511 >> 172.21.15.31:0/374587511 conn(0x5643f0688ca0 msgr2=0x5643f06b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/374587511 shutdown_connections 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.958+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/374587511 wait complete. 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.959+0000 7fc8b6d3bb80 1 -- start start 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.959+0000 7fc8b6d3bb80 1 Processor -- start 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f06a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f06a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f06a42b0 con 0x5643f0694d00 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f06a4420 con 0x5643f0693e50 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f06a45e0 con 0x5643f0693500 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8b6d3bb80 10 client.0 _ll_register_callbacks cb 0x7fc890099a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53410/0 (socket says 172.21.15.31:53410) 2023-12-14T18:27:43.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 -- 172.21.15.31:0/2445860726 learned_addr learned my addr 172.21.15.31:0/2445860726 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a3fff700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f06a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a8949700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f06a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 -- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 msgr2=0x5643f06a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f06a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 -- 172.21.15.31:0/2445860726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 msgr2=0x5643f06a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f06a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 -- 172.21.15.31:0/2445860726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643f04663b0 con 0x5643f0694d00 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc8a914a700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a3d70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0a0 tx=0x7fc89c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95a032563a488c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.960+0000 7fc883fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc883fff700 1 -- 172.21.15.31:0/2445860726 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00e040 con 0x5643f0694d00 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc883fff700 1 -- 172.21.15.31:0/2445860726 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc89c003e10 con 0x5643f0694d00 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc898011d10 con 0x5643f0694d00 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc883fff700 1 -- 172.21.15.31:0/2445860726 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc89c00be50 con 0x5643f0694d00 2023-12-14T18:27:43.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 10 client.15567 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5643f05ab170 con 0x5643f0694d00 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc883fff700 1 -- 172.21.15.31:0/2445860726 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc898011d10 con 0x5643f0694d00 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc883fff700 1 -- 172.21.15.31:0/2445860726 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc89c00b0d0 con 0x5643f0694d00 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 10 client.15567 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 20 client.15567 populate_metadata read hostname 'smithi031' 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f07069d0 0x5643f0708de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 4 client.15567 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.961+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5643f05dda00 con 0x5643f07069d0 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8a8949700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f07069d0 0x5643f0708de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8a8949700 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f07069d0 0x5643f0708de0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b222734b1f08657 server_cookie=1f94b27c3851af89 in_seq=0 out_seq=0 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc883fff700 10 client.15567 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc883fff700 1 -- 172.21.15.31:0/2445860726 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5643f05dda00 con 0x5643f07069d0 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc883fff700 10 client.15567 handle_command_reply: tid=0 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 client.15567 shutdown 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 msgr2=0x5643f06a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a3d70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0a0 tx=0x7fc89c00bce0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 shutdown_connections 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f07069d0 0x5643f0708de0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f0694d00 0x5643f06a3d70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f0693500 0x5643f06a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 --2- 172.21.15.31:0/2445860726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f0693e50 0x5643f06a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc883fff700 0 client.15567 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.962+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 >> 172.21.15.31:0/2445860726 conn(0x5643f0688ca0 msgr2=0x5643f069bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 shutdown_connections 2023-12-14T18:27:43.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8b6d3bb80 1 -- 172.21.15.31:0/2445860726 wait complete. 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8b6d3bb80 20 client.15567 trim_cache size 0 max 16384 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "0e1834e8-a70d-486a-9c72-4070b15837be": { 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "0e1834e8-a70d-486a-9c72-4070b15837be", 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:27:43.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:27:43.971 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:27:43.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc8940a1260 msgr2=0x7fc8940a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc8940a1260 0x7fc8940a3710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc898005950 tx=0x7fc8980058e0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 msgr2=0x7fc8a4196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:43.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 0x7fc8a4196630 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e9a0 tx=0x7fc89000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 shutdown_connections 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8a4105f60 0x7fc8a419d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fc8940a1260 0x7fc8940a3710 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8a4100d20 0x7fc8a41960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 --2- 172.21.15.31:0/1500341515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8a4103640 0x7fc8a4196630 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 >> 172.21.15.31:0/1500341515 conn(0x7fc8a40fa740 msgr2=0x7fc8a4101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 shutdown_connections 2023-12-14T18:27:43.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:43.963+0000 7fc8aabad700 1 -- 172.21.15.31:0/1500341515 wait complete. 2023-12-14T18:27:43.979 INFO:tasks.cephfs.filesystem:scrub status for tag:0e1834e8-a70d-486a-9c72-4070b15837be - {'path': '/', 'tag': '0e1834e8-a70d-486a-9c72-4070b15837be', 'options': 'recursive,force'} 2023-12-14T18:27:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:43 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1617497897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:27:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:43 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/868933659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:27:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:43 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2236833659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:27:44.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.024+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f38ec061980 con 0x7f38ec0ff140 2023-12-14T18:27:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.592+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f38dc00c950 con 0x7f38ec0ff140 2023-12-14T18:27:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.593+0000 7f38db7fe700 1 -- 172.21.15.31:0/1530300932 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f38dc08dfe0 con 0x7f38ec0ff140 2023-12-14T18:27:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.596+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38d40a11e0 msgr2=0x7f38d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.596+0000 7f38f0a08700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38d40a11e0 0x7f38d40a3690 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b2e0 tx=0x7f38e0005960 comp rx=0 tx=0).stop 2023-12-14T18:27:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.596+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 msgr2=0x7f38ec068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.596+0000 7f38f0a08700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 0x7f38ec068bd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00e970 tx=0x7f38dc00ec80 comp rx=0 tx=0).stop 2023-12-14T18:27:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 shutdown_connections 2023-12-14T18:27:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38ec100cc0 0x7f38ec19d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38ec0ff140 0x7f38ec068bd0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f38d40a11e0 0x7f38d40a3690 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 --2- 172.21.15.31:0/1530300932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec100330 0x7f38ec19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 >> 172.21.15.31:0/1530300932 conn(0x7f38ec0fa720 msgr2=0x7f38ec103810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 shutdown_connections 2023-12-14T18:27:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.597+0000 7f38f0a08700 1 -- 172.21.15.31:0/1530300932 wait complete. 2023-12-14T18:27:44.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.757+0000 7fe417372700 1 -- 172.21.15.31:0/4147992624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe410100330 msgr2=0x7fe410100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.757+0000 7fe417372700 1 --2- 172.21.15.31:0/4147992624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe410100330 0x7fe410100780 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe400009990 tx=0x7fe400009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 -- 172.21.15.31:0/4147992624 shutdown_connections 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 --2- 172.21.15.31:0/4147992624 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100cc0 0x7fe410107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 --2- 172.21.15.31:0/4147992624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4100ff140 0x7fe4100ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 --2- 172.21.15.31:0/4147992624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe410100330 0x7fe410100780 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 -- 172.21.15.31:0/4147992624 >> 172.21.15.31:0/4147992624 conn(0x7fe4100fa720 msgr2=0x7fe4100fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:44.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 -- 172.21.15.31:0/4147992624 shutdown_connections 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 -- 172.21.15.31:0/4147992624 wait complete. 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 Processor -- start 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.758+0000 7fe417372700 1 -- start start 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe417372700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 0x7fe4101981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe417372700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100330 0x7fe410198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe417372700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe410100cc0 0x7fe41019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe417372700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41019fd40 con 0x7fe4100ff140 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe417372700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41019fe80 con 0x7fe410100330 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe417372700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4101a0010 con 0x7fe410100cc0 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 0x7fe4101981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 0x7fe4101981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60656/0 (socket says 172.21.15.31:60656) 2023-12-14T18:27:44.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 -- 172.21.15.31:0/3883687189 learned_addr learned my addr 172.21.15.31:0/3883687189 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41490d700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100330 0x7fe410198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41590f700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe410100cc0 0x7fe41019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 -- 172.21.15.31:0/3883687189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe410100cc0 msgr2=0x7fe41019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe410100cc0 0x7fe41019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 -- 172.21.15.31:0/3883687189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100330 msgr2=0x7fe410198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100330 0x7fe410198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 -- 172.21.15.31:0/3883687189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009c40 con 0x7fe4100ff140 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41490d700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100330 0x7fe410198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.759+0000 7fe41510e700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 0x7fe4101981c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00e970 tx=0x7fe40c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.760+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c009790 con 0x7fe4100ff140 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.760+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe40c00c7c0 con 0x7fe4100ff140 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.760+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000095f0 con 0x7fe4100ff140 2023-12-14T18:27:44.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.760+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c003c20 con 0x7fe4100ff140 2023-12-14T18:27:44.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.760+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41010aaa0 con 0x7fe4100ff140 2023-12-14T18:27:44.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.761+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41010ac30 con 0x7fe4100ff140 2023-12-14T18:27:44.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.763+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7fe40c00c930 con 0x7fe4100ff140 2023-12-14T18:27:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.764+0000 7fe4067fc700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 0x7fe3fc0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.764+0000 7fe41490d700 1 -- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 msgr2=0x7fe3fc0a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.764+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe40c014070 con 0x7fe4100ff140 2023-12-14T18:27:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.764+0000 7fe41490d700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 0x7fe3fc0a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:27:44.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.768+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe40c096280 con 0x7fe4100ff140 2023-12-14T18:27:44.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:44 smithi204 ceph-mon[43866]: pgmap v285: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-14T18:27:44.961 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:44 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1530300932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-14T18:27:44.961 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:44 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: ignoring --setuser ceph since I am not root 2023-12-14T18:27:44.961 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:44 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: ignoring --setgroup ceph since I am not root 2023-12-14T18:27:44.962 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:44 smithi204 ceph-mgr[45807]: -- 172.21.15.204:0/3457903380 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5648d46a5520 con 0x5648d467ac00 2023-12-14T18:27:44.962 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:44 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:44.959+0000 7fd9dfe96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:27:44.962 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:44 smithi176 ceph-mon[43506]: pgmap v285: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-14T18:27:44.962 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:44 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1530300932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-14T18:27:44.963 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: ignoring --setuser ceph since I am not root 2023-12-14T18:27:44.963 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: ignoring --setgroup ceph since I am not root 2023-12-14T18:27:44.963 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:44 smithi176 ceph-mgr[44977]: -- 172.21.15.176:0/4267390849 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557de1269520 con 0x557de123f000 2023-12-14T18:27:44.963 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:44.960+0000 7fcf6d167200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:27:44.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:44 smithi031 ceph-mon[42085]: pgmap v285: 129 pgs: 129 active+clean; 644 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2023-12-14T18:27:44.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:44 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1530300932' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-14T18:27:44.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: ignoring --setuser ceph since I am not root 2023-12-14T18:27:44.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: ignoring --setgroup ceph since I am not root 2023-12-14T18:27:44.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:44 smithi031 ceph-mgr[42358]: -- 172.21.15.31:0/3139991154 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a5632cd520 con 0x55a5632a3000 2023-12-14T18:27:44.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.965+0000 7fe41490d700 1 -- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 msgr2=0x7fe3fc0a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:44.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.965+0000 7fe41490d700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 0x7fe3fc0a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:27:44.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.981+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe410067c10 con 0x7fe4100ff140 2023-12-14T18:27:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.987+0000 7fe4067fc700 1 -- 172.21.15.31:0/3883687189 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe40c0998a0 con 0x7fe4100ff140 2023-12-14T18:27:44.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.989+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 msgr2=0x7fe3fc0a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:27:44.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.989+0000 7fe417372700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 0x7fe3fc0a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 msgr2=0x7fe4101981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:44.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 0x7fe4101981c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00e970 tx=0x7fe40c00ec80 comp rx=0 tx=0).stop 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 shutdown_connections 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7fe3fc0a1230 0x7fe3fc0a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe410100cc0 0x7fe41019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4100ff140 0x7fe4101981c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 --2- 172.21.15.31:0/3883687189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe410100330 0x7fe410198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 >> 172.21.15.31:0/3883687189 conn(0x7fe4100fa720 msgr2=0x7fe410068100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 shutdown_connections 2023-12-14T18:27:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:44.990+0000 7fe417372700 1 -- 172.21.15.31:0/3883687189 wait complete. 2023-12-14T18:27:45.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-12-14T18:27:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 -- 172.21.15.31:0/3906364755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c00ff140 msgr2=0x7f81c00ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 --2- 172.21.15.31:0/3906364755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c00ff140 0x7f81c00ff550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009990 tx=0x7f81b0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 -- 172.21.15.31:0/3906364755 shutdown_connections 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 --2- 172.21.15.31:0/3906364755 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c0100cc0 0x7f81c0107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 --2- 172.21.15.31:0/3906364755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c00ff140 0x7f81c00ff550 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 --2- 172.21.15.31:0/3906364755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c0100330 0x7f81c0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 -- 172.21.15.31:0/3906364755 >> 172.21.15.31:0/3906364755 conn(0x7f81c00fa720 msgr2=0x7f81c00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 -- 172.21.15.31:0/3906364755 shutdown_connections 2023-12-14T18:27:45.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.163+0000 7f81c505c700 1 -- 172.21.15.31:0/3906364755 wait complete. 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.164+0000 7f81c505c700 1 Processor -- start 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.164+0000 7f81c505c700 1 -- start start 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81c505c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c00ff140 0x7f81c0196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81c505c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c0100330 0x7f81c01965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81c505c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 0x7f81c019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81c505c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019dbf0 con 0x7f81c0100cc0 2023-12-14T18:27:45.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81c505c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019dd30 con 0x7f81c00ff140 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81c505c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019dec0 con 0x7f81c0100330 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 0x7f81c019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bf59e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 0x7f81c019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60662/0 (socket says 172.21.15.31:60662) 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bf59e700 1 -- 172.21.15.31:0/4249907093 learned_addr learned my addr 172.21.15.31:0/4249907093 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c00ff140 0x7f81c0196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81be59c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c0100330 0x7f81c01965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 -- 172.21.15.31:0/4249907093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c0100330 msgr2=0x7f81c01965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c0100330 0x7f81c01965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 -- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 msgr2=0x7f81c019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 0x7f81c019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 -- 172.21.15.31:0/4249907093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00095f0 con 0x7f81c00ff140 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bf59e700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 0x7f81c019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.165+0000 7f81bed9d700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c00ff140 0x7f81c0196070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b620 tx=0x7f81b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.166+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b001d070 con 0x7f81c00ff140 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.166+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c010aab0 con 0x7f81c00ff140 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.166+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81b0022470 con 0x7f81c00ff140 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.166+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b0017650 con 0x7f81c00ff140 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.166+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c010a920 con 0x7f81c00ff140 2023-12-14T18:27:45.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.167+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c019e490 con 0x7f81c00ff140 2023-12-14T18:27:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.168+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f81b000fb10 con 0x7f81c00ff140 2023-12-14T18:27:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.168+0000 7f81a7fff700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 0x7f81a80a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.169+0000 7f81be59c700 1 -- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 msgr2=0x7f81a80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.169+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81b00ce1f0 con 0x7f81c00ff140 2023-12-14T18:27:45.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.169+0000 7f81be59c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 0x7f81a80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:27:45.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.173+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f81b0097500 con 0x7f81c00ff140 2023-12-14T18:27:45.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:45.061+0000 7fd9dfe96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:27:45.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:45.148+0000 7fd9dfe96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:27:45.236 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:45.061+0000 7fcf6d167200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:27:45.236 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:45.148+0000 7fcf6d167200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:27:45.257 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:45.012+0000 7f8883064200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:27:45.257 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:45.151+0000 7f8883064200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:27:45.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.370+0000 7f81be59c700 1 -- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 msgr2=0x7f81a80a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:45.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.370+0000 7f81be59c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 0x7f81a80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:27:45.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.382+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f81c019eb70 con 0x7f81c00ff140 2023-12-14T18:27:45.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.389+0000 7f81a7fff700 1 -- 172.21.15.31:0/4249907093 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f81b009ab20 con 0x7f81c00ff140 2023-12-14T18:27:45.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 msgr2=0x7f81a80a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 0x7f81a80a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c00ff140 msgr2=0x7f81c0196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c00ff140 0x7f81c0196070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81b000b620 tx=0x7f81b0004730 comp rx=0 tx=0).stop 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 shutdown_connections 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81c0100cc0 0x7f81c019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c00ff140 0x7f81c0196070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f81a80a1230 0x7f81a80a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 --2- 172.21.15.31:0/4249907093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81c0100330 0x7f81c01965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 >> 172.21.15.31:0/4249907093 conn(0x7f81c00fa720 msgr2=0x7f81c00680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 shutdown_connections 2023-12-14T18:27:45.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.392+0000 7f81c505c700 1 -- 172.21.15.31:0/4249907093 wait complete. 2023-12-14T18:27:45.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-12-14T18:27:45.492 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:45.234+0000 7fcf6d167200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 -- 172.21.15.31:0/2764133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee81036a0 msgr2=0x7f7ee8105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 --2- 172.21.15.31:0/2764133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee81036a0 0x7f7ee8105a80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0098f0 tx=0x7f7edc009c00 comp rx=0 tx=0).stop 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 -- 172.21.15.31:0/2764133742 shutdown_connections 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 --2- 172.21.15.31:0/2764133742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee8105fc0 0x7f7ee81083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 --2- 172.21.15.31:0/2764133742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee8100d80 0x7f7ee8103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 --2- 172.21.15.31:0/2764133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee81036a0 0x7f7ee8105a80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 -- 172.21.15.31:0/2764133742 >> 172.21.15.31:0/2764133742 conn(0x7f7ee80fa720 msgr2=0x7f7ee80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:45.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 -- 172.21.15.31:0/2764133742 shutdown_connections 2023-12-14T18:27:45.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 -- 172.21.15.31:0/2764133742 wait complete. 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.548+0000 7f7eedbea700 1 Processor -- start 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 -- start start 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee8100d80 0x7f7ee81006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee81036a0 0x7f7ee80fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee8105fc0 0x7f7ee80ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80ffa10 con 0x7f7ee81036a0 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80ffb80 con 0x7f7ee8100d80 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7eedbea700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee81a0080 con 0x7f7ee8105fc0 2023-12-14T18:27:45.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee81036a0 0x7f7ee80fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee77fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee8100d80 0x7f7ee81006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee81036a0 0x7f7ee80fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60678/0 (socket says 172.21.15.31:60678) 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 learned_addr learned my addr 172.21.15.31:0/3632794068 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee7fff700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee8105fc0 0x7f7ee80ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee77fe700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee8105fc0 msgr2=0x7f7ee80ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee77fe700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee8105fc0 0x7f7ee80ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee77fe700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee81036a0 msgr2=0x7f7ee80fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee77fe700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee81036a0 0x7f7ee80fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee77fe700 1 -- 172.21.15.31:0/3632794068 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0009d00 con 0x7f7ee8100d80 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.549+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee81036a0 0x7f7ee80fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.550+0000 7f7ee77fe700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee8100d80 0x7f7ee81006f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800e9f0 tx=0x7f7ed800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.550+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800c9c0 con 0x7f7ee8100d80 2023-12-14T18:27:45.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.550+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ed8010430 con 0x7f7ee8100d80 2023-12-14T18:27:45.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.550+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc009520 con 0x7f7ee8100d80 2023-12-14T18:27:45.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.550+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8018680 con 0x7f7ee8100d80 2023-12-14T18:27:45.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.550+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee810bc70 con 0x7f7ee8100d80 2023-12-14T18:27:45.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.551+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee8067c10 con 0x7f7ee8100d80 2023-12-14T18:27:45.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.552+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f7ed8010860 con 0x7f7ee8100d80 2023-12-14T18:27:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.553+0000 7f7ee4ff9700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.553+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.553+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ed8014070 con 0x7f7ee8100d80 2023-12-14T18:27:45.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.553+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:27:45.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.556+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7ed80961d0 con 0x7f7ee8100d80 2023-12-14T18:27:45.599 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:45.255+0000 7f8883064200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:27:45.599 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:45.351+0000 7f8883064200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:27:45.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.754+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:45.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.754+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:27:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:45.799+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 --> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ee81028f0 con 0x7f7ed00a1230 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1530300932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: mgrmap e18: x(active, since 6m), standbys: z, y 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3883687189' entity='client.admin' 2023-12-14T18:27:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:45 smithi031 ceph-mon[42085]: from='client.? ' entity='client.admin' 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:45.842+0000 7fcf6d167200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1530300932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-14T18:27:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: mgrmap e18: x(active, since 6m), standbys: z, y 2023-12-14T18:27:45.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3883687189' entity='client.admin' 2023-12-14T18:27:45.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:45 smithi176 ceph-mon[43506]: from='client.? ' entity='client.admin' 2023-12-14T18:27:46.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:27:46.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:27:46.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:27:46.079 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:27:46.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1530300932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-14T18:27:46.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: mgrmap e18: x(active, since 6m), standbys: z, y 2023-12-14T18:27:46.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3883687189' entity='client.admin' 2023-12-14T18:27:46.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-mon[43866]: from='client.? ' entity='client.admin' 2023-12-14T18:27:46.080 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:45.233+0000 7fd9dfe96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:27:46.080 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:45.843+0000 7fd9dfe96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:27:46.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:46.155+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:46.155+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-14T18:27:46.334 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.065+0000 7f8883064200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:27:46.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:46.077+0000 7fd9dfe96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:27:46.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:46.163+0000 7fd9dfe96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:27:46.339 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:46.272+0000 7fd9dfe96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:27:46.365 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:46.079+0000 7fcf6d167200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:27:46.365 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:46.164+0000 7fcf6d167200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:27:46.365 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:46.273+0000 7fcf6d167200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:27:46.645 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:46.363+0000 7fcf6d167200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:27:46.645 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:46.558+0000 7fcf6d167200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:27:46.645 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:46.363+0000 7fd9dfe96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:27:46.645 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:46.557+0000 7fd9dfe96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:27:46.661 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.332+0000 7f8883064200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:27:46.662 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.430+0000 7f8883064200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:27:46.662 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.555+0000 7f8883064200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:27:46.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:46.956+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:46.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:46.956+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-14T18:27:46.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.659+0000 7f8883064200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:27:46.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.887+0000 7f8883064200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:27:46.991 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:46.643+0000 7fcf6d167200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:27:47.019 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:46.643+0000 7fd9dfe96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:27:47.282 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:47.017+0000 7fcf6d167200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:27:47.283 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:47.017+0000 7fd9dfe96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:27:47.418 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:46.985+0000 7f8883064200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:27:47.568 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:47.281+0000 7fcf6d167200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:27:47.570 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:47.281+0000 7fd9dfe96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:27:47.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:47.416+0000 7f8883064200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:27:47.839 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:47.568+0000 7fd9dfe96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:27:47.839 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:47.663+0000 7fd9dfe96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:27:47.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:47.567+0000 7fcf6d167200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:27:47.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:47.661+0000 7fcf6d167200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:27:48.047 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:47.720+0000 7f8883064200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:27:48.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:48 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:48.045+0000 7f8883064200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:27:48.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:48 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:48.152+0000 7f8883064200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:27:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:48.557+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:48.557+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-14T18:27:48.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:48 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:48.414+0000 7fcf6d167200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:27:48.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:48 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:48.509+0000 7fcf6d167200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:27:48.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:48 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:48.658+0000 7fcf6d167200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:27:48.839 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:48 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:48.417+0000 7fd9dfe96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:27:48.839 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:48 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:48.513+0000 7fd9dfe96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:27:48.839 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:48 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:48.661+0000 7fd9dfe96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:27:49.306 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:49.022+0000 7f8883064200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:27:49.306 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:49.132+0000 7f8883064200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:27:49.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:49 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:49.289+0000 7fd9dfe96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:27:49.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:49 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:49.472+0000 7fd9dfe96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:27:49.665 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:49 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:49.287+0000 7fcf6d167200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:27:49.665 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:49 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:49.469+0000 7fcf6d167200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:27:49.713 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:49.303+0000 7f8883064200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:27:49.992 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:49 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:49.663+0000 7fcf6d167200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:27:50.089 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:49 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:49.668+0000 7fd9dfe96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:27:50.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:50 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:50.034+0000 7f8883064200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:27:50.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:50 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:50.247+0000 7f8883064200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:27:50.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:50 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:50.374+0000 7fcf6d167200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:27:50.839 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:50 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:50.378+0000 7fd9dfe96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:27:50.963 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:50 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:50.473+0000 7f8883064200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:27:51.130 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:50 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:50.850+0000 7fcf6d167200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:27:51.131 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:50 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:50.939+0000 7fcf6d167200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:27:51.134 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:50 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:50.853+0000 7fd9dfe96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:27:51.134 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:50 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:50.942+0000 7fd9dfe96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:27:51.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:51.203+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 19) v1 ==== 135783+0+0 (secure 0 0 0) 0x7f7ed80965a0 con 0x7f7ee8100d80 2023-12-14T18:27:51.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:51.294+0000 7f8883064200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:27:51.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: Standby manager daemon y restarted 2023-12-14T18:27:51.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: Standby manager daemon y started 2023-12-14T18:27:51.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:27:51.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: Standby manager daemon z restarted 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: Standby manager daemon z started 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:27:51.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-mon[42085]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:27:51.492 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y[44953]: 2023-12-14T18:27:51.129+0000 7fcf6d167200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:27:51.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: Standby manager daemon y restarted 2023-12-14T18:27:51.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: Standby manager daemon y started 2023-12-14T18:27:51.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:27:51.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:27:51.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: Standby manager daemon z restarted 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: Standby manager daemon z started 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:27:51.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:51 smithi176 ceph-mon[43506]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:27:51.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z[45741]: 2023-12-14T18:27:51.132+0000 7fd9dfe96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:27:51.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: Standby manager daemon y restarted 2023-12-14T18:27:51.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: Standby manager daemon y started 2023-12-14T18:27:51.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: Standby manager daemon z restarted 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: Standby manager daemon z started 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.176:0/2938530949' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:27:51.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:51 smithi204 ceph-mon[43866]: from='mgr.? 172.21.15.204:0/65632445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:27:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:51.761+0000 7f7ee6ffd700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2688246057 2023-12-14T18:27:51.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:51.761+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-14T18:27:52.176 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:51.847+0000 7f8883064200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:27:52.176 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:51 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:51.949+0000 7f8883064200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:27:52.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:52.201+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 8 ==== mgrmap(e 20) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f7ed809a0f0 con 0x7f7ee8100d80 2023-12-14T18:27:52.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:52.201+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 msgr2=0x7f7ed00a36e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:27:52.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:52.201+0000 7f7ee4ff9700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/2688246057,v1:172.21.15.31:6801/2688246057] conn(0x7f7ed00a1230 0x7f7ed00a36e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.174+0000 7f8883064200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: Active manager daemon x restarted 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: Activating manager daemon x 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: osdmap e88: 12 total, 12 up, 12 in 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: mgrmap e20: x(active, starting, since 0.01458s), standbys: z, y 2023-12-14T18:27:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:27:52.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:27:52.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:27:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-12-14T18:27:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: Active manager daemon x restarted 2023-12-14T18:27:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: Activating manager daemon x 2023-12-14T18:27:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: osdmap e88: 12 total, 12 up, 12 in 2023-12-14T18:27:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: mgrmap e20: x(active, starting, since 0.01458s), standbys: z, y 2023-12-14T18:27:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:27:52.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:27:52.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:27:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: mgrmap e19: x(active, since 7m), standbys: z, y 2023-12-14T18:27:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: Active manager daemon x restarted 2023-12-14T18:27:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: Activating manager daemon x 2023-12-14T18:27:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: osdmap e88: 12 total, 12 up, 12 in 2023-12-14T18:27:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: mgrmap e20: x(active, starting, since 0.01458s), standbys: z, y 2023-12-14T18:27:52.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:27:52.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:27:52.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.462+0000 7f87e1cb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.462+0000 7f87e1cb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.462+0000 7f87e1cb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.462+0000 7f87e1cb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.462+0000 7f87e1cb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.465+0000 7f87de4af700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.465+0000 7f87de4af700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.465+0000 7f87de4af700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.965 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.465+0000 7f87de4af700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:52.965 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:27:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:27:52.465+0000 7f87de4af700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:53.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.221+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mon.2 v2:172.21.15.176:3300/0 9 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f7ed8099380 con 0x7f7ee8100d80 2023-12-14T18:27:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.222+0000 7f7ee4ff9700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7ed001b210 0x7f7ed001b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:53.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.222+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 --> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ee81028f0 con 0x7f7ed001b210 2023-12-14T18:27:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.227+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7ed001b210 0x7f7ed001b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.230+0000 7f7ee6ffd700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7ed001b210 0x7f7ed001b5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c010 tx=0x7f7edc005dc0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:53.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.289+0000 7f7ee4ff9700 1 -- 172.21.15.31:0/3632794068 <== mgr.15585 v2:172.21.15.31:6800/1869071623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7ee81028f0 con 0x7f7ed001b210 2023-12-14T18:27:53.292 INFO:teuthology.orchestra.run.smithi031.stdout:Schedule set for path / 2023-12-14T18:27:53.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7ed001b210 msgr2=0x7f7ed001b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7eedbea700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7ed001b210 0x7f7ed001b5e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00c010 tx=0x7f7edc005dc0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee8100d80 msgr2=0x7f7ee81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7eedbea700 1 --2- 172.21.15.31:0/3632794068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee8100d80 0x7f7ee81006f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800e9f0 tx=0x7f7ed800ed00 comp rx=0 tx=0).stop 2023-12-14T18:27:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7ee7fff700 1 -- 172.21.15.31:0/3632794068 reap_dead start 2023-12-14T18:27:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 shutdown_connections 2023-12-14T18:27:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.292+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 >> 172.21.15.31:0/3632794068 conn(0x7f7ee80fa720 msgr2=0x7f7ee81021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:53.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.296+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 shutdown_connections 2023-12-14T18:27:53.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.296+0000 7f7eedbea700 1 -- 172.21.15.31:0/3632794068 wait complete. 2023-12-14T18:27:53.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-12-14T18:27:53.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 -- 172.21.15.31:0/1791394919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 msgr2=0x7f85a41059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/1791394919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41059c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8590009990 tx=0x7f8590009ca0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 -- 172.21.15.31:0/1791394919 shutdown_connections 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/1791394919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4105f00 0x7f85a4108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/1791394919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/1791394919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41059c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 -- 172.21.15.31:0/1791394919 >> 172.21.15.31:0/1791394919 conn(0x7f85a40fa720 msgr2=0x7f85a40fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 -- 172.21.15.31:0/1791394919 shutdown_connections 2023-12-14T18:27:53.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.467+0000 7f85aaeeb700 1 -- 172.21.15.31:0/1791394919 wait complete. 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 Processor -- start 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 -- start start 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a4195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4105f00 0x7f85a419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a419db30 con 0x7f85a41035e0 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a419dc70 con 0x7f85a4100cc0 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85aaeeb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a419de00 con 0x7f85a4105f00 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a4195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60858/0 (socket says 172.21.15.31:60858) 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a4195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:45508/0 (socket says 172.21.15.31:45508) 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a3fff700 1 -- 172.21.15.31:0/941306092 learned_addr learned my addr 172.21.15.31:0/941306092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a9488700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4105f00 0x7f85a419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 -- 172.21.15.31:0/941306092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4105f00 msgr2=0x7f85a419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4105f00 0x7f85a419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 -- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 msgr2=0x7f85a41964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.468+0000 7f85a8c87700 1 -- 172.21.15.31:0/941306092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900095f0 con 0x7f85a4100cc0 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.469+0000 7f85a8c87700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a4195fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f859800e9f0 tx=0x7f859800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:53.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.469+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800c9c0 con 0x7f85a4100cc0 2023-12-14T18:27:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.469+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a410bc50 con 0x7f85a4100cc0 2023-12-14T18:27:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.469+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8598010430 con 0x7f85a4100cc0 2023-12-14T18:27:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.469+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598018680 con 0x7f85a4100cc0 2023-12-14T18:27:53.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.469+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a410bac0 con 0x7f85a4100cc0 2023-12-14T18:27:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.470+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a40fc290 con 0x7f85a4100cc0 2023-12-14T18:27:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.471+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f8598010860 con 0x7f85a4100cc0 2023-12-14T18:27:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.472+0000 7f85a1ffb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85940a0fd0 0x7f85940a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.472+0000 7f85a3fff700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85940a0fd0 0x7f85940a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.472+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f8598014070 con 0x7f85a4100cc0 2023-12-14T18:27:53.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.473+0000 7f85a3fff700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85940a0fd0 0x7f85940a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8590006010 tx=0x7f859000b260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.475+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8598096320 con 0x7f85a4100cc0 2023-12-14T18:27:53.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:27:53.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:27:53.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:27:53.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: Manager daemon x is now available 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:53.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:27:53.496 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:27:53.496 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:27:53.496 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:53 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:27:53.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:27:53.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:27:53.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:27:53.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:27:53.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:27:53.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:27:53.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:27:53.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: Manager daemon x is now available 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:27:53.593 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:53 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:27:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:27:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:27:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:27:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:27:53.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:27:53.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: Manager daemon x is now available 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:27:53.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:27:53.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:53 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:27:53.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.767+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 --> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f85a4061c40 con 0x7f85940a0fd0 2023-12-14T18:27:53.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.827+0000 7f85a1ffb700 1 -- 172.21.15.31:0/941306092 <== mgr.15585 v2:172.21.15.31:6800/1869071623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f85a4061c40 con 0x7f85940a0fd0 2023-12-14T18:27:53.830 INFO:teuthology.orchestra.run.smithi031.stdout:Retention added to path / 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.830+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85940a0fd0 msgr2=0x7f85940a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.830+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85940a0fd0 0x7f85940a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8590006010 tx=0x7f859000b260 comp rx=0 tx=0).stop 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.830+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 msgr2=0x7f85a4195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.830+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a4195fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f859800e9f0 tx=0x7f859800ed00 comp rx=0 tx=0).stop 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 shutdown_connections 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85940a0fd0 0x7f85940a3480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4105f00 0x7f85a419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85a4100cc0 0x7f85a4195fb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 --2- 172.21.15.31:0/941306092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85a41035e0 0x7f85a41964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 >> 172.21.15.31:0/941306092 conn(0x7f85a40fa720 msgr2=0x7f85a41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 shutdown_connections 2023-12-14T18:27:53.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:53.831+0000 7f85aaeeb700 1 -- 172.21.15.31:0/941306092 wait complete. 2023-12-14T18:27:53.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-14T18:27:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 -- 172.21.15.31:0/1949834007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 msgr2=0x7fe260100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 --2- 172.21.15.31:0/1949834007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260100650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b0a0 tx=0x7fe25000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 -- 172.21.15.31:0/1949834007 shutdown_connections 2023-12-14T18:27:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 --2- 172.21.15.31:0/1949834007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe260101dc0 0x7fe260108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 --2- 172.21.15.31:0/1949834007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260100650 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 --2- 172.21.15.31:0/1949834007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe260101430 0x7fe260101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 -- 172.21.15.31:0/1949834007 >> 172.21.15.31:0/1949834007 conn(0x7fe2600fb840 msgr2=0x7fe2600fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 -- 172.21.15.31:0/1949834007 shutdown_connections 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.003+0000 7fe2659b0700 1 -- 172.21.15.31:0/1949834007 wait complete. 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 Processor -- start 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 -- start start 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe260101430 0x7fe260198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe260101dc0 0x7fe26019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26019ff90 con 0x7fe260100240 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2601a00d0 con 0x7fe260101430 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.004+0000 7fe2659b0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2601a0260 con 0x7fe260101dc0 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60876/0 (socket says 172.21.15.31:60876) 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25e7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe260101430 0x7fe260198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 -- 172.21.15.31:0/3858663950 learned_addr learned my addr 172.21.15.31:0/3858663950 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25f7fe700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe260101dc0 0x7fe26019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 -- 172.21.15.31:0/3858663950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe260101dc0 msgr2=0x7fe26019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe260101dc0 0x7fe26019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 -- 172.21.15.31:0/3858663950 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe260101430 msgr2=0x7fe260198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe260101430 0x7fe260198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 -- 172.21.15.31:0/3858663950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe250009d00 con 0x7fe260100240 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.005+0000 7fe25effd700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260198410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b070 tx=0x7fe25000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.006+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25000e040 con 0x7fe260100240 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.006+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe250003c00 con 0x7fe260100240 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.006+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26010bc10 con 0x7fe260100240 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.006+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2500052a0 con 0x7fe260100240 2023-12-14T18:27:54.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.006+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26010ba80 con 0x7fe260100240 2023-12-14T18:27:54.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.007+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe260192970 con 0x7fe260100240 2023-12-14T18:27:54.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.008+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fe25001d480 con 0x7fe260100240 2023-12-14T18:27:54.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.009+0000 7fe2649ae700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe24c0a0f50 0x7fe24c0a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.009+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fe2500cd9f0 con 0x7fe260100240 2023-12-14T18:27:54.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.009+0000 7fe25e7fc700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe24c0a0f50 0x7fe24c0a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.009+0000 7fe25e7fc700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe24c0a0f50 0x7fe24c0a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe248009c60 tx=0x7fe248009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:54.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.013+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe250097080 con 0x7fe260100240 2023-12-14T18:27:54.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.250+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 --> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe260061a50 con 0x7fe24c0a0f50 2023-12-14T18:27:54.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.261+0000 7fe2649ae700 1 -- 172.21.15.31:0/3858663950 <== mgr.15585 v2:172.21.15.31:6800/1869071623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fe260061a50 con 0x7fe24c0a0f50 2023-12-14T18:27:54.263 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-12-14T00:00:00", "created": "2023-12-14T18:27:53", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-12-14T18:27:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.263+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe24c0a0f50 msgr2=0x7fe24c0a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.263+0000 7fe2659b0700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe24c0a0f50 0x7fe24c0a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe248009c60 tx=0x7fe248009260 comp rx=0 tx=0).stop 2023-12-14T18:27:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.263+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 msgr2=0x7fe260198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.263+0000 7fe2659b0700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260198410 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe25000b070 tx=0x7fe25000ba50 comp rx=0 tx=0).stop 2023-12-14T18:27:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 shutdown_connections 2023-12-14T18:27:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe24c0a0f50 0x7fe24c0a3400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe260101dc0 0x7fe26019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe260100240 0x7fe260198410 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 --2- 172.21.15.31:0/3858663950 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe260101430 0x7fe260198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 >> 172.21.15.31:0/3858663950 conn(0x7fe2600fb840 msgr2=0x7fe260106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 shutdown_connections 2023-12-14T18:27:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.264+0000 7fe2659b0700 1 -- 172.21.15.31:0/3858663950 wait complete. 2023-12-14T18:27:54.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-14T18:27:54.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 -- 172.21.15.31:0/1144888438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 msgr2=0x7f0870101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1144888438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 0x7f0870101880 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f08600098f0 tx=0x7f0860009c00 comp rx=0 tx=0).stop 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 -- 172.21.15.31:0/1144888438 shutdown_connections 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1144888438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870101dc0 0x7f0870108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1144888438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870100240 0x7f0870100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1144888438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 0x7f0870101880 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 -- 172.21.15.31:0/1144888438 >> 172.21.15.31:0/1144888438 conn(0x7f08700fb880 msgr2=0x7f08700fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 -- 172.21.15.31:0/1144888438 shutdown_connections 2023-12-14T18:27:54.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.422+0000 7f0877fa8700 1 -- 172.21.15.31:0/1144888438 wait complete. 2023-12-14T18:27:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 Processor -- start 2023-12-14T18:27:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 -- start start 2023-12-14T18:27:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870100240 0x7f0870195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 0x7f0870196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 0x7f087019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087019db70 con 0x7f0870101dc0 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087019dcb0 con 0x7f0870100240 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0877fa8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087019de40 con 0x7f0870101430 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.423+0000 7f0876545700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 0x7f087019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0876545700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 0x7f087019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50540/0 (socket says 172.21.15.31:50540) 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0876545700 1 -- 172.21.15.31:0/1585440990 learned_addr learned my addr 172.21.15.31:0/1585440990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870100240 0x7f0870195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875543700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 0x7f0870196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 -- 172.21.15.31:0/1585440990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 msgr2=0x7f0870196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 0x7f0870196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 -- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 msgr2=0x7f087019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 0x7f087019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 -- 172.21.15.31:0/1585440990 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c0095f0 con 0x7f0870100240 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0876545700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 0x7f087019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0875d44700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870100240 0x7f0870195ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.424+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c00cbd0 con 0x7f0870100240 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.425+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0860009520 con 0x7f0870100240 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.425+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f086c00cd30 con 0x7f0870100240 2023-12-14T18:27:54.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.425+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086c018710 con 0x7f0870100240 2023-12-14T18:27:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.425+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087010bb90 con 0x7f0870100240 2023-12-14T18:27:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.426+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0870067380 con 0x7f0870100240 2023-12-14T18:27:54.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.428+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 21) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f086c010980 con 0x7f0870100240 2023-12-14T18:27:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.428+0000 7f0866ffd700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f085c0a0fd0 0x7f085c0a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.429+0000 7f0875543700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f085c0a0fd0 0x7f085c0a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.429+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f086c014070 con 0x7f0870100240 2023-12-14T18:27:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.429+0000 7f0875543700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f085c0a0fd0 0x7f085c0a3480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0860005e90 tx=0x7f0860019040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:54.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.435+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f086c096e20 con 0x7f0870100240 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: [14/Dec/2023:18:27:52] ENGINE Bus STARTING 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: [14/Dec/2023:18:27:52] ENGINE Serving on https://172.21.15.31:7150 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: [14/Dec/2023:18:27:52] ENGINE Serving on http://172.21.15.31:8765 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: [14/Dec/2023:18:27:52] ENGINE Bus STARTED 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: mgrmap e21: x(active, since 1.03335s), standbys: z, y 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:54.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:54 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: [14/Dec/2023:18:27:52] ENGINE Bus STARTING 2023-12-14T18:27:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: [14/Dec/2023:18:27:52] ENGINE Serving on https://172.21.15.31:7150 2023-12-14T18:27:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: [14/Dec/2023:18:27:52] ENGINE Serving on http://172.21.15.31:8765 2023-12-14T18:27:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: [14/Dec/2023:18:27:52] ENGINE Bus STARTED 2023-12-14T18:27:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: mgrmap e21: x(active, since 1.03335s), standbys: z, y 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:54 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.690+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 --> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0870073dc0 con 0x7f085c0a0fd0 2023-12-14T18:27:54.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.697+0000 7f0866ffd700 1 -- 172.21.15.31:0/1585440990 <== mgr.15585 v2:172.21.15.31:6800/1869071623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f0870073dc0 con 0x7f085c0a0fd0 2023-12-14T18:27:54.700 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1M 6M3h 2023-12-14T18:27:54.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f085c0a0fd0 msgr2=0x7f085c0a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f085c0a0fd0 0x7f085c0a3480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0860005e90 tx=0x7f0860019040 comp rx=0 tx=0).stop 2023-12-14T18:27:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870100240 msgr2=0x7f0870195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870100240 0x7f0870195ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f086c005fd0 tx=0x7f086c00ee70 comp rx=0 tx=0).stop 2023-12-14T18:27:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 shutdown_connections 2023-12-14T18:27:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0870101dc0 0x7f087019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0870100240 0x7f0870195ff0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f085c0a0fd0 0x7f085c0a3480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 --2- 172.21.15.31:0/1585440990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0870101430 0x7f0870196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 >> 172.21.15.31:0/1585440990 conn(0x7f08700fb880 msgr2=0x7f08700fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 shutdown_connections 2023-12-14T18:27:54.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:27:54.700+0000 7f0877fa8700 1 -- 172.21.15.31:0/1585440990 wait complete. 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: [14/Dec/2023:18:27:52] ENGINE Bus STARTING 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: [14/Dec/2023:18:27:52] ENGINE Serving on https://172.21.15.31:7150 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: [14/Dec/2023:18:27:52] ENGINE Serving on http://172.21.15.31:8765 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: [14/Dec/2023:18:27:52] ENGINE Bus STARTED 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: mgrmap e21: x(active, since 1.03335s), standbys: z, y 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='client.25460 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:54 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:54.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-12-14T18:27:54.751 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-12-14T18:27:54.762 INFO:teuthology.run_tasks:Running task background_exec... 2023-12-14T18:27:54.772 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:27:54.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-12-14T18:27:54.794 INFO:teuthology.run_tasks:Running task workunit... 2023-12-14T18:27:54.806 INFO:tasks.workunit:Pulling workunits from ref bf026da9d69000f3d4a9d16360156095d6612a2d 2023-12-14T18:27:54.807 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-14T18:27:54.807 INFO:tasks.workunit:timeout=3h 2023-12-14T18:27:54.807 INFO:tasks.workunit:cleanup=True 2023-12-14T18:27:54.807 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:54.829 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:54.829 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-12-14T18:27:54.829 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-12-14T18:27:54.830 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:27:54.830 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:cephfs_t:s0 2023-12-14T18:27:54.830 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-12-14 18:26:23.890815384 +0000 2023-12-14T18:27:54.830 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-12-14 18:26:23.890815384 +0000 2023-12-14T18:27:54.830 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-12-14 18:27:35.004943363 +0000 2023-12-14T18:27:54.830 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-12-14 18:26:23.890815384 +0000 2023-12-14T18:27:54.830 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-14T18:27:54.830 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-14T18:27:54.916 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/chrisphoffman/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bf026da9d69000f3d4a9d16360156095d6612a2d 2023-12-14T18:27:54.937 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-14T18:27:55.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='client.25499 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:55.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:55.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: mgrmap e22: x(active, since 2s), standbys: z, y 2023-12-14T18:27:55.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:55.495 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:55 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:55.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='client.25499 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:55.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:55.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: mgrmap e22: x(active, since 2s), standbys: z, y 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:55.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:55 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='client.25499 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: mgrmap e22: x(active, since 2s), standbys: z, y 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.716 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:55.717 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:55 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:56.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi176 to 182.5M 2023-12-14T18:27:56.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Unable to set osd_memory_target on smithi176 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2023-12-14T18:27:56.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='client.25511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi204 to 182.4M 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Unable to set osd_memory_target on smithi204 to 191346892: error parsing value: Value '191346892' is below minimum 939524096 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi031 to 183.5M 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Unable to set osd_memory_target on smithi031 to 192481587: error parsing value: Value '192481587' is below minimum 939524096 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:56 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi176 to 182.5M 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Unable to set osd_memory_target on smithi176 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='client.25511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi204 to 182.4M 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Unable to set osd_memory_target on smithi204 to 191346892: error parsing value: Value '191346892' is below minimum 939524096 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi031 to 183.5M 2023-12-14T18:27:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Unable to set osd_memory_target on smithi031 to 192481587: error parsing value: Value '192481587' is below minimum 939524096 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:56 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi176 to 182.5M 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Unable to set osd_memory_target on smithi176 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='client.25511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi204 to 182.4M 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Unable to set osd_memory_target on smithi204 to 191346892: error parsing value: Value '191346892' is below minimum 939524096 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi031 to 183.5M 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Unable to set osd_memory_target on smithi031 to 192481587: error parsing value: Value '192481587' is below minimum 939524096 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi031:/etc/ceph/ceph.conf 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi176:/etc/ceph/ceph.conf 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi204:/etc/ceph/ceph.conf 2023-12-14T18:27:56.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:56.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:56 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:57.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:57.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:57.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:27:57.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:57 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:57.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:27:57.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:57 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Updating smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Updating smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Updating smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:57.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:57.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:27:57.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:27:57.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:57.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:27:57.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:57 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:27:59.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:59 smithi204 ceph-mon[43866]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:59.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:27:59 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:27:59.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:59 smithi031 ceph-mon[42085]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:59.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:27:59 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:27:59.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:59 smithi176 ceph-mon[43506]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:27:59.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:27:59 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:28:01.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:28:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:28:00.838+0000 7f87c0133700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:28:01.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:28:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:28:00.838+0000 7f87c0133700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:28:01.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:28:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:28:00.838+0000 7f87c0133700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:28:01.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:28:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:28:00.838+0000 7f87c0133700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:28:01.214 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:28:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:28:00.838+0000 7f87c0133700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:28:01.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:01 smithi204 ceph-mon[43866]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-12-14T18:28:01.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:01 smithi031 ceph-mon[42085]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-12-14T18:28:01.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:01 smithi176 ceph-mon[43506]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-12-14T18:28:03.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:03 smithi031 ceph-mon[42085]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-12-14T18:28:03.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:03 smithi031 ceph-mon[42085]: mgrmap e23: x(active, since 10s), standbys: z, y 2023-12-14T18:28:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:03 smithi176 ceph-mon[43506]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-12-14T18:28:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:03 smithi176 ceph-mon[43506]: mgrmap e23: x(active, since 10s), standbys: z, y 2023-12-14T18:28:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:03 smithi204 ceph-mon[43866]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-12-14T18:28:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:03 smithi204 ceph-mon[43866]: mgrmap e23: x(active, since 10s), standbys: z, y 2023-12-14T18:28:05.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:05 smithi031 ceph-mon[42085]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 108 KiB/s wr, 49 op/s 2023-12-14T18:28:05.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:05 smithi176 ceph-mon[43506]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 108 KiB/s wr, 49 op/s 2023-12-14T18:28:05.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:05 smithi204 ceph-mon[43866]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 108 KiB/s wr, 49 op/s 2023-12-14T18:28:07.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:07 smithi031 ceph-mon[42085]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 99 KiB/s wr, 44 op/s 2023-12-14T18:28:07.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:07 smithi176 ceph-mon[43506]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 99 KiB/s wr, 44 op/s 2023-12-14T18:28:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:07 smithi204 ceph-mon[43866]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 99 KiB/s wr, 44 op/s 2023-12-14T18:28:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3073410885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd80ff380 msgr2=0x7f3dd8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3073410885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd80ff380 0x7f3dd8104e50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0099c0 tx=0x7f3dcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3073410885 shutdown_connections 2023-12-14T18:28:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3073410885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd8105580 0x7f3dd810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3073410885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3dd80fe9e0 0x7f3dd80fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3073410885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd80ff380 0x7f3dd8104e50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3073410885 >> 172.21.15.31:0/3073410885 conn(0x7f3dd8075810 msgr2=0x7f3dd8075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3073410885 shutdown_connections 2023-12-14T18:28:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.992+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3073410885 wait complete. 2023-12-14T18:28:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 Processor -- start 2023-12-14T18:28:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 -- start start 2023-12-14T18:28:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd80fe9e0 0x7f3dd81963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3dd80ff380 0x7f3dd8196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 0x7f3dd819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819b3a0 con 0x7f3dd8105580 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819b510 con 0x7f3dd80ff380 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dddcbf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819b6d0 con 0x7f3dd80fe9e0 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 0x7f3dd819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 0x7f3dd819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54500/0 (socket says 172.21.15.31:54500) 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dd7fff700 1 -- 172.21.15.31:0/3048723920 learned_addr learned my addr 172.21.15.31:0/3048723920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3dd80ff380 0x7f3dd8196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.993+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd80fe9e0 0x7f3dd81963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd7fff700 1 -- 172.21.15.31:0/3048723920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd80fe9e0 msgr2=0x7f3dd81963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd80fe9e0 0x7f3dd81963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd7fff700 1 -- 172.21.15.31:0/3048723920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3dd80ff380 msgr2=0x7f3dd8196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3dd80ff380 0x7f3dd8196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd7fff700 1 -- 172.21.15.31:0/3048723920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0095f0 con 0x7f3dd8105580 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 0x7f3dd819ad60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000ef30 tx=0x7f3dd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd4ff9700 1 -- 172.21.15.31:0/3048723920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0018070 con 0x7f3dd8105580 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd81102b0 con 0x7f3dd8105580 2023-12-14T18:28:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd4ff9700 1 -- 172.21.15.31:0/3048723920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dd000f040 con 0x7f3dd8105580 2023-12-14T18:28:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dd4ff9700 1 -- 172.21.15.31:0/3048723920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000cd00 con 0x7f3dd8105580 2023-12-14T18:28:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.994+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd8110120 con 0x7f3dd8105580 2023-12-14T18:28:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.997+0000 7f3dd4ff9700 1 -- 172.21.15.31:0/3048723920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f3dd0003a00 con 0x7f3dd8105580 2023-12-14T18:28:08.008 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.998+0000 7f3dd4ff9700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3dc00a13e0 0x7f3dc00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.998+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3dc00a13e0 0x7f3dc00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.998+0000 7f3dd4ff9700 1 -- 172.21.15.31:0/3048723920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f3dd00ccda0 con 0x7f3dd8105580 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 Processor -- start 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3dc00a13e0 0x7f3dc00a3890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8007be0 tx=0x7f3dc80073d0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 -- start start 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c389e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c38a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3974d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c274580 con 0x555d1c3950f0 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c2e3810 con 0x555d1c38a370 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c2dcca0 con 0x555d1c1e7430 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3dd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3dd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54512/0 (socket says 172.21.15.31:54512) 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2515293435 learned_addr learned my addr 172.21.15.31:0/2515293435 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c389e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:07.999+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c38a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.010 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2515293435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 msgr2=0x555d1c389e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c389e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2515293435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 msgr2=0x555d1c38a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c38a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2515293435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1c15e350 con 0x555d1c3950f0 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3974d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0099690 tx=0x7f3dd00994d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7d924f322d7802 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dbdffb700 1 -- 172.21.15.31:0/2515293435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0004ad0 con 0x555d1c3950f0 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dbdffb700 1 -- 172.21.15.31:0/2515293435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dd0099990 con 0x555d1c3950f0 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.000+0000 7f3dbdffb700 1 -- 172.21.15.31:0/2515293435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0099e50 con 0x555d1c3950f0 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2515293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 msgr2=0x555d1c3974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3974d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0099690 tx=0x7f3dd00994d0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2515293435 shutdown_connections 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3974d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c389e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2515293435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c38a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2515293435 >> 172.21.15.31:0/2515293435 conn(0x555d1c380890 msgr2=0x555d1c386200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2515293435 shutdown_connections 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.001+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2515293435 wait complete. 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 -- start start 2023-12-14T18:28:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 Processor -- start 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c394ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c391e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c3950f0 0x555d1c3924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c278f30 con 0x555d1c38a370 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c278bd0 con 0x555d1c3950f0 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c160640 con 0x555d1c1e7430 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3de9e4db80 10 client.0 _ll_register_callbacks cb 0x555d1c29c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3dd6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c391e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3dd6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c391e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54518/0 (socket says 172.21.15.31:54518) 2023-12-14T18:28:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.002+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/217778250 learned_addr learned my addr 172.21.15.31:0/217778250 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c394ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c3950f0 0x555d1c3924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 msgr2=0x555d1c394ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c394ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/217778250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c3950f0 msgr2=0x555d1c3924f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c3950f0 0x555d1c3924f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/217778250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd8110120 con 0x555d1c38a370 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c391e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009990 tx=0x7f3dcc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba49b93c131c0b9e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3db6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3db6ffd700 1 -- 172.21.15.31:0/217778250 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00b790 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3db6ffd700 1 -- 172.21.15.31:0/217778250 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f3dcc00b8f0 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d1c37dea0 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3db6ffd700 1 -- 172.21.15.31:0/217778250 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dcc00bab0 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.003+0000 7f3de9e4db80 10 client.15678 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555d1c2a3110 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3db6ffd700 1 -- 172.21.15.31:0/217778250 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d1c37dea0 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3db6ffd700 1 -- 172.21.15.31:0/217778250 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3dcc024ba0 con 0x555d1c38a370 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 10 client.15678 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 20 client.15678 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3a4be0 0x555d1c3a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 4 client.15678 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d1c2d5c80 con 0x555d1c3a4be0 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.004+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3a4be0 0x555d1c3a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.005+0000 7f3db6ffd700 10 client.15678 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.005+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3a4be0 0x555d1c3a6ff0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f316c884edddc9 server_cookie=70c458c792317c15 in_seq=0 out_seq=0 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.005+0000 7f3db6ffd700 1 -- 172.21.15.31:0/217778250 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d1c2d5c80 con 0x555d1c3a4be0 2023-12-14T18:28:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.005+0000 7f3db6ffd700 10 client.15678 handle_command_reply: tid=0 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.005+0000 7f3de9e4db80 1 client.15678 shutdown 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 msgr2=0x555d1c391e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c391e70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009990 tx=0x7f3dcc005040 comp rx=0 tx=0).stop 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 shutdown_connections 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3a4be0 0x555d1c3a6ff0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c3950f0 0x555d1c3924f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3db6ffd700 0 client.15678 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c394ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/217778250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c391e70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 >> 172.21.15.31:0/217778250 conn(0x555d1c380890 msgr2=0x555d1c385d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 shutdown_connections 2023-12-14T18:28:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 1 -- 172.21.15.31:0/217778250 wait complete. 2023-12-14T18:28:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.006+0000 7f3de9e4db80 20 client.15678 trim_cache size 0 max 16384 2023-12-14T18:28:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.021+0000 7f3de9e4db80 1 Processor -- start 2023-12-14T18:28:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.021+0000 7f3de9e4db80 1 -- start start 2023-12-14T18:28:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c1e7430 0x555d1c38a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c39bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c3950f0 0x555d1c385d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c3804d0 con 0x555d1c38a370 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c38dce0 con 0x555d1c1e7430 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c387760 con 0x555d1c3950f0 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c39bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c39bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54526/0 (socket says 172.21.15.31:54526) 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/530664364 learned_addr learned my addr 172.21.15.31:0/530664364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c1e7430 0x555d1c38a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c3950f0 0x555d1c385d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/530664364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c3950f0 msgr2=0x555d1c385d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c3950f0 0x555d1c385d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/530664364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c1e7430 msgr2=0x555d1c38a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c1e7430 0x555d1c38a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 -- 172.21.15.31:0/530664364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1c2a2e60 con 0x555d1c38a370 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.022+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c39bc90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc024f50 tx=0x7f3dcc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a3545f2d656164 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.023+0000 7f3dbe7fc700 1 -- 172.21.15.31:0/530664364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00f6a0 con 0x555d1c38a370 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.023+0000 7f3dbe7fc700 1 -- 172.21.15.31:0/530664364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dcc01e040 con 0x555d1c38a370 2023-12-14T18:28:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.023+0000 7f3dbe7fc700 1 -- 172.21.15.31:0/530664364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc024480 con 0x555d1c38a370 2023-12-14T18:28:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 -- 172.21.15.31:0/530664364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 msgr2=0x555d1c39bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c39bc90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc024f50 tx=0x7f3dcc005e80 comp rx=0 tx=0).stop 2023-12-14T18:28:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 -- 172.21.15.31:0/530664364 shutdown_connections 2023-12-14T18:28:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c3950f0 0x555d1c385d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c1e7430 0x555d1c38a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/530664364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c38a370 0x555d1c39bc90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 -- 172.21.15.31:0/530664364 >> 172.21.15.31:0/530664364 conn(0x555d1c380890 msgr2=0x555d1c3ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 -- 172.21.15.31:0/530664364 shutdown_connections 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 -- 172.21.15.31:0/530664364 wait complete. 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.024+0000 7f3de9e4db80 1 -- start start 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 Processor -- start 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c399e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c39a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c39aab0 con 0x555d1c3950f0 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c3f7fe0 con 0x555d1c38a370 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1c3f8120 con 0x555d1c1e7430 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3de9e4db80 10 client.0 _ll_register_callbacks cb 0x7f3dcc003ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54530/0 (socket says 172.21.15.31:54530) 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2108908550 learned_addr learned my addr 172.21.15.31:0/2108908550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd6ffd700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c39a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c399e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 msgr2=0x555d1c399e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c399e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2108908550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 msgr2=0x555d1c39a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c39a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.025+0000 7f3dd7fff700 1 -- 172.21.15.31:0/2108908550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d1c2a3110 con 0x555d1c3950f0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3dd7fff700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3f7aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0096400 tx=0x7f3dd00060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dbcefaca74b11e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3db6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3db6ffd700 1 -- 172.21.15.31:0/2108908550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd00c5030 con 0x555d1c3950f0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3db6ffd700 1 -- 172.21.15.31:0/2108908550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f3dd00c6070 con 0x555d1c3950f0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dd0099990 con 0x555d1c3950f0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3db6ffd700 1 -- 172.21.15.31:0/2108908550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dd0004ad0 con 0x555d1c3950f0 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 10 client.15684 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555d1c289940 con 0x555d1c3950f0 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3db6ffd700 1 -- 172.21.15.31:0/2108908550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3dd0099990 con 0x555d1c3950f0 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3db6ffd700 1 -- 172.21.15.31:0/2108908550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3dd0099bf0 con 0x555d1c3950f0 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 10 client.15684 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 20 client.15684 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.026+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3f9350 0x555d1c3fb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.027+0000 7f3de9e4db80 4 client.15684 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.027+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d1c2d59a0 con 0x555d1c3f9350 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.027+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3f9350 0x555d1c3fb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.027+0000 7f3db6ffd700 10 client.15684 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.027+0000 7f3dd77fe700 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3f9350 0x555d1c3fb730 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab794e2aa44a375 server_cookie=fb20ecfa70cc83b in_seq=0 out_seq=0 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.041+0000 7f3db6ffd700 1 -- 172.21.15.31:0/2108908550 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d1c2d59a0 con 0x555d1c3f9350 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.041+0000 7f3db6ffd700 10 client.15684 handle_command_reply: tid=0 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.041+0000 7f3de9e4db80 1 client.15684 shutdown 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.041+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 msgr2=0x555d1c3f7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.041+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3f7aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0096400 tx=0x7f3dd00060e0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 shutdown_connections 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555d1c3f9350 0x555d1c3fb730 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d1c3950f0 0x555d1c3f7aa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d1c1e7430 0x555d1c399e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3db6ffd700 0 client.15684 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 --2- 172.21.15.31:0/2108908550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d1c38a370 0x555d1c39a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 >> 172.21.15.31:0/2108908550 conn(0x555d1c380890 msgr2=0x555d1c385d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 shutdown_connections 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 1 -- 172.21.15.31:0/2108908550 wait complete. 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3de9e4db80 20 client.15684 trim_cache size 0 max 16384 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:28:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:28:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:28:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3dc00a13e0 msgr2=0x7f3dc00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:08.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3dc00a13e0 0x7f3dc00a3890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8007be0 tx=0x7f3dc80073d0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 msgr2=0x7f3dd819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.042+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 0x7f3dd819ad60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000ef30 tx=0x7f3dd000c3a0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 shutdown_connections 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd8105580 0x7f3dd819ad60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3dc00a13e0 0x7f3dc00a3890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd80fe9e0 0x7f3dd81963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 --2- 172.21.15.31:0/3048723920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3dd80ff380 0x7f3dd8196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 >> 172.21.15.31:0/3048723920 conn(0x7f3dd8075810 msgr2=0x7f3dd80fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 shutdown_connections 2023-12-14T18:28:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:08.043+0000 7f3dddcbf700 1 -- 172.21.15.31:0/3048723920 wait complete. 2023-12-14T18:28:09.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:09 smithi031 ceph-mon[42085]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 99 KiB/s wr, 44 op/s 2023-12-14T18:28:09.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:09 smithi176 ceph-mon[43506]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 99 KiB/s wr, 44 op/s 2023-12-14T18:28:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:09 smithi204 ceph-mon[43866]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 99 KiB/s wr, 44 op/s 2023-12-14T18:28:11.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:11 smithi031 ceph-mon[42085]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-12-14T18:28:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:11 smithi176 ceph-mon[43506]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-12-14T18:28:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:11 smithi204 ceph-mon[43866]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 102 KiB/s wr, 52 op/s 2023-12-14T18:28:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:13 smithi031 ceph-mon[42085]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-12-14T18:28:13.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:13 smithi176 ceph-mon[43506]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-12-14T18:28:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:13 smithi204 ceph-mon[43866]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 22 op/s 2023-12-14T18:28:13.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:28:14.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 -- 172.21.15.31:0/769120088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42880ffd30 msgr2=0x7f4288103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 --2- 172.21.15.31:0/769120088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42880ffd30 0x7f4288103cf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f42780099c0 tx=0x7f4278009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 -- 172.21.15.31:0/769120088 shutdown_connections 2023-12-14T18:28:14.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 --2- 172.21.15.31:0/769120088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42881043d0 0x7f428810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 --2- 172.21.15.31:0/769120088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42880ff390 0x7f42880ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 --2- 172.21.15.31:0/769120088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42880ffd30 0x7f4288103cf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 -- 172.21.15.31:0/769120088 >> 172.21.15.31:0/769120088 conn(0x7f4288075fd0 msgr2=0x7f42880763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 -- 172.21.15.31:0/769120088 shutdown_connections 2023-12-14T18:28:14.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.191+0000 7f4290764700 1 -- 172.21.15.31:0/769120088 wait complete. 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 Processor -- start 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 -- start start 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42880ff390 0x7f42881941d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42880ffd30 0x7f4288194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 0x7f4288198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42881991b0 con 0x7f42881043d0 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4288199320 con 0x7f42880ff390 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.192+0000 7f4290764700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42881994e0 con 0x7f42880ffd30 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 0x7f4288198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 0x7f4288198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54552/0 (socket says 172.21.15.31:54552) 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 -- 172.21.15.31:0/786581119 learned_addr learned my addr 172.21.15.31:0/786581119 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428e500700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42880ff390 0x7f42881941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428dcff700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42880ffd30 0x7f4288194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 -- 172.21.15.31:0/786581119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42880ffd30 msgr2=0x7f4288194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42880ffd30 0x7f4288194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 -- 172.21.15.31:0/786581119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42880ff390 msgr2=0x7f42881941d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42880ff390 0x7f42881941d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 -- 172.21.15.31:0/786581119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780095f0 con 0x7f42881043d0 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f428ed01700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 0x7f4288198b70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4280007ee0 tx=0x7f428000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.193+0000 7f427f7fe700 1 -- 172.21.15.31:0/786581119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000dd20 con 0x7f42881043d0 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.194+0000 7f427f7fe700 1 -- 172.21.15.31:0/786581119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f428000f040 con 0x7f42881043d0 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.194+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428810e130 con 0x7f42881043d0 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.194+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428810dfa0 con 0x7f42881043d0 2023-12-14T18:28:14.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.194+0000 7f427f7fe700 1 -- 172.21.15.31:0/786581119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42800048a0 con 0x7f42881043d0 2023-12-14T18:28:14.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.198+0000 7f427f7fe700 1 -- 172.21.15.31:0/786581119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f4280004020 con 0x7f42881043d0 2023-12-14T18:28:14.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.199+0000 7f427f7fe700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f42740a14e0 0x7f42740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.200+0000 7f427f7fe700 1 -- 172.21.15.31:0/786581119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f42800cca20 con 0x7f42881043d0 2023-12-14T18:28:14.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.201+0000 7f428e500700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f42740a14e0 0x7f42740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.201+0000 7f429c8f2b80 1 Processor -- start 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.202+0000 7f429c8f2b80 1 -- start start 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.202+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6380 0x55cc366c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.202+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.202+0000 7f428e500700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f42740a14e0 0x7f42740a3990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4284005950 tx=0x7f428400b700 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.202+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366d1b50 0x55cc366d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc365b05e0 con 0x55cc366c6c90 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3661f870 con 0x55cc366c6380 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc36618d00 con 0x55cc366d1b50 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54558/0 (socket says 172.21.15.31:54558) 2023-12-14T18:28:14.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 -- 172.21.15.31:0/2135478278 learned_addr learned my addr 172.21.15.31:0/2135478278 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428e500700 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6380 0x55cc366c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 -- 172.21.15.31:0/2135478278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366d1b50 msgr2=0x55cc366d3f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366d1b50 0x55cc366d3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 -- 172.21.15.31:0/2135478278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6380 msgr2=0x55cc366c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6380 0x55cc366c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 -- 172.21.15.31:0/2135478278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc3649a3b0 con 0x55cc366c6c90 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.203+0000 7f428dcff700 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366d1610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4278000c00 tx=0x7f4278005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860d9fa938a5f6e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.204+0000 7f427cff9700 1 -- 172.21.15.31:0/2135478278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427801d070 con 0x55cc366c6c90 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.204+0000 7f427cff9700 1 -- 172.21.15.31:0/2135478278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f427801e040 con 0x55cc366c6c90 2023-12-14T18:28:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.204+0000 7f427cff9700 1 -- 172.21.15.31:0/2135478278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278003b20 con 0x55cc366c6c90 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 -- 172.21.15.31:0/2135478278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 msgr2=0x55cc366d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366d1610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4278000c00 tx=0x7f4278005ee0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 -- 172.21.15.31:0/2135478278 shutdown_connections 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366d1b50 0x55cc366d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6380 0x55cc366c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/2135478278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366d1610 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.205+0000 7f429c8f2b80 1 -- 172.21.15.31:0/2135478278 >> 172.21.15.31:0/2135478278 conn(0x55cc366bcca0 msgr2=0x55cc366c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.206+0000 7f429c8f2b80 1 -- 172.21.15.31:0/2135478278 shutdown_connections 2023-12-14T18:28:14.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.206+0000 7f429c8f2b80 1 -- 172.21.15.31:0/2135478278 wait complete. 2023-12-14T18:28:14.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.206+0000 7f429c8f2b80 1 -- start start 2023-12-14T18:28:14.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.207+0000 7f429c8f2b80 1 Processor -- start 2023-12-14T18:28:14.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc365b4f90 con 0x55cc366d1b50 2023-12-14T18:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc365b4c30 con 0x55cc366c6c90 2023-12-14T18:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc3649c6a0 con 0x55cc366c6380 2023-12-14T18:28:14.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.208+0000 7f429c8f2b80 10 client.0 _ll_register_callbacks cb 0x55cc365d8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54574/0 (socket says 172.21.15.31:54574) 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 -- 172.21.15.31:0/4197482070 learned_addr learned my addr 172.21.15.31:0/4197482070 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428dcff700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 -- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 msgr2=0x55cc366d14d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366d14d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 -- 172.21.15.31:0/4197482070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 msgr2=0x55cc366ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428ed01700 1 -- 172.21.15.31:0/4197482070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4288067f90 con 0x55cc366d1b50 2023-12-14T18:28:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.209+0000 7f428dcff700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f428ed01700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366ced20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4280099220 tx=0x7f4280099110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70c461a116e4a0e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f4271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f4271ffb700 1 -- 172.21.15.31:0/4197482070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428009d2d0 con 0x55cc366d1b50 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f4271ffb700 1 -- 172.21.15.31:0/4197482070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f4280099520 con 0x55cc366d1b50 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc366ba2b0 con 0x55cc366d1b50 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f4271ffb700 1 -- 172.21.15.31:0/4197482070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f428009d550 con 0x55cc366d1b50 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f429c8f2b80 10 client.15696 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cc365df170 con 0x55cc366d1b50 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f4271ffb700 1 -- 172.21.15.31:0/4197482070 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc366ba2b0 con 0x55cc366d1b50 2023-12-14T18:28:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.210+0000 7f4271ffb700 1 -- 172.21.15.31:0/4197482070 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4280098a10 con 0x55cc366d1b50 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.211+0000 7f429c8f2b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.211+0000 7f429c8f2b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.211+0000 7f429c8f2b80 20 client.15696 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.211+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc366f2500 0x55cc366f4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.211+0000 7f429c8f2b80 4 client.15696 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.211+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc36611ce0 con 0x55cc366f2500 2023-12-14T18:28:14.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.212+0000 7f428e500700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc366f2500 0x55cc366f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.212+0000 7f428e500700 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc366f2500 0x55cc366f4910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=764fb3b47c011147 server_cookie=126e5f2e4e9ca380 in_seq=0 out_seq=0 2023-12-14T18:28:14.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.212+0000 7f4271ffb700 10 client.15696 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f4271ffb700 1 -- 172.21.15.31:0/4197482070 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc36611ce0 con 0x55cc366f2500 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f4271ffb700 10 client.15696 handle_command_reply: tid=0 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 client.15696 shutdown 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 msgr2=0x55cc366ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366ced20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4280099220 tx=0x7f4280099110 comp rx=0 tx=0).stop 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 shutdown_connections 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc366f2500 0x55cc366f4910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366ced20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366d14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4197482070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 >> 172.21.15.31:0/4197482070 conn(0x55cc366bcca0 msgr2=0x55cc366c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f4271ffb700 0 client.15696 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 shutdown_connections 2023-12-14T18:28:14.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4197482070 wait complete. 2023-12-14T18:28:14.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.213+0000 7f429c8f2b80 20 client.15696 trim_cache size 0 max 16384 2023-12-14T18:28:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 Processor -- start 2023-12-14T18:28:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 -- start start 2023-12-14T18:28:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc366d42b0 con 0x55cc366d1b50 2023-12-14T18:28:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc366c7330 con 0x55cc366c6c90 2023-12-14T18:28:14.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.232+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc366ccce0 con 0x55cc366c6380 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366d14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54584/0 (socket says 172.21.15.31:54584) 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428e500700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 -- 172.21.15.31:0/4054027643 learned_addr learned my addr 172.21.15.31:0/4054027643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428dcff700 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 -- 172.21.15.31:0/4054027643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 msgr2=0x55cc366e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366e9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 -- 172.21.15.31:0/4054027643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 msgr2=0x55cc366d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 -- 172.21.15.31:0/4054027643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc365deec0 con 0x55cc366d1b50 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428e500700 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.233+0000 7f428ed01700 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366d14d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4280004a20 tx=0x7f4280004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4693f4dd9d136f85 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.234+0000 7f427d7fa700 1 -- 172.21.15.31:0/4054027643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280004e60 con 0x55cc366d1b50 2023-12-14T18:28:14.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.235+0000 7f427d7fa700 1 -- 172.21.15.31:0/4054027643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42800c9bb0 con 0x55cc366d1b50 2023-12-14T18:28:14.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.235+0000 7f427d7fa700 1 -- 172.21.15.31:0/4054027643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42800a14e0 con 0x55cc366d1b50 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4054027643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 msgr2=0x55cc366d14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366d14d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4280004a20 tx=0x7f4280004ad0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4054027643 shutdown_connections 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366d1b50 0x55cc366d14d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/4054027643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366c6c90 0x55cc366d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.236+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4054027643 >> 172.21.15.31:0/4054027643 conn(0x55cc366bcca0 msgr2=0x55cc366f4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.237+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4054027643 shutdown_connections 2023-12-14T18:28:14.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.237+0000 7f429c8f2b80 1 -- 172.21.15.31:0/4054027643 wait complete. 2023-12-14T18:28:14.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.237+0000 7f429c8f2b80 1 -- start start 2023-12-14T18:28:14.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 Processor -- start 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 0x55cc366caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc366caf90 con 0x55cc366c6c90 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc366cb100 con 0x55cc366d1b50 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc366cb2c0 con 0x55cc366c6380 2023-12-14T18:28:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f429c8f2b80 10 client.0 _ll_register_callbacks cb 0x7f4280095860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f428ed01700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 0x55cc366caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f428ed01700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 0x55cc366caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:43814/0 (socket says 172.21.15.31:43814) 2023-12-14T18:28:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f428ed01700 1 -- 172.21.15.31:0/3861415328 learned_addr learned my addr 172.21.15.31:0/3861415328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.238+0000 7f428e500700 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f428ed01700 1 -- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 msgr2=0x55cc366cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f428ed01700 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f428ed01700 1 -- 172.21.15.31:0/3861415328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 msgr2=0x55cc366cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f428ed01700 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f428ed01700 1 -- 172.21.15.31:0/3861415328 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4284022040 con 0x55cc366d1b50 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f428ed01700 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 0x55cc366caa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4280099220 tx=0x7f4280021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4003f7828c2d04 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f4271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f4271ffb700 1 -- 172.21.15.31:0/3861415328 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428009d550 con 0x55cc366d1b50 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f427801e040 con 0x55cc366d1b50 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f4271ffb700 1 -- 172.21.15.31:0/3861415328 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f42800a1970 con 0x55cc366d1b50 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f4271ffb700 1 -- 172.21.15.31:0/3861415328 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4280004bd0 con 0x55cc366d1b50 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f429c8f2b80 10 client.25553 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:14.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.239+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cc365df170 con 0x55cc366d1b50 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f4271ffb700 1 -- 172.21.15.31:0/3861415328 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4280082080 con 0x55cc366d1b50 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f4271ffb700 1 -- 172.21.15.31:0/3861415328 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42800b7b50 con 0x55cc366d1b50 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f429c8f2b80 10 client.25553 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f429c8f2b80 10 client.25553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f429c8f2b80 20 client.25553 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc3673eec0 0x55cc367412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f429c8f2b80 4 client.25553 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc36611a00 con 0x55cc3673eec0 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f428e500700 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc3673eec0 0x55cc367412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f4271ffb700 10 client.25553 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.240+0000 7f428e500700 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc3673eec0 0x55cc367412a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0042def24b9e989 server_cookie=80087ce5da5fd9b6 in_seq=0 out_seq=0 2023-12-14T18:28:14.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f4271ffb700 1 -- 172.21.15.31:0/3861415328 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cc36611a00 con 0x55cc3673eec0 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f4271ffb700 10 client.25553 handle_command_reply: tid=0 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 client.25553 shutdown 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 msgr2=0x55cc366caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 0x55cc366caa00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4280099220 tx=0x7f4280021000 comp rx=0 tx=0).stop 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 shutdown_connections 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55cc3673eec0 0x55cc367412a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cc366d1b50 0x55cc366caa00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc366c6380 0x55cc366cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 --2- 172.21.15.31:0/3861415328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cc366c6c90 0x55cc366cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 >> 172.21.15.31:0/3861415328 conn(0x55cc366bcca0 msgr2=0x55cc366c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f4271ffb700 0 client.25553 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 shutdown_connections 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 1 -- 172.21.15.31:0/3861415328 wait complete. 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.241+0000 7f429c8f2b80 20 client.25553 trim_cache size 0 max 16384 2023-12-14T18:28:14.244 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:28:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:28:14.245 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:28:14.245 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:28:14.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f42740a14e0 msgr2=0x7f42740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f42740a14e0 0x7f42740a3990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4284005950 tx=0x7f428400b700 comp rx=0 tx=0).stop 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 msgr2=0x7f4288198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 0x7f4288198b70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4280007ee0 tx=0x7f428000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 shutdown_connections 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42881043d0 0x7f4288198b70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42880ff390 0x7f42881941d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f42740a14e0 0x7f42740a3990 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 --2- 172.21.15.31:0/786581119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42880ffd30 0x7f4288194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 >> 172.21.15.31:0/786581119 conn(0x7f4288075fd0 msgr2=0x7f4288102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 shutdown_connections 2023-12-14T18:28:14.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.242+0000 7f4290764700 1 -- 172.21.15.31:0/786581119 wait complete. 2023-12-14T18:28:14.262 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:28:14.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:28:14.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 -- 172.21.15.31:0/2140309204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c101c50 msgr2=0x7feb8c10a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 --2- 172.21.15.31:0/2140309204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c101c50 0x7feb8c10a320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009990 tx=0x7feb7c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 -- 172.21.15.31:0/2140309204 shutdown_connections 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 --2- 172.21.15.31:0/2140309204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c10a860 0x7feb8c10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 --2- 172.21.15.31:0/2140309204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101340 0x7feb8c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 --2- 172.21.15.31:0/2140309204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c101c50 0x7feb8c10a320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 -- 172.21.15.31:0/2140309204 >> 172.21.15.31:0/2140309204 conn(0x7feb8c075fd0 msgr2=0x7feb8c0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 -- 172.21.15.31:0/2140309204 shutdown_connections 2023-12-14T18:28:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.403+0000 7feb93851700 1 -- 172.21.15.31:0/2140309204 wait complete. 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.404+0000 7feb93851700 1 Processor -- start 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.404+0000 7feb93851700 1 -- start start 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb93851700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c101340 0x7feb8c0ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb93851700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 0x7feb8c0ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb93851700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c10a860 0x7feb8c1a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb93851700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0ffee0 con 0x7feb8c101c50 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb93851700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1a4ad0 con 0x7feb8c101340 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb93851700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1a4c10 con 0x7feb8c10a860 2023-12-14T18:28:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb90dec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 0x7feb8c0ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb90dec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 0x7feb8c0ff6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41424/0 (socket says 172.21.15.31:41424) 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb90dec700 1 -- 172.21.15.31:0/850785442 learned_addr learned my addr 172.21.15.31:0/850785442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb915ed700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c101340 0x7feb8c0ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb91dee700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c10a860 0x7feb8c1a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb915ed700 1 -- 172.21.15.31:0/850785442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c10a860 msgr2=0x7feb8c1a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb915ed700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c10a860 0x7feb8c1a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb915ed700 1 -- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 msgr2=0x7feb8c0ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb915ed700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 0x7feb8c0ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb915ed700 1 -- 172.21.15.31:0/850785442 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009c40 con 0x7feb8c101340 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.405+0000 7feb90dec700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 0x7feb8c0ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.406+0000 7feb915ed700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c101340 0x7feb8c0ff170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb8800d750 tx=0x7feb8800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.406+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88004220 con 0x7feb8c101340 2023-12-14T18:28:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.406+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb88004cf0 con 0x7feb8c101340 2023-12-14T18:28:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.406+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88003680 con 0x7feb8c101340 2023-12-14T18:28:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.406+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c0095f0 con 0x7feb8c101340 2023-12-14T18:28:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.406+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c110330 con 0x7feb8c101340 2023-12-14T18:28:14.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.407+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c1101a0 con 0x7feb8c101340 2023-12-14T18:28:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.409+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7feb88004750 con 0x7feb8c101340 2023-12-14T18:28:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.410+0000 7feb827fc700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7feb780a13e0 0x7feb780a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.410+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7feb880cd560 con 0x7feb8c101340 2023-12-14T18:28:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.410+0000 7feb90dec700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7feb780a13e0 0x7feb780a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.411+0000 7feb90dec700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7feb780a13e0 0x7feb780a3890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feb8c100960 tx=0x7feb7c00b260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.413+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7feb88096400 con 0x7feb8c101340 2023-12-14T18:28:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.729+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb8c1006a0 con 0x7feb8c101340 2023-12-14T18:28:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.730+0000 7feb827fc700 1 -- 172.21.15.31:0/850785442 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7feb88016090 con 0x7feb8c101340 2023-12-14T18:28:14.731 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:28:14.731 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:28:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7feb780a13e0 msgr2=0x7feb780a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7feb780a13e0 0x7feb780a3890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feb8c100960 tx=0x7feb7c00b260 comp rx=0 tx=0).stop 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c101340 msgr2=0x7feb8c0ff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c101340 0x7feb8c0ff170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb8800d750 tx=0x7feb8800da60 comp rx=0 tx=0).stop 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 shutdown_connections 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7feb780a13e0 0x7feb780a3890 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb8c10a860 0x7feb8c1a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c101340 0x7feb8c0ff170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 --2- 172.21.15.31:0/850785442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb8c101c50 0x7feb8c0ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 >> 172.21.15.31:0/850785442 conn(0x7feb8c075fd0 msgr2=0x7feb8c0fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 shutdown_connections 2023-12-14T18:28:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.733+0000 7feb93851700 1 -- 172.21.15.31:0/850785442 wait complete. 2023-12-14T18:28:14.736 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:28:14.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:28:14.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 -- 172.21.15.31:0/3009737614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4768069a90 msgr2=0x7f4768104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 --2- 172.21.15.31:0/3009737614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4768069a90 0x7f4768104e10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f475c0099c0 tx=0x7f475c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 -- 172.21.15.31:0/3009737614 shutdown_connections 2023-12-14T18:28:14.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 --2- 172.21.15.31:0/3009737614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47681054f0 0x7f476810d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 --2- 172.21.15.31:0/3009737614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f47680690f0 0x7f47680694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 --2- 172.21.15.31:0/3009737614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4768069a90 0x7f4768104e10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 -- 172.21.15.31:0/3009737614 >> 172.21.15.31:0/3009737614 conn(0x7f4768075fa0 msgr2=0x7f47680763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 -- 172.21.15.31:0/3009737614 shutdown_connections 2023-12-14T18:28:14.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.905+0000 7f476da00700 1 -- 172.21.15.31:0/3009737614 wait complete. 2023-12-14T18:28:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 Processor -- start 2023-12-14T18:28:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 -- start start 2023-12-14T18:28:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 0x7f47680ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4768069a90 0x7f47680ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47681054f0 0x7f47681a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47680fff00 con 0x7f47681054f0 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681a4b50 con 0x7f4768069a90 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.906+0000 7f476da00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681a4c90 con 0x7f47680690f0 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 0x7f47680ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 0x7f47680ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:42886/0 (socket says 172.21.15.31:42886) 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 -- 172.21.15.31:0/2705278134 learned_addr learned my addr 172.21.15.31:0/2705278134 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f47677fe700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47681054f0 0x7f47681a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f47667fc700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4768069a90 0x7f47680ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 -- 172.21.15.31:0/2705278134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4768069a90 msgr2=0x7f47680ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4768069a90 0x7f47680ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 -- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47681054f0 msgr2=0x7f47681a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47681054f0 0x7f47681a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 -- 172.21.15.31:0/2705278134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0095f0 con 0x7f47680690f0 2023-12-14T18:28:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.907+0000 7f4766ffd700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 0x7f47680ff190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f475800e9a0 tx=0x7f475800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.908+0000 7f476c9fe700 1 -- 172.21.15.31:0/2705278134 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758009790 con 0x7f47680690f0 2023-12-14T18:28:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.908+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47681102c0 con 0x7f47680690f0 2023-12-14T18:28:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.908+0000 7f476c9fe700 1 -- 172.21.15.31:0/2705278134 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4758010430 con 0x7f47680690f0 2023-12-14T18:28:14.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.908+0000 7f476c9fe700 1 -- 172.21.15.31:0/2705278134 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758018680 con 0x7f47680690f0 2023-12-14T18:28:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.909+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768110130 con 0x7f47680690f0 2023-12-14T18:28:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.910+0000 7f476c9fe700 1 -- 172.21.15.31:0/2705278134 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f47580105a0 con 0x7f47680690f0 2023-12-14T18:28:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.911+0000 7f476c9fe700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f47500a1460 0x7f47500a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.911+0000 7f47667fc700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f47500a1460 0x7f47500a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.911+0000 7f476c9fe700 1 -- 172.21.15.31:0/2705278134 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f4758014070 con 0x7f47680690f0 2023-12-14T18:28:14.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.912+0000 7f47667fc700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f47500a1460 0x7f47500a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f475c0098e0 tx=0x7f475c003680 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.914+0000 7f4779b8eb80 1 Processor -- start 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.914+0000 7f4779b8eb80 1 -- start start 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 0x562ddc904750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc917e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc918360 0x562ddc91a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc7ee5e0 con 0x562ddc918360 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc85d870 con 0x562ddc904380 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc856d00 con 0x562ddc904c90 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 0x562ddc904750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f47677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc918360 0x562ddc91a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 0x562ddc904750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46700/0 (socket says 172.21.15.31:46700) 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 -- 172.21.15.31:0/1880461485 learned_addr learned my addr 172.21.15.31:0/1880461485 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f47677fe700 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc918360 0x562ddc91a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41446/0 (socket says 172.21.15.31:41446) 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f47667fc700 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc917e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 -- 172.21.15.31:0/1880461485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 msgr2=0x562ddc917e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc917e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 -- 172.21.15.31:0/1880461485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc918360 msgr2=0x562ddc91a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc918360 0x562ddc91a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.915+0000 7f4766ffd700 1 -- 172.21.15.31:0/1880461485 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ddc6d83b0 con 0x562ddc904380 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.916+0000 7f4766ffd700 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 0x562ddc904750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4758096a60 tx=0x7f475809a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c0a961f65f6973 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.916+0000 7f474dffb700 1 -- 172.21.15.31:0/1880461485 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580a2440 con 0x562ddc904380 2023-12-14T18:28:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.916+0000 7f474dffb700 1 -- 172.21.15.31:0/1880461485 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4758010880 con 0x562ddc904380 2023-12-14T18:28:14.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.916+0000 7f474dffb700 1 -- 172.21.15.31:0/1880461485 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580a2210 con 0x562ddc904380 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 -- 172.21.15.31:0/1880461485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 msgr2=0x562ddc904750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 0x562ddc904750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4758096a60 tx=0x7f475809a110 comp rx=0 tx=0).stop 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 -- 172.21.15.31:0/1880461485 shutdown_connections 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc918360 0x562ddc91a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc904380 0x562ddc904750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/1880461485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc917e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 -- 172.21.15.31:0/1880461485 >> 172.21.15.31:0/1880461485 conn(0x562ddc8faca0 msgr2=0x562ddc9007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 -- 172.21.15.31:0/1880461485 shutdown_connections 2023-12-14T18:28:14.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 -- 172.21.15.31:0/1880461485 wait complete. 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.917+0000 7f4779b8eb80 1 -- start start 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 Processor -- start 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904380 0x562ddc911740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc911c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc917460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc7f2f90 con 0x562ddc904380 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc7f2c30 con 0x562ddc918360 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc6da6a0 con 0x562ddc904c90 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4779b8eb80 10 client.0 _ll_register_callbacks cb 0x562ddc816910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:14.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904380 0x562ddc911740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904380 0x562ddc911740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41452/0 (socket says 172.21.15.31:41452) 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f4766ffd700 1 -- 172.21.15.31:0/4111777947 learned_addr learned my addr 172.21.15.31:0/4111777947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f47667fc700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc911c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.918+0000 7f47677fe700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc917460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f47667fc700 1 -- 172.21.15.31:0/4111777947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 msgr2=0x562ddc917460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f47667fc700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc917460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f47667fc700 1 -- 172.21.15.31:0/4111777947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904380 msgr2=0x562ddc911740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f47667fc700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904380 0x562ddc911740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f47667fc700 1 -- 172.21.15.31:0/4111777947 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768110130 con 0x562ddc904c90 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f47667fc700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc911c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058e0 tx=0x7f475c009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01abdfdf6f75bc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4746ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4746ffd700 1 -- 172.21.15.31:0/4111777947 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c02c040 con 0x562ddc904c90 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4746ffd700 1 -- 172.21.15.31:0/4111777947 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f475c00fc50 con 0x562ddc904c90 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ddc8f82b0 con 0x562ddc904c90 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4746ffd700 1 -- 172.21.15.31:0/4111777947 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c00fe30 con 0x562ddc904c90 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4779b8eb80 10 client.25570 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.919+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562ddc81d170 con 0x562ddc904c90 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4746ffd700 1 -- 172.21.15.31:0/4111777947 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562ddc8f82b0 con 0x562ddc904c90 2023-12-14T18:28:14.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4746ffd700 1 -- 172.21.15.31:0/4111777947 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f475c024420 con 0x562ddc904c90 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4779b8eb80 10 client.25570 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4779b8eb80 10 client.25570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4779b8eb80 20 client.25570 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc930550 0x562ddc932960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4779b8eb80 4 client.25570 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.920+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ddc84fce0 con 0x562ddc930550 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.921+0000 7f4766ffd700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc930550 0x562ddc932960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.921+0000 7f4746ffd700 10 client.25570 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.921+0000 7f4766ffd700 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc930550 0x562ddc932960 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e2951b79cc38a40 server_cookie=1b2c9446aac38f0d in_seq=0 out_seq=0 2023-12-14T18:28:14.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4746ffd700 1 -- 172.21.15.31:0/4111777947 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562ddc84fce0 con 0x562ddc930550 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4746ffd700 10 client.25570 handle_command_reply: tid=0 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 client.25570 shutdown 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 msgr2=0x562ddc911c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc911c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058e0 tx=0x7f475c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 shutdown_connections 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc930550 0x562ddc932960 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc917460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904380 0x562ddc911740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/4111777947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904c90 0x562ddc911c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 >> 172.21.15.31:0/4111777947 conn(0x562ddc8faca0 msgr2=0x562ddc9003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4746ffd700 0 client.25570 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 shutdown_connections 2023-12-14T18:28:14.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.922+0000 7f4779b8eb80 1 -- 172.21.15.31:0/4111777947 wait complete. 2023-12-14T18:28:14.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.923+0000 7f4779b8eb80 20 client.25570 trim_cache size 0 max 16384 2023-12-14T18:28:14.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 Processor -- start 2023-12-14T18:28:14.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 -- start start 2023-12-14T18:28:14.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc917370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 0x562ddc909cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc90a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc92dca0 con 0x562ddc904c90 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc905390 con 0x562ddc918360 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc9055b0 con 0x562ddc904380 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc917370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc917370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:42906/0 (socket says 172.21.15.31:42906) 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.937+0000 7f4766ffd700 1 -- 172.21.15.31:0/2990423991 learned_addr learned my addr 172.21.15.31:0/2990423991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f4766ffd700 1 -- 172.21.15.31:0/2990423991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 msgr2=0x562ddc90a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f4766ffd700 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc90a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f4766ffd700 1 -- 172.21.15.31:0/2990423991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 msgr2=0x562ddc909cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:28:14.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f4766ffd700 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 0x562ddc909cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f4766ffd700 1 -- 172.21.15.31:0/2990423991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ddc81cec0 con 0x562ddc904380 2023-12-14T18:28:14.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f4766ffd700 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc917370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47580ca850 tx=0x7f47580cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d1c096502f6775 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f474e7fc700 1 -- 172.21.15.31:0/2990423991 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580ca070 con 0x562ddc904380 2023-12-14T18:28:14.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f474e7fc700 1 -- 172.21.15.31:0/2990423991 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47580b6070 con 0x562ddc904380 2023-12-14T18:28:14.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.938+0000 7f474e7fc700 1 -- 172.21.15.31:0/2990423991 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580b8040 con 0x562ddc904380 2023-12-14T18:28:14.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 -- 172.21.15.31:0/2990423991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 msgr2=0x562ddc917370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc917370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47580ca850 tx=0x7f47580cc040 comp rx=0 tx=0).stop 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 -- 172.21.15.31:0/2990423991 shutdown_connections 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc90a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc917370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/2990423991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 0x562ddc909cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 -- 172.21.15.31:0/2990423991 >> 172.21.15.31:0/2990423991 conn(0x562ddc8faca0 msgr2=0x562ddc907dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 -- 172.21.15.31:0/2990423991 shutdown_connections 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.939+0000 7f4779b8eb80 1 -- 172.21.15.31:0/2990423991 wait complete. 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.940+0000 7f4779b8eb80 1 -- start start 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.940+0000 7f4779b8eb80 1 Processor -- start 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc9149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 0x562ddc914ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc915430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc917850 con 0x562ddc904c90 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc983400 con 0x562ddc918360 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ddc9835c0 con 0x562ddc904380 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.941+0000 7f4779b8eb80 10 client.0 _ll_register_callbacks cb 0x7f475c00ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc9149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc9149b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:42922/0 (socket says 172.21.15.31:42922) 2023-12-14T18:28:14.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 -- 172.21.15.31:0/578784793 learned_addr learned my addr 172.21.15.31:0/578784793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:14.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 -- 172.21.15.31:0/578784793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 msgr2=0x562ddc915430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc915430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 -- 172.21.15.31:0/578784793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 msgr2=0x562ddc914ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 0x562ddc914ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 -- 172.21.15.31:0/578784793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ddc6d83b0 con 0x562ddc904380 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4766ffd700 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc9149b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47580a1f10 tx=0x7f47580984c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999590806e2fefba server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4746ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4746ffd700 1 -- 172.21.15.31:0/578784793 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580ca040 con 0x562ddc904380 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.942+0000 7f4746ffd700 1 -- 172.21.15.31:0/578784793 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f47580b83b0 con 0x562ddc904380 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4758010880 con 0x562ddc904380 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4746ffd700 1 -- 172.21.15.31:0/578784793 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47580b8040 con 0x562ddc904380 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 10 client.25576 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562ddc8039a0 con 0x562ddc904380 2023-12-14T18:28:14.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4746ffd700 1 -- 172.21.15.31:0/578784793 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4758010880 con 0x562ddc904380 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4746ffd700 1 -- 172.21.15.31:0/578784793 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47580a20d0 con 0x562ddc904380 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 10 client.25576 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 10 client.25576 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 20 client.25576 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc9849d0 0x562ddc986db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 4 client.25576 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.943+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562ddc84fa00 con 0x562ddc9849d0 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f47667fc700 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc9849d0 0x562ddc986db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f47667fc700 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc9849d0 0x562ddc986db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbcfb7e3b527f3cf server_cookie=a00ea83a44550022 in_seq=0 out_seq=0 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f4746ffd700 10 client.25576 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f4746ffd700 1 -- 172.21.15.31:0/578784793 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562ddc84fa00 con 0x562ddc9849d0 2023-12-14T18:28:14.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f4746ffd700 10 client.25576 handle_command_reply: tid=0 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f4779b8eb80 1 client.25576 shutdown 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 msgr2=0x562ddc9149b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.944+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc9149b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47580a1f10 tx=0x7f47580984c0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 shutdown_connections 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x562ddc9849d0 0x562ddc986db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562ddc918360 0x562ddc915430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ddc904380 0x562ddc9149b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4746ffd700 0 client.25576 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 --2- 172.21.15.31:0/578784793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562ddc904c90 0x562ddc914ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 >> 172.21.15.31:0/578784793 conn(0x562ddc8faca0 msgr2=0x562ddc906260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 shutdown_connections 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 1 -- 172.21.15.31:0/578784793 wait complete. 2023-12-14T18:28:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f4779b8eb80 20 client.25576 trim_cache size 0 max 16384 2023-12-14T18:28:14.950 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f47500a1460 msgr2=0x7f47500a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f47500a1460 0x7f47500a3910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f475c0098e0 tx=0x7f475c003680 comp rx=0 tx=0).stop 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 msgr2=0x7f47680ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 0x7f47680ff190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f475800e9a0 tx=0x7f475800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 shutdown_connections 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47681054f0 0x7f47681a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f47500a1460 0x7f47500a3910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47680690f0 0x7f47680ff190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 --2- 172.21.15.31:0/2705278134 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4768069a90 0x7f47680ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.945+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 >> 172.21.15.31:0/2705278134 conn(0x7f4768075fa0 msgr2=0x7f47680fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.946+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 shutdown_connections 2023-12-14T18:28:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:14.946+0000 7f476da00700 1 -- 172.21.15.31:0/2705278134 wait complete. 2023-12-14T18:28:14.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:28:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 -- 172.21.15.31:0/597733121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd00ff3d0 msgr2=0x7fbbd00ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/597733121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd00ff7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0099c0 tx=0x7fbbcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 -- 172.21.15.31:0/597733121 shutdown_connections 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/597733121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd0104410 0x7fbbd010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/597733121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd00ff7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/597733121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 0x7fbbd0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 -- 172.21.15.31:0/597733121 >> 172.21.15.31:0/597733121 conn(0x7fbbd0076260 msgr2=0x7fbbd0076660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.109+0000 7fbbd75b7700 1 -- 172.21.15.31:0/597733121 shutdown_connections 2023-12-14T18:28:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.110+0000 7fbbd75b7700 1 -- 172.21.15.31:0/597733121 wait complete. 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.110+0000 7fbbd75b7700 1 Processor -- start 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 -- start start 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd0196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 0x7fbbd01968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd0104410 0x7fbbd019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd019b350 con 0x7fbbd00ff3d0 2023-12-14T18:28:15.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd019b4c0 con 0x7fbbd00ffd70 2023-12-14T18:28:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd75b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd019b680 con 0x7fbbd0104410 2023-12-14T18:28:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd5353700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd0196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd5353700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd0196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41482/0 (socket says 172.21.15.31:41482) 2023-12-14T18:28:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd5353700 1 -- 172.21.15.31:0/3702782496 learned_addr learned my addr 172.21.15.31:0/3702782496 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 0x7fbbd01968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.111+0000 7fbbd5b54700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd0104410 0x7fbbd019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd4b52700 1 -- 172.21.15.31:0/3702782496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd0104410 msgr2=0x7fbbd019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd0104410 0x7fbbd019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd4b52700 1 -- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd00ff3d0 msgr2=0x7fbbd0196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd0196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd4b52700 1 -- 172.21.15.31:0/3702782496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0095f0 con 0x7fbbd00ffd70 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 0x7fbbd01968b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0100140 tx=0x7fbbc000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbc67fc700 1 -- 172.21.15.31:0/3702782496 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0010900 con 0x7fbbd00ffd70 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbc67fc700 1 -- 172.21.15.31:0/3702782496 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbc000c430 con 0x7fbbd00ffd70 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbc67fc700 1 -- 172.21.15.31:0/3702782496 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0018680 con 0x7fbbd00ffd70 2023-12-14T18:28:15.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd010f220 con 0x7fbbd00ffd70 2023-12-14T18:28:15.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.112+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd010f090 con 0x7fbbd00ffd70 2023-12-14T18:28:15.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.116+0000 7fbbc67fc700 1 -- 172.21.15.31:0/3702782496 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fbbc000c5a0 con 0x7fbbd00ffd70 2023-12-14T18:28:15.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.116+0000 7fbbc67fc700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbbc0a13e0 0x7fbbbc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.117+0000 7fbbc67fc700 1 -- 172.21.15.31:0/3702782496 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fbbc0014070 con 0x7fbbd00ffd70 2023-12-14T18:28:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.117+0000 7fbbd5353700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbbc0a13e0 0x7fbbbc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.117+0000 7fbbd5353700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbbc0a13e0 0x7fbbbc0a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0036c0 tx=0x7fbbcc019040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.118+0000 7fbbe3745b80 1 Processor -- start 2023-12-14T18:28:15.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 -- start start 2023-12-14T18:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae8d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae9af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ad775e0 con 0x55801ae98b50 2023-12-14T18:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ade6870 con 0x55801ae8dc90 2023-12-14T18:28:15.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801addfd00 con 0x55801ae8d380 2023-12-14T18:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbd5b54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae9af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbd5b54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae9af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41486/0 (socket says 172.21.15.31:41486) 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46758/0 (socket says 172.21.15.31:46758) 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbd5b54700 1 -- 172.21.15.31:0/2956286619 learned_addr learned my addr 172.21.15.31:0/2956286619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.119+0000 7fbbd5353700 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae8d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbd4b52700 1 -- 172.21.15.31:0/2956286619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 msgr2=0x55801ae8d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae8d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbd4b52700 1 -- 172.21.15.31:0/2956286619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 msgr2=0x55801ae9af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae9af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbd4b52700 1 -- 172.21.15.31:0/2956286619 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55801ac613b0 con 0x55801ae8dc90 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00ca660 tx=0x7fbbc0099b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b565b9597d4327a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbbb7fe700 1 -- 172.21.15.31:0/2956286619 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc009df40 con 0x55801ae8dc90 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbbb7fe700 1 -- 172.21.15.31:0/2956286619 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbc00a2360 con 0x55801ae8dc90 2023-12-14T18:28:15.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.120+0000 7fbbbb7fe700 1 -- 172.21.15.31:0/2956286619 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0099690 con 0x55801ae8dc90 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 -- 172.21.15.31:0/2956286619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 msgr2=0x55801ae98610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00ca660 tx=0x7fbbc0099b70 comp rx=0 tx=0).stop 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 -- 172.21.15.31:0/2956286619 shutdown_connections 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae9af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae8d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/2956286619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 -- 172.21.15.31:0/2956286619 >> 172.21.15.31:0/2956286619 conn(0x55801ae83ca0 msgr2=0x55801ae897e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 -- 172.21.15.31:0/2956286619 shutdown_connections 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 -- 172.21.15.31:0/2956286619 wait complete. 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.121+0000 7fbbe3745b80 1 -- start start 2023-12-14T18:28:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 Processor -- start 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae95830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae95f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ad7bf90 con 0x55801ae98b50 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ad7bc30 con 0x55801ae8dc90 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ac636a0 con 0x55801ae8d380 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.122+0000 7fbbe3745b80 10 client.0 _ll_register_callbacks cb 0x55801ad9f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae95830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae95830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46774/0 (socket says 172.21.15.31:46774) 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 -- 172.21.15.31:0/680293914 learned_addr learned my addr 172.21.15.31:0/680293914 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 -- 172.21.15.31:0/680293914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 msgr2=0x55801ae98580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 -- 172.21.15.31:0/680293914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 msgr2=0x55801ae95f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae95f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 -- 172.21.15.31:0/680293914 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd010f090 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae95830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0099c10 tx=0x7fbbc00ca5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433aa2f57057c560 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/680293914 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00a1ed0 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55801ae812b0 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.123+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/680293914 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fbbc00a2030 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/680293914 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbc00ca940 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 10 client.25583 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55801ada6170 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/680293914 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55801ae812b0 con 0x55801ae8dc90 2023-12-14T18:28:15.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/680293914 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbbc00b7f90 con 0x55801ae8dc90 2023-12-14T18:28:15.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 10 client.25583 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 20 client.25583 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801ae9ef80 0x55801aea1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 4 client.25583 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55801add8ce0 con 0x55801ae9ef80 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.124+0000 7fbbd5353700 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801ae9ef80 0x55801aea1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.125+0000 7fbbd5353700 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801ae9ef80 0x55801aea1390 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eeaf1287951a78a2 server_cookie=924f2ccdced2bf2b in_seq=0 out_seq=0 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.125+0000 7fbbb8ff9700 10 client.25583 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/680293914 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55801add8ce0 con 0x55801ae9ef80 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbb8ff9700 10 client.25583 handle_command_reply: tid=0 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 client.25583 shutdown 2023-12-14T18:28:15.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 msgr2=0x55801ae95830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae95830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0099c10 tx=0x7fbbc00ca5d0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 shutdown_connections 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae95f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801ae9ef80 0x55801aea1390 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801ae98580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/680293914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae95830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 >> 172.21.15.31:0/680293914 conn(0x55801ae83ca0 msgr2=0x55801ae89370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbb8ff9700 0 client.25583 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 shutdown_connections 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 1 -- 172.21.15.31:0/680293914 wait complete. 2023-12-14T18:28:15.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.126+0000 7fbbe3745b80 20 client.25583 trim_cache size 0 max 16384 2023-12-14T18:28:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.140+0000 7fbbe3745b80 1 Processor -- start 2023-12-14T18:28:15.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.140+0000 7fbbe3745b80 1 -- start start 2023-12-14T18:28:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.140+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801aea0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.140+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ae8ee70 con 0x55801ae98b50 2023-12-14T18:28:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ae91680 con 0x55801ae8dc90 2023-12-14T18:28:15.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801ae85210 con 0x55801ae8d380 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd5353700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801aea0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd5353700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801aea0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:42964/0 (socket says 172.21.15.31:42964) 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd5353700 1 -- 172.21.15.31:0/590219402 learned_addr learned my addr 172.21.15.31:0/590219402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd5b54700 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 -- 172.21.15.31:0/590219402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 msgr2=0x55801aea0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801aea0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 -- 172.21.15.31:0/590219402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 msgr2=0x55801ae98580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae98580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 -- 172.21.15.31:0/590219402 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55801ada5ec0 con 0x55801ae8dc90 2023-12-14T18:28:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.141+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0096410 tx=0x7fbbc00984c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91397c83e557eaaa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.142+0000 7fbbbbfff700 1 -- 172.21.15.31:0/590219402 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0098ec0 con 0x55801ae8dc90 2023-12-14T18:28:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.142+0000 7fbbbbfff700 1 -- 172.21.15.31:0/590219402 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbc000c880 con 0x55801ae8dc90 2023-12-14T18:28:15.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.142+0000 7fbbbbfff700 1 -- 172.21.15.31:0/590219402 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00a1ed0 con 0x55801ae8dc90 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.142+0000 7fbbe3745b80 1 -- 172.21.15.31:0/590219402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 msgr2=0x55801ae98040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.142+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0096410 tx=0x7fbbc00984c0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 -- 172.21.15.31:0/590219402 shutdown_connections 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae98b50 0x55801ae98580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae8d380 0x55801aea0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/590219402 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8dc90 0x55801ae98040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 -- 172.21.15.31:0/590219402 >> 172.21.15.31:0/590219402 conn(0x55801ae83ca0 msgr2=0x55801ae9b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 -- 172.21.15.31:0/590219402 shutdown_connections 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 -- 172.21.15.31:0/590219402 wait complete. 2023-12-14T18:28:15.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 -- start start 2023-12-14T18:28:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.143+0000 7fbbe3745b80 1 Processor -- start 2023-12-14T18:28:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8d380 0x55801aea7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 0x55801aea5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae98b50 0x55801aea57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801aea5f20 con 0x55801ae8dc90 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801aea6090 con 0x55801ae8d380 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55801af0c030 con 0x55801ae98b50 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbe3745b80 10 client.0 _ll_register_callbacks cb 0x7fbbc8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 0x55801aea5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd4b52700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 0x55801aea5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41520/0 (socket says 172.21.15.31:41520) 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd4b52700 1 -- 172.21.15.31:0/1938660609 learned_addr learned my addr 172.21.15.31:0/1938660609 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5353700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8d380 0x55801aea7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5b54700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae98b50 0x55801aea57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5353700 1 -- 172.21.15.31:0/1938660609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae98b50 msgr2=0x55801aea57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5353700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae98b50 0x55801aea57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5353700 1 -- 172.21.15.31:0/1938660609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 msgr2=0x55801aea5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5353700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 0x55801aea5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd5353700 1 -- 172.21.15.31:0/1938660609 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55801ada6170 con 0x55801ae8d380 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.144+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 0x55801aea5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbd5353700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8d380 0x55801aea7e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009fd0 tx=0x7fbbcc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7833d6d9c5ef6c8f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/1938660609 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc033d60 con 0x55801ae8d380 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/1938660609 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fbbcc004830 con 0x55801ae8d380 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbc00a2360 con 0x55801ae8d380 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/1938660609 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbcc0049f0 con 0x55801ae8d380 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbe3745b80 10 client.25589 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55801ad8c9a0 con 0x55801ae8d380 2023-12-14T18:28:15.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.145+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/1938660609 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbc00a2360 con 0x55801ae8d380 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/1938660609 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbbcc00f460 con 0x55801ae8d380 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbe3745b80 10 client.25589 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbe3745b80 10 client.25589 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbe3745b80 20 client.25589 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801aead380 0x55801aeaf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbe3745b80 4 client.25589 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55801add8a00 con 0x55801aead380 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801aead380 0x55801aeaf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbd4b52700 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801aead380 0x55801aeaf790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69364dc8b381c99f server_cookie=7f8993f4447cad3e in_seq=0 out_seq=0 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.146+0000 7fbbb8ff9700 10 client.25589 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbb8ff9700 1 -- 172.21.15.31:0/1938660609 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55801add8a00 con 0x55801aead380 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbb8ff9700 10 client.25589 handle_command_reply: tid=0 2023-12-14T18:28:15.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 client.25589 shutdown 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8d380 msgr2=0x55801aea7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8d380 0x55801aea7e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009fd0 tx=0x7fbbcc009310 comp rx=0 tx=0).stop 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 shutdown_connections 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55801aead380 0x55801aeaf790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55801ae98b50 0x55801aea57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55801ae8d380 0x55801aea7e90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 --2- 172.21.15.31:0/1938660609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55801ae8dc90 0x55801aea5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 >> 172.21.15.31:0/1938660609 conn(0x55801ae83ca0 msgr2=0x55801aea0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbb8ff9700 0 client.25589 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 shutdown_connections 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 1 -- 172.21.15.31:0/1938660609 wait complete. 2023-12-14T18:28:15.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.147+0000 7fbbe3745b80 20 client.25589 trim_cache size 0 max 16384 2023-12-14T18:28:15.153 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:28:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbbc0a13e0 msgr2=0x7fbbbc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbbc0a13e0 0x7fbbbc0a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0036c0 tx=0x7fbbcc019040 comp rx=0 tx=0).stop 2023-12-14T18:28:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 msgr2=0x7fbbd01968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 0x7fbbd01968b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0100140 tx=0x7fbbc000eca0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 shutdown_connections 2023-12-14T18:28:15.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbbc0a13e0 0x7fbbbc0a3890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbd0104410 0x7fbbd019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbd00ff3d0 0x7fbbd0196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 --2- 172.21.15.31:0/3702782496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbd00ffd70 0x7fbbd01968b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 >> 172.21.15.31:0/3702782496 conn(0x7fbbd0076260 msgr2=0x7fbbd0102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 shutdown_connections 2023-12-14T18:28:15.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.148+0000 7fbbd75b7700 1 -- 172.21.15.31:0/3702782496 wait complete. 2023-12-14T18:28:15.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:28:15.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 -- 172.21.15.31:0/3654429561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 msgr2=0x7fecf4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/3654429561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 0x7fecf4101810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009990 tx=0x7fecf0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 -- 172.21.15.31:0/3654429561 shutdown_connections 2023-12-14T18:28:15.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/3654429561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf410a810 0x7fecf410cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/3654429561 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 0x7fecf4101810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/3654429561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf4101d50 0x7fecf410a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.306+0000 7fecfb6eb700 1 -- 172.21.15.31:0/3654429561 >> 172.21.15.31:0/3654429561 conn(0x7fecf4076230 msgr2=0x7fecf4076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.307+0000 7fecfb6eb700 1 -- 172.21.15.31:0/3654429561 shutdown_connections 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.307+0000 7fecfb6eb700 1 -- 172.21.15.31:0/3654429561 wait complete. 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.307+0000 7fecfb6eb700 1 Processor -- start 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.307+0000 7fecfb6eb700 1 -- start start 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecfb6eb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 0x7fecf41963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecfb6eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 0x7fecf4196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecfb6eb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf410a810 0x7fecf419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecfb6eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf419b3a0 con 0x7fecf4101d50 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecfb6eb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf419b510 con 0x7fecf4101440 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecfb6eb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf419b6d0 con 0x7fecf410a810 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 0x7fecf4196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 0x7fecf4196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41548/0 (socket says 172.21.15.31:41548) 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 -- 172.21.15.31:0/1169045007 learned_addr learned my addr 172.21.15.31:0/1169045007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf9487700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 0x7fecf41963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf9c88700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf410a810 0x7fecf419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 -- 172.21.15.31:0/1169045007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf410a810 msgr2=0x7fecf419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf410a810 0x7fecf419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 -- 172.21.15.31:0/1169045007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 msgr2=0x7fecf41963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 0x7fecf41963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.308+0000 7fecf8c86700 1 -- 172.21.15.31:0/1169045007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf00095f0 con 0x7fecf4101d50 2023-12-14T18:28:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.309+0000 7fecf8c86700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 0x7fecf4196900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fece400e9a0 tx=0x7fece400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.309+0000 7fecea7fc700 1 -- 172.21.15.31:0/1169045007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4009790 con 0x7fecf4101d50 2023-12-14T18:28:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.309+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf41102d0 con 0x7fecf4101d50 2023-12-14T18:28:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.309+0000 7fecea7fc700 1 -- 172.21.15.31:0/1169045007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece4010430 con 0x7fecf4101d50 2023-12-14T18:28:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.309+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf4110140 con 0x7fecf4101d50 2023-12-14T18:28:15.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.309+0000 7fecea7fc700 1 -- 172.21.15.31:0/1169045007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4018680 con 0x7fecf4101d50 2023-12-14T18:28:15.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.313+0000 7fecea7fc700 1 -- 172.21.15.31:0/1169045007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fece400ca10 con 0x7fecf4101d50 2023-12-14T18:28:15.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.313+0000 7fecea7fc700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fece00a1460 0x7fece00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fecea7fc700 1 -- 172.21.15.31:0/1169045007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fece4014070 con 0x7fecf4101d50 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 Processor -- start 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 -- start start 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b37750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37c90 0x557d59b4ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b4b360 0x557d59b4d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59a215e0 con 0x557d59b37380 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59a90870 con 0x557d59b37c90 2023-12-14T18:28:15.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.314+0000 7fed07879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59a89d00 con 0x557d59b4b360 2023-12-14T18:28:15.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf9487700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b37750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf9487700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b37750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41552/0 (socket says 172.21.15.31:41552) 2023-12-14T18:28:15.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37c90 0x557d59b4ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf9487700 1 -- 172.21.15.31:0/2538992126 learned_addr learned my addr 172.21.15.31:0/2538992126 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf9c88700 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b4b360 0x557d59b4d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 -- 172.21.15.31:0/2538992126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b4b360 msgr2=0x557d59b4d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b4b360 0x557d59b4d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 -- 172.21.15.31:0/2538992126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 msgr2=0x557d59b37750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b37750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 -- 172.21.15.31:0/2538992126 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d5990b3b0 con 0x557d59b37c90 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf9487700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fece00a1460 0x7fece00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.315+0000 7fecf8c86700 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37c90 0x557d59b4ae20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fece40ca580 tx=0x7fece4099ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e53e094f06978f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.316+0000 7fecdf7fe700 1 -- 172.21.15.31:0/2538992126 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece40a22b0 con 0x557d59b37c90 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.316+0000 7fecf9487700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fece00a1460 0x7fece00a3910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fecf0015000 tx=0x7fecf0005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.316+0000 7fecdf7fe700 1 -- 172.21.15.31:0/2538992126 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece409a0c0 con 0x557d59b37c90 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.316+0000 7fecdf7fe700 1 -- 172.21.15.31:0/2538992126 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece40a2080 con 0x557d59b37c90 2023-12-14T18:28:15.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 -- 172.21.15.31:0/2538992126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37c90 msgr2=0x557d59b4ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37c90 0x557d59b4ae20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fece40ca580 tx=0x7fece4099ee0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 -- 172.21.15.31:0/2538992126 shutdown_connections 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b4b360 0x557d59b4d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b37750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 --2- 172.21.15.31:0/2538992126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37c90 0x557d59b4ae20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 -- 172.21.15.31:0/2538992126 >> 172.21.15.31:0/2538992126 conn(0x557d59b2dca0 msgr2=0x557d59b33720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 -- 172.21.15.31:0/2538992126 shutdown_connections 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 -- 172.21.15.31:0/2538992126 wait complete. 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 -- start start 2023-12-14T18:28:15.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.317+0000 7fed07879b80 1 Processor -- start 2023-12-14T18:28:15.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b47f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59a25f90 con 0x557d59b37380 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59a25c30 con 0x557d59b4b360 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5990d6a0 con 0x557d59b37c90 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.318+0000 7fed07879b80 10 client.0 _ll_register_callbacks cb 0x557d59a49910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46816/0 (socket says 172.21.15.31:46816) 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 -- 172.21.15.31:0/3119430225 learned_addr learned my addr 172.21.15.31:0/3119430225 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf8c86700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b47f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9487700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b4ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 -- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 msgr2=0x557d59b47f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b47f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 -- 172.21.15.31:0/3119430225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 msgr2=0x557d59b4ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b4ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 -- 172.21.15.31:0/3119430225 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf4110140 con 0x557d59b4b360 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9487700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecf9c88700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b484c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fecec011e80 tx=0x7fecec014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c8b005561919c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecdcff9700 1 -- 172.21.15.31:0/3119430225 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00e040 con 0x557d59b4b360 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d59b2b2b0 con 0x557d59b4b360 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.319+0000 7fecdcff9700 1 -- 172.21.15.31:0/3119430225 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fecec01a070 con 0x557d59b4b360 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fecdcff9700 1 -- 172.21.15.31:0/3119430225 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecec01e620 con 0x557d59b4b360 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 10 client.25601 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:15.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557d59a50170 con 0x557d59b4b360 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fecdcff9700 1 -- 172.21.15.31:0/3119430225 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d59b2b2b0 con 0x557d59b4b360 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fecdcff9700 1 -- 172.21.15.31:0/3119430225 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecec0166a0 con 0x557d59b4b360 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 10 client.25601 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 20 client.25601 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59b3d230 0x557d59b3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 4 client.25601 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d59a82ce0 con 0x557d59b3d230 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.320+0000 7fecf9487700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59b3d230 0x557d59b3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.321+0000 7fecdcff9700 10 client.25601 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.321+0000 7fecf9487700 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59b3d230 0x557d59b3f640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ebf71c65a90deaa server_cookie=290597a838397bf0 in_seq=0 out_seq=0 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fecdcff9700 1 -- 172.21.15.31:0/3119430225 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557d59a82ce0 con 0x557d59b3d230 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fecdcff9700 10 client.25601 handle_command_reply: tid=0 2023-12-14T18:28:15.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 client.25601 shutdown 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 msgr2=0x557d59b484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b484c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fecec011e80 tx=0x7fecec014600 comp rx=0 tx=0).stop 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 shutdown_connections 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59b3d230 0x557d59b3f640 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b484c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b4ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 --2- 172.21.15.31:0/3119430225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b47f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 >> 172.21.15.31:0/3119430225 conn(0x557d59b2dca0 msgr2=0x557d59b33310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fecdcff9700 0 client.25601 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 shutdown_connections 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 1 -- 172.21.15.31:0/3119430225 wait complete. 2023-12-14T18:28:15.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.322+0000 7fed07879b80 20 client.25601 trim_cache size 0 max 16384 2023-12-14T18:28:15.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 Processor -- start 2023-12-14T18:28:15.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 -- start start 2023-12-14T18:28:15.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b43320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b4a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59b38640 con 0x557d59b37380 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59b490c0 con 0x557d59b4b360 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fed07879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59b3a910 con 0x557d59b37c90 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fecf9c88700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b4ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fecf9487700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b43320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fecf9c88700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b4ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46818/0 (socket says 172.21.15.31:46818) 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fecf9487700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b43320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41574/0 (socket says 172.21.15.31:41574) 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.350+0000 7fecf9c88700 1 -- 172.21.15.31:0/2059939390 learned_addr learned my addr 172.21.15.31:0/2059939390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf8c86700 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b4a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf9c88700 1 -- 172.21.15.31:0/2059939390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 msgr2=0x557d59b4a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf9c88700 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b4a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf9c88700 1 -- 172.21.15.31:0/2059939390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 msgr2=0x557d59b43320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf9c88700 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b43320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf9c88700 1 -- 172.21.15.31:0/2059939390 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d59b2f4e0 con 0x557d59b4b360 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf8c86700 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b4a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecf9c88700 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b4ac60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fecec00b010 tx=0x7fecec014a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee09ba89c98516e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecdffff700 1 -- 172.21.15.31:0/2059939390 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00e040 con 0x557d59b4b360 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecdffff700 1 -- 172.21.15.31:0/2059939390 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecec00f040 con 0x557d59b4b360 2023-12-14T18:28:15.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.351+0000 7fecdffff700 1 -- 172.21.15.31:0/2059939390 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec005370 con 0x557d59b4b360 2023-12-14T18:28:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 -- 172.21.15.31:0/2059939390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 msgr2=0x557d59b4ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b4ac60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fecec00b010 tx=0x7fecec014a00 comp rx=0 tx=0).stop 2023-12-14T18:28:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 -- 172.21.15.31:0/2059939390 shutdown_connections 2023-12-14T18:28:15.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b4b360 0x557d59b4ac60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b37380 0x557d59b43320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 --2- 172.21.15.31:0/2059939390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b4a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 -- 172.21.15.31:0/2059939390 >> 172.21.15.31:0/2059939390 conn(0x557d59b2dca0 msgr2=0x557d59b3f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 -- 172.21.15.31:0/2059939390 shutdown_connections 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.352+0000 7fed07879b80 1 -- 172.21.15.31:0/2059939390 wait complete. 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 -- start start 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 Processor -- start 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 0x557d59b4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b4b360 0x557d59bb6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59b507c0 con 0x557d59b4b360 2023-12-14T18:28:15.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59bb6570 con 0x557d59b37380 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.353+0000 7fed07879b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d59bb6700 con 0x557d59b37c90 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fed07879b80 10 client.0 _ll_register_callbacks cb 0x7fecec0117b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 0x557d59b4fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9c88700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b4b360 0x557d59bb6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 0x557d59b4fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46832/0 (socket says 172.21.15.31:46832) 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 -- 172.21.15.31:0/464660146 learned_addr learned my addr 172.21.15.31:0/464660146 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9c88700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b4b360 0x557d59bb6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41582/0 (socket says 172.21.15.31:41582) 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf8c86700 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 -- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 msgr2=0x557d59b50090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b50090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 -- 172.21.15.31:0/464660146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b4b360 msgr2=0x557d59bb6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b4b360 0x557d59bb6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 -- 172.21.15.31:0/464660146 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d59a50170 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecf9487700 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 0x557d59b4fb50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x557d59b3ff70 tx=0x7fecf001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94b9e6ec1f26af0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.354+0000 7fecdcff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fecdcff9700 1 -- 172.21.15.31:0/464660146 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf00325c0 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fece409a0c0 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fecdcff9700 1 -- 172.21.15.31:0/464660146 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fecf002c030 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fecdcff9700 1 -- 172.21.15.31:0/464660146 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecf002c1f0 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 10 client.25607 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557d598b7b60 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fecdcff9700 1 -- 172.21.15.31:0/464660146 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fece409a0c0 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fecdcff9700 1 -- 172.21.15.31:0/464660146 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecf0032b90 con 0x557d59b37380 2023-12-14T18:28:15.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 10 client.25607 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 10 client.25607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 20 client.25607 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59bb7e70 0x557d59bba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 4 client.25607 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:28:15.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.355+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557d59a82a00 con 0x557d59bb7e70 2023-12-14T18:28:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.358+0000 7fecf8c86700 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59bb7e70 0x557d59bba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.359+0000 7fecf8c86700 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59bb7e70 0x557d59bba280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f8c1fbf92f127 server_cookie=5574ba54ef18b9d3 in_seq=0 out_seq=0 2023-12-14T18:28:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.359+0000 7fecdcff9700 10 client.25607 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.359+0000 7fecdcff9700 1 -- 172.21.15.31:0/464660146 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557d59a82a00 con 0x557d59bb7e70 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.359+0000 7fecdcff9700 10 client.25607 handle_command_reply: tid=0 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.359+0000 7fed07879b80 1 client.25607 shutdown 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 msgr2=0x557d59b4fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 0x557d59b4fb50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x557d59b3ff70 tx=0x7fecf001a040 comp rx=0 tx=0).stop 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 shutdown_connections 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557d59bb7e70 0x557d59bba280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557d59b4b360 0x557d59bb6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557d59b37380 0x557d59b4fb50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 --2- 172.21.15.31:0/464660146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d59b37c90 0x557d59b50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 >> 172.21.15.31:0/464660146 conn(0x557d59b2dca0 msgr2=0x557d59b43320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fecdcff9700 0 client.25607 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 shutdown_connections 2023-12-14T18:28:15.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 1 -- 172.21.15.31:0/464660146 wait complete. 2023-12-14T18:28:15.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.360+0000 7fed07879b80 20 client.25607 trim_cache size 0 max 16384 2023-12-14T18:28:15.364 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fece00a1460 msgr2=0x7fece00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fece00a1460 0x7fece00a3910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fecf0015000 tx=0x7fecf0005fb0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 msgr2=0x7fecf4196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 0x7fecf4196900 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fece400e9a0 tx=0x7fece400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 shutdown_connections 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fece00a1460 0x7fece00a3910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecf410a810 0x7fecf419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fecf4101440 0x7fecf41963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 --2- 172.21.15.31:0/1169045007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecf4101d50 0x7fecf4196900 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.361+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 >> 172.21.15.31:0/1169045007 conn(0x7fecf4076230 msgr2=0x7fecf41021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:15.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.362+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 shutdown_connections 2023-12-14T18:28:15.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:15.362+0000 7fecfb6eb700 1 -- 172.21.15.31:0/1169045007 wait complete. 2023-12-14T18:28:15.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:15 smithi031 ceph-mon[42085]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-12-14T18:28:15.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:15 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/850785442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:28:15.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:15 smithi176 ceph-mon[43506]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-12-14T18:28:15.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:15 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/850785442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:28:15.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:15 smithi204 ceph-mon[43866]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2023-12-14T18:28:15.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:15 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/850785442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:28:16.380 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:28:16.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:28:16.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 -- 172.21.15.31:0/1911655030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 msgr2=0x7ff170103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 --2- 172.21.15.31:0/1911655030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170103d50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff1600099c0 tx=0x7ff160009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 -- 172.21.15.31:0/1911655030 shutdown_connections 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 --2- 172.21.15.31:0/1911655030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff170104430 0x7ff17010c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 --2- 172.21.15.31:0/1911655030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1700ff3f0 0x7ff1700ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 --2- 172.21.15.31:0/1911655030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170103d50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 -- 172.21.15.31:0/1911655030 >> 172.21.15.31:0/1911655030 conn(0x7ff170075fd0 msgr2=0x7ff1700763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 -- 172.21.15.31:0/1911655030 shutdown_connections 2023-12-14T18:28:16.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.540+0000 7ff17786f700 1 -- 172.21.15.31:0/1911655030 wait complete. 2023-12-14T18:28:16.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.541+0000 7ff17786f700 1 Processor -- start 2023-12-14T18:28:16.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.541+0000 7ff17786f700 1 -- start start 2023-12-14T18:28:16.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17786f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1700ff3f0 0x7ff170100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17786f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17786f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170104430 0x7ff1701a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17786f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff170101b60 con 0x7ff1700ffd90 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17786f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1701a4ae0 con 0x7ff1700ff3f0 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17786f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1701a4c20 con 0x7ff170104430 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff174e0a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff174e0a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41612/0 (socket says 172.21.15.31:41612) 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff174e0a700 1 -- 172.21.15.31:0/1196679611 learned_addr learned my addr 172.21.15.31:0/1196679611 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17560b700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1700ff3f0 0x7ff170100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff175e0c700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170104430 0x7ff1701a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17560b700 1 -- 172.21.15.31:0/1196679611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170104430 msgr2=0x7ff1701a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17560b700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170104430 0x7ff1701a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17560b700 1 -- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 msgr2=0x7ff170101330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17560b700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170101330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff17560b700 1 -- 172.21.15.31:0/1196679611 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1600095f0 con 0x7ff1700ff3f0 2023-12-14T18:28:16.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.542+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:16.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.543+0000 7ff17560b700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1700ff3f0 0x7ff170100df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00b450 tx=0x7ff16c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.543+0000 7ff1667fc700 1 -- 172.21.15.31:0/1196679611 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c011830 con 0x7ff1700ff3f0 2023-12-14T18:28:16.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.543+0000 7ff1667fc700 1 -- 172.21.15.31:0/1196679611 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16c00be60 con 0x7ff1700ff3f0 2023-12-14T18:28:16.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.543+0000 7ff1667fc700 1 -- 172.21.15.31:0/1196679611 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c010d20 con 0x7ff1700ff3f0 2023-12-14T18:28:16.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.543+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17010f230 con 0x7ff1700ff3f0 2023-12-14T18:28:16.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.543+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17010f0a0 con 0x7ff1700ff3f0 2023-12-14T18:28:16.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.546+0000 7ff1667fc700 1 -- 172.21.15.31:0/1196679611 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff16c010380 con 0x7ff1700ff3f0 2023-12-14T18:28:16.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.547+0000 7ff1667fc700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff15c0a1460 0x7ff15c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.548+0000 7ff1667fc700 1 -- 172.21.15.31:0/1196679611 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7ff16c0ccad0 con 0x7ff1700ff3f0 2023-12-14T18:28:16.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 Processor -- start 2023-12-14T18:28:16.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff15c0a1460 0x7ff15c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 -- start start 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73fac07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fad43e0 0x55c73fad67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73f9aa660 con 0x55c73fad43e0 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fa198f0 con 0x55c73fac0400 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.549+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fa12d80 con 0x55c73fac0d10 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff175e0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fad43e0 0x55c73fad67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73fac07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff174e0a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73fac07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46854/0 (socket says 172.21.15.31:46854) 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 -- 172.21.15.31:0/3104343069 learned_addr learned my addr 172.21.15.31:0/3104343069 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff175e0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fad43e0 0x55c73fad67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41618/0 (socket says 172.21.15.31:41618) 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 -- 172.21.15.31:0/3104343069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 msgr2=0x55c73facb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 -- 172.21.15.31:0/3104343069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fad43e0 msgr2=0x55c73fad67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fad43e0 0x55c73fad67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 -- 172.21.15.31:0/3104343069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c73f894430 con 0x55c73fac0400 2023-12-14T18:28:16.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.550+0000 7ff17560b700 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73fac07d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00bfd0 tx=0x7ff16c099350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=379bf63fe218aad7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.551+0000 7ff15b7fe700 1 -- 172.21.15.31:0/3104343069 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c09d770 con 0x55c73fac0400 2023-12-14T18:28:16.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.551+0000 7ff15b7fe700 1 -- 172.21.15.31:0/3104343069 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16c0a1b90 con 0x55c73fac0400 2023-12-14T18:28:16.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.551+0000 7ff15b7fe700 1 -- 172.21.15.31:0/3104343069 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c098e90 con 0x55c73fac0400 2023-12-14T18:28:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 -- 172.21.15.31:0/3104343069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 msgr2=0x55c73fac07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73fac07d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00bfd0 tx=0x7ff16c099350 comp rx=0 tx=0).stop 2023-12-14T18:28:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff15c0a1460 0x7ff15c0a3910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff160009fd0 tx=0x7ff16000bc80 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 -- 172.21.15.31:0/3104343069 shutdown_connections 2023-12-14T18:28:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fad43e0 0x55c73fad67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73fac07d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/3104343069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 -- 172.21.15.31:0/3104343069 >> 172.21.15.31:0/3104343069 conn(0x55c73fab6d20 msgr2=0x55c73fabc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 -- 172.21.15.31:0/3104343069 shutdown_connections 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 -- 172.21.15.31:0/3104343069 wait complete. 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.552+0000 7ff1839fdb80 1 -- start start 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 Processor -- start 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0400 0x55c73facb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73fac8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fad43e0 0x55c73fac8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73f9af010 con 0x55c73fac0400 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73f9aecb0 con 0x55c73fad43e0 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73f896720 con 0x55c73fac0d10 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.553+0000 7ff1839fdb80 10 client.0 _ll_register_callbacks cb 0x55c73f9d2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73fac8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73fac8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:43042/0 (socket says 172.21.15.31:43042) 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff17560b700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0400 0x55c73facb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 -- 172.21.15.31:0/1216390602 learned_addr learned my addr 172.21.15.31:0/1216390602 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff175e0c700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fad43e0 0x55c73fac8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 -- 172.21.15.31:0/1216390602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fad43e0 msgr2=0x55c73fac8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fad43e0 0x55c73fac8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 -- 172.21.15.31:0/1216390602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0400 msgr2=0x55c73facb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0400 0x55c73facb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 -- 172.21.15.31:0/1216390602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17010f0a0 con 0x55c73fac0d10 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73fac8860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b580 tx=0x7ff160009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7032e9887c3a164a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.554+0000 7ff158ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff158ff9700 1 -- 172.21.15.31:0/1216390602 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16001d040 con 0x55c73fac0d10 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff158ff9700 1 -- 172.21.15.31:0/1216390602 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7ff160003cc0 con 0x55c73fac0d10 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c73fab4330 con 0x55c73fac0d10 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff158ff9700 1 -- 172.21.15.31:0/1216390602 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff160003e80 con 0x55c73fac0d10 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff1839fdb80 10 client.25618 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c73f9d91f0 con 0x55c73fac0d10 2023-12-14T18:28:16.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.555+0000 7ff158ff9700 1 -- 172.21.15.31:0/1216390602 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c73fab4330 con 0x55c73fac0d10 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.556+0000 7ff158ff9700 1 -- 172.21.15.31:0/1216390602 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff160031020 con 0x55c73fac0d10 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff1839fdb80 10 client.25618 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff1839fdb80 10 client.25618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff1839fdb80 20 client.25618 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fae3c40 0x55c73fae6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff1839fdb80 4 client.25618 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c73fa0bd60 con 0x55c73fae3c40 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff17560b700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fae3c40 0x55c73fae6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff17560b700 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fae3c40 0x55c73fae6050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4be84ea5d202ed server_cookie=b1e793c36d9084b0 in_seq=0 out_seq=0 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.557+0000 7ff158ff9700 10 client.25618 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff158ff9700 1 -- 172.21.15.31:0/1216390602 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c73fa0bd60 con 0x55c73fae3c40 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff158ff9700 10 client.25618 handle_command_reply: tid=0 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 client.25618 shutdown 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 msgr2=0x55c73fac8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73fac8860 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff16000b580 tx=0x7ff160009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 shutdown_connections 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fae3c40 0x55c73fae6050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fad43e0 0x55c73fac8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0400 0x55c73facb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1216390602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73fac8860 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 >> 172.21.15.31:0/1216390602 conn(0x55c73fab6d20 msgr2=0x55c73fabc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.558+0000 7ff158ff9700 0 client.25618 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.559+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 shutdown_connections 2023-12-14T18:28:16.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.559+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1216390602 wait complete. 2023-12-14T18:28:16.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.559+0000 7ff1839fdb80 20 client.25618 trim_cache size 0 max 16384 2023-12-14T18:28:16.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 Processor -- start 2023-12-14T18:28:16.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 -- start start 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 0x55c73fadaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73facb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fab6b50 con 0x55c73fab6d20 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fabb6b0 con 0x55c73fac0400 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fac1410 con 0x55c73fac0d10 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff17560b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 0x55c73fadaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff17560b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 0x55c73fadaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41640/0 (socket says 172.21.15.31:41640) 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.593+0000 7ff17560b700 1 -- 172.21.15.31:0/4163482317 learned_addr learned my addr 172.21.15.31:0/4163482317 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff175e0c700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73facb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 -- 172.21.15.31:0/4163482317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 msgr2=0x55c73facb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 -- 172.21.15.31:0/4163482317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 msgr2=0x55c73fadaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 0x55c73fadaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 -- 172.21.15.31:0/4163482317 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c73fab8560 con 0x55c73fac0400 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff175e0c700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff17560b700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 0x55c73fadaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff174e0a700 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73facb070 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff160006010 tx=0x7ff160028480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade184ea2a36e930 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.594+0000 7ff15bfff700 1 -- 172.21.15.31:0/4163482317 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16001d070 con 0x55c73fac0400 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.595+0000 7ff15bfff700 1 -- 172.21.15.31:0/4163482317 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff16000f620 con 0x55c73fac0400 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.595+0000 7ff15bfff700 1 -- 172.21.15.31:0/4163482317 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1600050c0 con 0x55c73fac0400 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 -- 172.21.15.31:0/4163482317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 msgr2=0x55c73facb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73facb070 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff160006010 tx=0x7ff160028480 comp rx=0 tx=0).stop 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 -- 172.21.15.31:0/4163482317 shutdown_connections 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0d10 0x55c73facb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fab6d20 0x55c73fadaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/4163482317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fac0400 0x55c73facb070 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 -- 172.21.15.31:0/4163482317 >> 172.21.15.31:0/4163482317 conn(0x55c73fae0390 msgr2=0x55c73fac3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 -- 172.21.15.31:0/4163482317 shutdown_connections 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 -- 172.21.15.31:0/4163482317 wait complete. 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 -- start start 2023-12-14T18:28:16.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.596+0000 7ff1839fdb80 1 Processor -- start 2023-12-14T18:28:16.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fab6d20 0x55c73fadad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0400 0x55c73fad9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 0x55c73fad9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fad9a80 con 0x55c73fac0d10 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fad9bf0 con 0x55c73fab6d20 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73fad9d60 con 0x55c73fac0400 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff1839fdb80 10 client.0 _ll_register_callbacks cb 0x7ff16000be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff175e0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 0x55c73fad9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff175e0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 0x55c73fad9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41644/0 (socket says 172.21.15.31:41644) 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff175e0c700 1 -- 172.21.15.31:0/1074650881 learned_addr learned my addr 172.21.15.31:0/1074650881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.597+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0400 0x55c73fad9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff174e0a700 1 -- 172.21.15.31:0/1074650881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fab6d20 msgr2=0x55c73fadad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fab6d20 0x55c73fadad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff174e0a700 1 -- 172.21.15.31:0/1074650881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 msgr2=0x55c73fad9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 0x55c73fad9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff174e0a700 1 -- 172.21.15.31:0/1074650881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c73f894430 con 0x55c73fac0400 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff175e0c700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 0x55c73fad9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff174e0a700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0400 0x55c73fad9000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55c73fab6930 tx=0x7ff160028d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f092ac8c98beca6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff158ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff158ff9700 1 -- 172.21.15.31:0/1074650881 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16001d070 con 0x55c73fac0400 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff158ff9700 1 -- 172.21.15.31:0/1074650881 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7ff160003d20 con 0x55c73fac0400 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.598+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c73fa0ba80 con 0x55c73fac0400 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff158ff9700 1 -- 172.21.15.31:0/1074650881 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff16000f620 con 0x55c73fac0400 2023-12-14T18:28:16.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 10 client.25624 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c73f9d91f0 con 0x55c73fac0400 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff158ff9700 1 -- 172.21.15.31:0/1074650881 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c73fa0ba80 con 0x55c73fac0400 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff158ff9700 1 -- 172.21.15.31:0/1074650881 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff160010ac0 con 0x55c73fac0400 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 10 client.25624 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 20 client.25624 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fad43e0 0x55c73fb3aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 4 client.25624 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.599+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c73f861d70 con 0x55c73fad43e0 2023-12-14T18:28:16.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.601+0000 7ff17560b700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fad43e0 0x55c73fb3aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.601+0000 7ff158ff9700 10 client.25624 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.601+0000 7ff17560b700 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fad43e0 0x55c73fb3aa80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb12b2b945181ff server_cookie=1e9fc6431ca43b14 in_seq=0 out_seq=0 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.602+0000 7ff158ff9700 1 -- 172.21.15.31:0/1074650881 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c73f861d70 con 0x55c73fad43e0 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.602+0000 7ff158ff9700 10 client.25624 handle_command_reply: tid=0 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.602+0000 7ff1839fdb80 1 client.25624 shutdown 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.603+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0400 msgr2=0x55c73fad9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.603+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0400 0x55c73fad9000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55c73fab6930 tx=0x7ff160028d60 comp rx=0 tx=0).stop 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 shutdown_connections 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c73fad43e0 0x55c73fb3aa80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c73fac0d10 0x55c73fad9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c73fab6d20 0x55c73fadad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 --2- 172.21.15.31:0/1074650881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c73fac0400 0x55c73fad9000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 >> 172.21.15.31:0/1074650881 conn(0x55c73fae0390 msgr2=0x55c73fabc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff158ff9700 0 client.25624 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 shutdown_connections 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.604+0000 7ff1839fdb80 1 -- 172.21.15.31:0/1074650881 wait complete. 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff1839fdb80 20 client.25624 trim_cache size 0 max 16384 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:28:16.607 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "be9ca9cc-0920-446f-96f1-151b4a537d78", 2023-12-14T18:28:16.608 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:28:16.608 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:28:16.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff15c0a1460 msgr2=0x7ff15c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff15c0a1460 0x7ff15c0a3910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff160009fd0 tx=0x7ff16000bc80 comp rx=0 tx=0).stop 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1700ff3f0 msgr2=0x7ff170100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1700ff3f0 0x7ff170100df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00b450 tx=0x7ff16c00b760 comp rx=0 tx=0).stop 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 shutdown_connections 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff15c0a1460 0x7ff15c0a3910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170104430 0x7ff1701a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff1700ff3f0 0x7ff170100df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 --2- 172.21.15.31:0/1196679611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1700ffd90 0x7ff170101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.605+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 >> 172.21.15.31:0/1196679611 conn(0x7ff170075fd0 msgr2=0x7ff170102ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.606+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 shutdown_connections 2023-12-14T18:28:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.606+0000 7ff17786f700 1 -- 172.21.15.31:0/1196679611 wait complete. 2023-12-14T18:28:16.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 -- 172.21.15.31:0/2617665353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c0ff430 msgr2=0x7f245c0ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 --2- 172.21.15.31:0/2617665353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c0ff430 0x7f245c0ff800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f244c009990 tx=0x7f244c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 -- 172.21.15.31:0/2617665353 shutdown_connections 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 --2- 172.21.15.31:0/2617665353 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c10acf0 0x7f245c10d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 --2- 172.21.15.31:0/2617665353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c0ff430 0x7f245c0ff800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 --2- 172.21.15.31:0/2617665353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ffd40 0x7f245c10a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 -- 172.21.15.31:0/2617665353 >> 172.21.15.31:0/2617665353 conn(0x7f245c0749c0 msgr2=0x7f245c074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 -- 172.21.15.31:0/2617665353 shutdown_connections 2023-12-14T18:28:16.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.769+0000 7f2461633700 1 -- 172.21.15.31:0/2617665353 wait complete. 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 Processor -- start 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 -- start start 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ff430 0x7f245c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 0x7f245c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c10acf0 0x7f245c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19b400 con 0x7f245c0ff430 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19b570 con 0x7f245c0ffd40 2023-12-14T18:28:16.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f2461633700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245c19b730 con 0x7f245c10acf0 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f245a7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 0x7f245c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f245a7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 0x7f245c196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46896/0 (socket says 172.21.15.31:46896) 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.770+0000 7f245a7fc700 1 -- 172.21.15.31:0/1924762046 learned_addr learned my addr 172.21.15.31:0/1924762046 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245affd700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ff430 0x7f245c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245b7fe700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c10acf0 0x7f245c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245affd700 1 -- 172.21.15.31:0/1924762046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c10acf0 msgr2=0x7f245c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245affd700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c10acf0 0x7f245c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245affd700 1 -- 172.21.15.31:0/1924762046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 msgr2=0x7f245c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245affd700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 0x7f245c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245affd700 1 -- 172.21.15.31:0/1924762046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450009520 con 0x7f245c0ff430 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.771+0000 7f245a7fc700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 0x7f245c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.772+0000 7f245affd700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ff430 0x7f245c196420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.772+0000 7f2443fff700 1 -- 172.21.15.31:0/1924762046 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c01d070 con 0x7f245c0ff430 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.772+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2454009c40 con 0x7f245c0ff430 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.772+0000 7f2443fff700 1 -- 172.21.15.31:0/1924762046 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f244c022470 con 0x7f245c0ff430 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.772+0000 7f2443fff700 1 -- 172.21.15.31:0/1924762046 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c017650 con 0x7f245c0ff430 2023-12-14T18:28:16.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.772+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c0095f0 con 0x7f245c0ff430 2023-12-14T18:28:16.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.776+0000 7f2443fff700 1 -- 172.21.15.31:0/1924762046 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f244c022890 con 0x7f245c0ff430 2023-12-14T18:28:16.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.776+0000 7f246d7c1b80 1 Processor -- start 2023-12-14T18:28:16.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f2443fff700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f24440a13e0 0x7f24440a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f2443fff700 1 -- 172.21.15.31:0/1924762046 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f244c0d0220 con 0x7f245c0ff430 2023-12-14T18:28:16.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f245b7fe700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f24440a13e0 0x7f24440a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 -- start start 2023-12-14T18:28:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206380 0x561c9f206750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 0x561c9f215a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f215f60 0x561c9f218340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f0f05e0 con 0x561c9f215f60 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f15f870 con 0x561c9f206c90 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.777+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f158d00 con 0x561c9f206380 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 0x561c9f215a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 0x561c9f215a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:46904/0 (socket says 172.21.15.31:46904) 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 -- 172.21.15.31:0/2126836825 learned_addr learned my addr 172.21.15.31:0/2126836825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245b7fe700 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f215f60 0x561c9f218340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245affd700 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206380 0x561c9f206750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 -- 172.21.15.31:0/2126836825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206380 msgr2=0x561c9f206750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206380 0x561c9f206750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 -- 172.21.15.31:0/2126836825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f215f60 msgr2=0x561c9f218340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f215f60 0x561c9f218340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 -- 172.21.15.31:0/2126836825 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9efda3b0 con 0x561c9f206c90 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245affd700 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206380 0x561c9f206750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245b7fe700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f24440a13e0 0x7f24440a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2454009fd0 tx=0x7f245400b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f245a7fc700 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 0x561c9f215a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2450011810 tx=0x7f2450011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e5db95f3cf9379 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f24417fa700 1 -- 172.21.15.31:0/2126836825 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450004020 con 0x561c9f206c90 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f24417fa700 1 -- 172.21.15.31:0/2126836825 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2450004830 con 0x561c9f206c90 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.778+0000 7f24417fa700 1 -- 172.21.15.31:0/2126836825 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450005440 con 0x561c9f206c90 2023-12-14T18:28:16.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.779+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2126836825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 msgr2=0x561c9f215a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.779+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 0x561c9f215a20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2450011810 tx=0x7f2450011b20 comp rx=0 tx=0).stop 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.783+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2126836825 shutdown_connections 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.783+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f215f60 0x561c9f218340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.783+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206380 0x561c9f206750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.783+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2126836825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206c90 0x561c9f215a20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.784+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2126836825 >> 172.21.15.31:0/2126836825 conn(0x561c9f1fcca0 msgr2=0x561c9f2027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.784+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2126836825 shutdown_connections 2023-12-14T18:28:16.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.784+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2126836825 wait complete. 2023-12-14T18:28:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.784+0000 7f246d7c1b80 1 -- start start 2023-12-14T18:28:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.784+0000 7f246d7c1b80 1 Processor -- start 2023-12-14T18:28:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 0x561c9f215710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f212c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f215f60 0x561c9f2131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f0f4f90 con 0x561c9f206380 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f0f4c30 con 0x561c9f215f60 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9efdc6a0 con 0x561c9f206c90 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f246d7c1b80 10 client.0 _ll_register_callbacks cb 0x561c9f118910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 0x561c9f215710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 0x561c9f215710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41682/0 (socket says 172.21.15.31:41682) 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 -- 172.21.15.31:0/212016675 learned_addr learned my addr 172.21.15.31:0/212016675 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 -- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 msgr2=0x561c9f212c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f212c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 -- 172.21.15.31:0/212016675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f215f60 msgr2=0x561c9f2131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f215f60 0x561c9f2131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.785+0000 7f245affd700 1 -- 172.21.15.31:0/212016675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2450011440 con 0x561c9f206380 2023-12-14T18:28:16.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f245affd700 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 0x561c9f215710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f244c0a2d60 tx=0x7f244c09b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4936d32e37eae6fe server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f243eff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f243eff5700 1 -- 172.21.15.31:0/212016675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c09aa10 con 0x561c9f206380 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f243eff5700 1 -- 172.21.15.31:0/212016675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f244c09aed0 con 0x561c9f206380 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c9f1fa2b0 con 0x561c9f206380 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f243eff5700 1 -- 172.21.15.31:0/212016675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244c09a830 con 0x561c9f206380 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f246d7c1b80 10 client.15777 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.786+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561c9f11f170 con 0x561c9f206380 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.787+0000 7f243eff5700 1 -- 172.21.15.31:0/212016675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c9f1fa2b0 con 0x561c9f206380 2023-12-14T18:28:16.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.787+0000 7f243eff5700 1 -- 172.21.15.31:0/212016675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f244c0cb870 con 0x561c9f206380 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f246d7c1b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f246d7c1b80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f246d7c1b80 20 client.15777 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f21b2e0 0x561c9f21d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f246d7c1b80 4 client.15777 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c9f151ce0 con 0x561c9f21b2e0 2023-12-14T18:28:16.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f245a7fc700 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f21b2e0 0x561c9f21d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f243eff5700 10 client.15777 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.788+0000 7f245a7fc700 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f21b2e0 0x561c9f21d710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f881916da2e324d7 server_cookie=2064861e4db0c332 in_seq=0 out_seq=0 2023-12-14T18:28:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f243eff5700 1 -- 172.21.15.31:0/212016675 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c9f151ce0 con 0x561c9f21b2e0 2023-12-14T18:28:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f243eff5700 10 client.15777 handle_command_reply: tid=0 2023-12-14T18:28:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 client.15777 shutdown 2023-12-14T18:28:16.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 msgr2=0x561c9f215710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 0x561c9f215710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f244c0a2d60 tx=0x7f244c09b410 comp rx=0 tx=0).stop 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 shutdown_connections 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f21b2e0 0x561c9f21d710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f215f60 0x561c9f2131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206380 0x561c9f215710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f243eff5700 0 client.15777 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/212016675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f212c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.789+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 >> 172.21.15.31:0/212016675 conn(0x561c9f1fcca0 msgr2=0x561c9f2023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.790+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 shutdown_connections 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.790+0000 7f246d7c1b80 1 -- 172.21.15.31:0/212016675 wait complete. 2023-12-14T18:28:16.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.790+0000 7f246d7c1b80 20 client.15777 trim_cache size 0 max 16384 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 Processor -- start 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 -- start start 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f1fcca0 0x561c9f211cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f215430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f20ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f213540 con 0x561c9f1fcca0 2023-12-14T18:28:16.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f203bc0 con 0x561c9f206380 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.808+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f211210 con 0x561c9f206c90 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f1fcca0 0x561c9f211cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f20ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f1fcca0 0x561c9f211cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41696/0 (socket says 172.21.15.31:41696) 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245affd700 1 -- 172.21.15.31:0/2675910741 learned_addr learned my addr 172.21.15.31:0/2675910741 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245a7fc700 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f215430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 -- 172.21.15.31:0/2675910741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 msgr2=0x561c9f215430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f215430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 -- 172.21.15.31:0/2675910741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f1fcca0 msgr2=0x561c9f211cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f1fcca0 0x561c9f211cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 -- 172.21.15.31:0/2675910741 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9f11eec0 con 0x561c9f206c90 2023-12-14T18:28:16.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f245b7fe700 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f20ff10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x561c9f207c80 tx=0x7f2454004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04df4071a6b852e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.809+0000 7f2441ffb700 1 -- 172.21.15.31:0/2675910741 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400e040 con 0x561c9f206c90 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f2441ffb700 1 -- 172.21.15.31:0/2675910741 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f245400f040 con 0x561c9f206c90 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f2441ffb700 1 -- 172.21.15.31:0/2675910741 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454017440 con 0x561c9f206c90 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2675910741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 msgr2=0x561c9f20ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f20ff10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x561c9f207c80 tx=0x7f2454004b40 comp rx=0 tx=0).stop 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2675910741 shutdown_connections 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f206c90 0x561c9f20ff10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f1fcca0 0x561c9f211cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/2675910741 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f215430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2675910741 >> 172.21.15.31:0/2675910741 conn(0x561c9f218610 msgr2=0x561c9f21d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2675910741 shutdown_connections 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.810+0000 7f246d7c1b80 1 -- 172.21.15.31:0/2675910741 wait complete. 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 -- start start 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 Processor -- start 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f1fcca0 0x561c9f27e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f27ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206c90 0x561c9f20b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f20ba80 con 0x561c9f206c90 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f20bbf0 con 0x561c9f206380 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9f20bdb0 con 0x561c9f1fcca0 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f246d7c1b80 10 client.0 _ll_register_callbacks cb 0x7f244c00b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.811+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206c90 0x561c9f20b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206c90 0x561c9f20b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41704/0 (socket says 172.21.15.31:41704) 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245b7fe700 1 -- 172.21.15.31:0/1183674184 learned_addr learned my addr 172.21.15.31:0/1183674184 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f1fcca0 0x561c9f27e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245a7fc700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f27ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 -- 172.21.15.31:0/1183674184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 msgr2=0x561c9f27ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f27ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 -- 172.21.15.31:0/1183674184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206c90 msgr2=0x561c9f20b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206c90 0x561c9f20b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 -- 172.21.15.31:0/1183674184 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2454009140 con 0x561c9f1fcca0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f245affd700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f1fcca0 0x561c9f27e830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f244c09b530 tx=0x7f244c0cbaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=426b048ff6460ab4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f243eff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f243eff5700 1 -- 172.21.15.31:0/1183674184 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c09ef00 con 0x561c9f1fcca0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f243eff5700 1 -- 172.21.15.31:0/1183674184 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f244c09aed0 con 0x561c9f1fcca0 2023-12-14T18:28:16.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.812+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2450004830 con 0x561c9f1fcca0 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f243eff5700 1 -- 172.21.15.31:0/1183674184 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244c09a830 con 0x561c9f1fcca0 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 10 client.25636 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561c9f1059a0 con 0x561c9f1fcca0 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f243eff5700 1 -- 172.21.15.31:0/1183674184 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2450004830 con 0x561c9f1fcca0 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f243eff5700 1 -- 172.21.15.31:0/1183674184 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f244c096cc0 con 0x561c9f1fcca0 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 10 client.25636 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 20 client.25636 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f215f60 0x561c9f289130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 4 client.25636 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561c9f151a00 con 0x561c9f215f60 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.813+0000 7f245a7fc700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f215f60 0x561c9f289130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:16.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.814+0000 7f243eff5700 10 client.25636 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.814+0000 7f245a7fc700 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f215f60 0x561c9f289130 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360568c7622f1ae1 server_cookie=57f564517f4614e1 in_seq=0 out_seq=0 2023-12-14T18:28:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.814+0000 7f243eff5700 1 -- 172.21.15.31:0/1183674184 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561c9f151a00 con 0x561c9f215f60 2023-12-14T18:28:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.814+0000 7f243eff5700 10 client.25636 handle_command_reply: tid=0 2023-12-14T18:28:16.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.814+0000 7f246d7c1b80 1 client.25636 shutdown 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f1fcca0 msgr2=0x561c9f27e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f1fcca0 0x561c9f27e830 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f244c09b530 tx=0x7f244c0cbaa0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 shutdown_connections 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c9f215f60 0x561c9f289130 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c9f206c90 0x561c9f20b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c9f1fcca0 0x561c9f27e830 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f243eff5700 0 client.25636 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:16.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 --2- 172.21.15.31:0/1183674184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c9f206380 0x561c9f27ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 >> 172.21.15.31:0/1183674184 conn(0x561c9f218610 msgr2=0x561c9f226790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 shutdown_connections 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 1 -- 172.21.15.31:0/1183674184 wait complete. 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.818+0000 7f246d7c1b80 20 client.25636 trim_cache size 0 max 16384 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout: "be9ca9cc-0920-446f-96f1-151b4a537d78": { 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "be9ca9cc-0920-446f-96f1-151b4a537d78", 2023-12-14T18:28:16.821 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:28:16.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:28:16.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:28:16.822 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:28:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f24440a13e0 msgr2=0x7f24440a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f24440a13e0 0x7f24440a3890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2454009fd0 tx=0x7f245400b040 comp rx=0 tx=0).stop 2023-12-14T18:28:16.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ff430 msgr2=0x7f245c196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ff430 0x7f245c196420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f244c000c00 tx=0x7f244c009580 comp rx=0 tx=0).stop 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 shutdown_connections 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f24440a13e0 0x7f24440a3890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f245c10acf0 0x7f245c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f245c0ff430 0x7f245c196420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 --2- 172.21.15.31:0/1924762046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f245c0ffd40 0x7f245c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 >> 172.21.15.31:0/1924762046 conn(0x7f245c0749c0 msgr2=0x7f245c0fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 shutdown_connections 2023-12-14T18:28:16.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:16.819+0000 7f2461633700 1 -- 172.21.15.31:0/1924762046 wait complete. 2023-12-14T18:28:16.839 INFO:tasks.cephfs.filesystem:scrub status for tag:be9ca9cc-0920-446f-96f1-151b4a537d78 - {'path': '/', 'tag': 'be9ca9cc-0920-446f-96f1-151b4a537d78', 'options': 'recursive,force'} 2023-12-14T18:28:17.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:17 smithi031 ceph-mon[42085]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-12-14T18:28:17.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:17 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:28:17.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:17 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:28:17.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:17 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:28:17.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:17 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:28:17.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:17 smithi176 ceph-mon[43506]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-12-14T18:28:17.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:17 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:28:17.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:17 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:28:17.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:17 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:28:17.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:17 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:28:17.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:17 smithi204 ceph-mon[43866]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-12-14T18:28:17.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:17 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:28:17.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:17 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:28:17.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:17 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:28:17.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:17 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:28:19.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:19 smithi031 ceph-mon[42085]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-12-14T18:28:19.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:19 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:28:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:19 smithi176 ceph-mon[43506]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-12-14T18:28:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:19 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:28:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:19 smithi204 ceph-mon[43866]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-12-14T18:28:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:19 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:28:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/2740436083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ff3c0 msgr2=0x7fa7a40ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/2740436083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a40ff790 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa7940099c0 tx=0x7fa794009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/2740436083 shutdown_connections 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/2740436083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/2740436083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a40ff790 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/2740436083 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ffd60 0x7fa7a4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/2740436083 >> 172.21.15.31:0/2740436083 conn(0x7fa7a40763a0 msgr2=0x7fa7a40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/2740436083 shutdown_connections 2023-12-14T18:28:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.165+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/2740436083 wait complete. 2023-12-14T18:28:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 Processor -- start 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 -- start start 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a4196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ffd60 0x7fa7a4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a419b2f0 con 0x7fa7a4104400 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a419b460 con 0x7fa7a40ff3c0 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.166+0000 7fa7aa0c6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a419b620 con 0x7fa7a40ffd60 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41726/0 (socket says 172.21.15.31:41726) 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 -- 172.21.15.31:0/3126204336 learned_addr learned my addr 172.21.15.31:0/3126204336 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a4196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ffd60 0x7fa7a4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 -- 172.21.15.31:0/3126204336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ffd60 msgr2=0x7fa7a4196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ffd60 0x7fa7a4196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 -- 172.21.15.31:0/3126204336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ff3c0 msgr2=0x7fa7a4196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a4196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 -- 172.21.15.31:0/3126204336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa798009520 con 0x7fa7a4104400 2023-12-14T18:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a4196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.167+0000 7fa7a3fff700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a419acb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00ef30 tx=0x7fa79c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.168+0000 7fa7a0ff9700 1 -- 172.21.15.31:0/3126204336 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c018070 con 0x7fa7a4104400 2023-12-14T18:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.168+0000 7fa7a0ff9700 1 -- 172.21.15.31:0/3126204336 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa79c00f040 con 0x7fa7a4104400 2023-12-14T18:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.168+0000 7fa7a0ff9700 1 -- 172.21.15.31:0/3126204336 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79c00cd00 con 0x7fa7a4104400 2023-12-14T18:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.168+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7940095f0 con 0x7fa7a4104400 2023-12-14T18:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.168+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a4110340 con 0x7fa7a4104400 2023-12-14T18:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.172+0000 7fa7a0ff9700 1 -- 172.21.15.31:0/3126204336 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fa79c003a00 con 0x7fa7a4104400 2023-12-14T18:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.172+0000 7fa7b6254b80 1 Processor -- start 2023-12-14T18:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.172+0000 7fa7b6254b80 1 -- start start 2023-12-14T18:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4095f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409b30 0x55f6bc3f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc3fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc2e4580 con 0x55f6bc257430 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc353810 con 0x55f6bc409b30 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc34cca0 con 0x55f6bc409fa0 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a0ff9700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa78c0a1460 0x7fa78c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a3fff700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa78c0a1460 0x7fa78c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41738/0 (socket says 172.21.15.31:41738) 2023-12-14T18:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/1912141208 learned_addr learned my addr 172.21.15.31:0/1912141208 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a0ff9700 1 -- 172.21.15.31:0/3126204336 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fa79c0ccda0 con 0x7fa7a4104400 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc3fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/1912141208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 msgr2=0x55f6bc3fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.173+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc3fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/1912141208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409b30 msgr2=0x55f6bc3f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409b30 0x55f6bc3f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409b30 0x55f6bc3f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/1912141208 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6bc1ce350 con 0x55f6bc257430 2023-12-14T18:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409b30 0x55f6bc3f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a3fff700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa78c0a1460 0x7fa78c0a3910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0ca050 tx=0x7fa79c06ca50 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4095f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa798009f30 tx=0x7fa7980099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f3373a4b90ef97b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa789ffb700 1 -- 172.21.15.31:0/1912141208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798004050 con 0x55f6bc257430 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa789ffb700 1 -- 172.21.15.31:0/1912141208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa798003680 con 0x55f6bc257430 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.174+0000 7fa789ffb700 1 -- 172.21.15.31:0/1912141208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798010730 con 0x55f6bc257430 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1912141208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 msgr2=0x55f6bc4095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4095f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa798009f30 tx=0x7fa7980099c0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1912141208 shutdown_connections 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc3fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4095f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1912141208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409b30 0x55f6bc3f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1912141208 >> 172.21.15.31:0/1912141208 conn(0x55f6bc3f0890 msgr2=0x55f6bc3fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1912141208 shutdown_connections 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.175+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1912141208 wait complete. 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 -- start start 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 Processor -- start 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc257430 0x55f6bc40fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 0x55f6bc40d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc2e8f30 con 0x55f6bc409b30 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc2e8bd0 con 0x55f6bc409fa0 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc1d0640 con 0x55f6bc257430 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.176+0000 7fa7b6254b80 10 client.0 _ll_register_callbacks cb 0x55f6bc30c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 0x55f6bc40d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 0x55f6bc40d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41750/0 (socket says 172.21.15.31:41750) 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 -- 172.21.15.31:0/1568459476 learned_addr learned my addr 172.21.15.31:0/1568459476 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc257430 0x55f6bc40fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 -- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc257430 msgr2=0x55f6bc40fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc257430 0x55f6bc40fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 -- 172.21.15.31:0/1568459476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 msgr2=0x55f6bc40d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 -- 172.21.15.31:0/1568459476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa798013460 con 0x55f6bc409b30 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 0x55f6bc40d080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7940058e0 tx=0x7fa79400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66fb84ba79d966b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa782ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.177+0000 7fa782ffd700 1 -- 172.21.15.31:0/1568459476 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79401d050 con 0x55f6bc409b30 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.178+0000 7fa782ffd700 1 -- 172.21.15.31:0/1568459476 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fa79400fcc0 con 0x55f6bc409b30 2023-12-14T18:28:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.178+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6bc3edea0 con 0x55f6bc409b30 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.178+0000 7fa782ffd700 1 -- 172.21.15.31:0/1568459476 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa79400fe80 con 0x55f6bc409b30 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.178+0000 7fa7b6254b80 10 client.15795 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.178+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6bc313110 con 0x55f6bc409b30 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.178+0000 7fa782ffd700 1 -- 172.21.15.31:0/1568459476 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6bc3edea0 con 0x55f6bc409b30 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa782ffd700 1 -- 172.21.15.31:0/1568459476 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa794003c10 con 0x55f6bc409b30 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7b6254b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7b6254b80 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7b6254b80 20 client.15795 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc41d140 0x55f6bc41f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7b6254b80 4 client.15795 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6bc345c80 con 0x55f6bc41d140 2023-12-14T18:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.179+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc41d140 0x55f6bc41f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.180+0000 7fa782ffd700 10 client.15795 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.180+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc41d140 0x55f6bc41f5b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c1bdca6e1fd401 server_cookie=3890e7a1806b5681 in_seq=0 out_seq=0 2023-12-14T18:28:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.180+0000 7fa782ffd700 1 -- 172.21.15.31:0/1568459476 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f6bc345c80 con 0x55f6bc41d140 2023-12-14T18:28:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.180+0000 7fa782ffd700 10 client.15795 handle_command_reply: tid=0 2023-12-14T18:28:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.180+0000 7fa7b6254b80 1 client.15795 shutdown 2023-12-14T18:28:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 msgr2=0x55f6bc40d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 0x55f6bc40d080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7940058e0 tx=0x7fa79400f690 comp rx=0 tx=0).stop 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 shutdown_connections 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc41d140 0x55f6bc41f5b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc257430 0x55f6bc40fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa782ffd700 0 client.15795 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/1568459476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc409b30 0x55f6bc40d080 secure :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7940058e0 tx=0x7fa79400f690 comp rx=0 tx=0).stop 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 >> 172.21.15.31:0/1568459476 conn(0x55f6bc3f0890 msgr2=0x55f6bc3f53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 shutdown_connections 2023-12-14T18:28:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.181+0000 7fa7b6254b80 1 -- 172.21.15.31:0/1568459476 wait complete. 2023-12-14T18:28:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.185+0000 7fa7b6254b80 20 client.15795 trim_cache size 0 max 16384 2023-12-14T18:28:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.214+0000 7fa7b6254b80 1 Processor -- start 2023-12-14T18:28:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 -- start start 2023-12-14T18:28:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc40fdd0 0x55f6bc3fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc40e660 con 0x55f6bc257430 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc3f6f70 con 0x55f6bc409fa0 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc41fc30 con 0x55f6bc40fdd0 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7a2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41766/0 (socket says 172.21.15.31:41766) 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2052831291 learned_addr learned my addr 172.21.15.31:0/2052831291 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.215+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc40fdd0 0x55f6bc3fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2052831291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 msgr2=0x55f6bc40f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2052831291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 msgr2=0x55f6bc4144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a2ffd700 1 -- 172.21.15.31:0/2052831291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6bc3f20d0 con 0x55f6bc40fdd0 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc40fdd0 0x55f6bc3fae10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa798013430 tx=0x7fa798013680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6833a6748d44ec3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa78a7fc700 1 -- 172.21.15.31:0/2052831291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798030570 con 0x55f6bc40fdd0 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa78a7fc700 1 -- 172.21.15.31:0/2052831291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa798005300 con 0x55f6bc40fdd0 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.216+0000 7fa78a7fc700 1 -- 172.21.15.31:0/2052831291 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7980104f0 con 0x55f6bc40fdd0 2023-12-14T18:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.217+0000 7fa7b6254b80 1 -- 172.21.15.31:0/2052831291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc40fdd0 msgr2=0x55f6bc3fae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.217+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc40fdd0 0x55f6bc3fae10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa798013430 tx=0x7fa798013680 comp rx=0 tx=0).stop 2023-12-14T18:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 -- 172.21.15.31:0/2052831291 shutdown_connections 2023-12-14T18:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc40fdd0 0x55f6bc3fae10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc257430 0x55f6bc4144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/2052831291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc409fa0 0x55f6bc40f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 -- 172.21.15.31:0/2052831291 >> 172.21.15.31:0/2052831291 conn(0x55f6bc3f0890 msgr2=0x55f6bc4072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 -- 172.21.15.31:0/2052831291 shutdown_connections 2023-12-14T18:28:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 -- 172.21.15.31:0/2052831291 wait complete. 2023-12-14T18:28:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.223+0000 7fa7b6254b80 1 -- start start 2023-12-14T18:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 Processor -- start 2023-12-14T18:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc257430 0x55f6bc412d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc413280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc40fdd0 0x55f6bc4137c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc4711a0 con 0x55f6bc40fdd0 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc3fc0e0 con 0x55f6bc257430 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bc3fc2a0 con 0x55f6bc409fa0 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7b6254b80 10 client.0 _ll_register_callbacks cb 0x55f6bc30c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc413280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7a37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc413280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:43212/0 (socket says 172.21.15.31:43212) 2023-12-14T18:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.224+0000 7fa7a37fe700 1 -- 172.21.15.31:0/4120778960 learned_addr learned my addr 172.21.15.31:0/4120778960 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7a37fe700 1 -- 172.21.15.31:0/4120778960 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc257430 msgr2=0x55f6bc412d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc257430 0x55f6bc412d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7a37fe700 1 -- 172.21.15.31:0/4120778960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc40fdd0 msgr2=0x55f6bc4137c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc40fdd0 0x55f6bc4137c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7a37fe700 1 -- 172.21.15.31:0/4120778960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6bc1ce350 con 0x55f6bc409fa0 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7a37fe700 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc413280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a010 tx=0x7fa794009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85b7dcbdbd52f63 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa782ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa782ffd700 1 -- 172.21.15.31:0/4120778960 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79401d030 con 0x55f6bc409fa0 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa782ffd700 1 -- 172.21.15.31:0/4120778960 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fa794000e40 con 0x55f6bc409fa0 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.225+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa798003680 con 0x55f6bc409fa0 2023-12-14T18:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa782ffd700 1 -- 172.21.15.31:0/4120778960 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa794005bc0 con 0x55f6bc409fa0 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 10 client.25654 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6bc17ab00 con 0x55f6bc409fa0 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa782ffd700 1 -- 172.21.15.31:0/4120778960 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa798003680 con 0x55f6bc409fa0 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa782ffd700 1 -- 172.21.15.31:0/4120778960 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa794003b00 con 0x55f6bc409fa0 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 10 client.25654 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 20 client.25654 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc3fd590 0x55f6bc3ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 4 client.25654 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.226+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f6bc3459a0 con 0x55f6bc3fd590 2023-12-14T18:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.227+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc3fd590 0x55f6bc3ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.227+0000 7fa7a2ffd700 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc3fd590 0x55f6bc3ff9a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cac4f8dd6a16eb server_cookie=be5ce0008d7c13bf in_seq=0 out_seq=0 2023-12-14T18:28:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.227+0000 7fa782ffd700 10 client.25654 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.233+0000 7fa782ffd700 1 -- 172.21.15.31:0/4120778960 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f6bc3459a0 con 0x55f6bc3fd590 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.233+0000 7fa782ffd700 10 client.25654 handle_command_reply: tid=0 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.233+0000 7fa7b6254b80 1 client.25654 shutdown 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 msgr2=0x55f6bc413280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc413280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa79400a010 tx=0x7fa794009310 comp rx=0 tx=0).stop 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 shutdown_connections 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f6bc3fd590 0x55f6bc3ff9a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f6bc40fdd0 0x55f6bc4137c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa782ffd700 0 client.25654 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f6bc257430 0x55f6bc412d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 --2- 172.21.15.31:0/4120778960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f6bc409fa0 0x55f6bc413280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 >> 172.21.15.31:0/4120778960 conn(0x55f6bc3f0890 msgr2=0x55f6bc3fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 shutdown_connections 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 1 -- 172.21.15.31:0/4120778960 wait complete. 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.234+0000 7fa7b6254b80 20 client.25654 trim_cache size 0 max 16384 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:28:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa78c0a1460 msgr2=0x7fa78c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa78c0a1460 0x7fa78c0a3910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa79c0ca050 tx=0x7fa79c06ca50 comp rx=0 tx=0).stop 2023-12-14T18:28:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 msgr2=0x7fa7a419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a419acb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa79c00ef30 tx=0x7fa79c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 shutdown_connections 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4104400 0x7fa7a419acb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa7a40ff3c0 0x7fa7a4196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa78c0a1460 0x7fa78c0a3910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 --2- 172.21.15.31:0/3126204336 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a40ffd60 0x7fa7a4196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 >> 172.21.15.31:0/3126204336 conn(0x7fa7a40763a0 msgr2=0x7fa7a40fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.235+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 shutdown_connections 2023-12-14T18:28:21.239 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:21.236+0000 7fa7aa0c6700 1 -- 172.21.15.31:0/3126204336 wait complete. 2023-12-14T18:28:21.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:21 smithi031 ceph-mon[42085]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2023-12-14T18:28:21.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:21 smithi176 ceph-mon[43506]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2023-12-14T18:28:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:21 smithi204 ceph-mon[43866]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.8 KiB/s wr, 13 op/s 2023-12-14T18:28:23.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:23 smithi031 ceph-mon[42085]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2023-12-14T18:28:23.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:23 smithi176 ceph-mon[43506]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2023-12-14T18:28:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:23 smithi204 ceph-mon[43866]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2023-12-14T18:28:25.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:25 smithi031 ceph-mon[42085]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-14T18:28:25.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:25 smithi176 ceph-mon[43506]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-14T18:28:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:25 smithi204 ceph-mon[43866]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-14T18:28:27.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:27 smithi031 ceph-mon[42085]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-14T18:28:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:27 smithi176 ceph-mon[43506]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-14T18:28:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:27 smithi204 ceph-mon[43866]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-14T18:28:29.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:29 smithi031 ceph-mon[42085]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-14T18:28:29.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:29 smithi176 ceph-mon[43506]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-14T18:28:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:29 smithi204 ceph-mon[43866]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-12-14T18:28:31.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:31 smithi031 ceph-mon[42085]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-14T18:28:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:31 smithi176 ceph-mon[43506]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-14T18:28:31.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:31 smithi204 ceph-mon[43866]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 9 op/s 2023-12-14T18:28:33.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:33 smithi031 ceph-mon[42085]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:28:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:33 smithi176 ceph-mon[43506]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:28:33.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:33 smithi204 ceph-mon[43866]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2023-12-14T18:28:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 -- 172.21.15.31:0/3132121215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 msgr2=0x7f8a2410c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 --2- 172.21.15.31:0/3132121215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2410c030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00b0d0 tx=0x7f8a1c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 -- 172.21.15.31:0/3132121215 shutdown_connections 2023-12-14T18:28:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 --2- 172.21.15.31:0/3132121215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2410c030 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 --2- 172.21.15.31:0/3132121215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 0x7f8a240ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 --2- 172.21.15.31:0/3132121215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 0x7f8a24103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 -- 172.21.15.31:0/3132121215 >> 172.21.15.31:0/3132121215 conn(0x7f8a24075fc0 msgr2=0x7f8a240763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 -- 172.21.15.31:0/3132121215 shutdown_connections 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.405+0000 7f8a2b744700 1 -- 172.21.15.31:0/3132121215 wait complete. 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 Processor -- start 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 -- start start 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 0x7f8a241964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 0x7f8a24196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419b4c0 con 0x7f8a24104460 2023-12-14T18:28:34.411 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419b630 con 0x7f8a240ffdc0 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a2b744700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419b7f0 con 0x7f8a240ff420 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a29ce1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 0x7f8a241964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a29ce1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54620/0 (socket says 172.21.15.31:54620) 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a29ce1700 1 -- 172.21.15.31:0/602029886 learned_addr learned my addr 172.21.15.31:0/602029886 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.406+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 0x7f8a24196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a294e0700 1 -- 172.21.15.31:0/602029886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 msgr2=0x7f8a24196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a294e0700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 0x7f8a24196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a294e0700 1 -- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 msgr2=0x7f8a2419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a294e0700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a294e0700 1 -- 172.21.15.31:0/602029886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c009d00 con 0x7f8a240ff420 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 0x7f8a24196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a294e0700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 0x7f8a241964e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000be10 tx=0x7f8a2000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a1a7fc700 1 -- 172.21.15.31:0/602029886 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20012c90 con 0x7f8a240ff420 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a1a7fc700 1 -- 172.21.15.31:0/602029886 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a20016070 con 0x7f8a240ff420 2023-12-14T18:28:34.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a1a7fc700 1 -- 172.21.15.31:0/602029886 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2001b6a0 con 0x7f8a240ff420 2023-12-14T18:28:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a241103d0 con 0x7f8a240ff420 2023-12-14T18:28:34.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.407+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a24110240 con 0x7f8a240ff420 2023-12-14T18:28:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.410+0000 7f8a1a7fc700 1 -- 172.21.15.31:0/602029886 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f8a2001c950 con 0x7f8a240ff420 2023-12-14T18:28:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.411+0000 7f8a1a7fc700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8a100a1460 0x7f8a100a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.411+0000 7f8a1a7fc700 1 -- 172.21.15.31:0/602029886 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f8a200d0970 con 0x7f8a240ff420 2023-12-14T18:28:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 Processor -- start 2023-12-14T18:28:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8a100a1460 0x7f8a100a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 -- start start 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd1a7430 0x55d5cd349e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd34a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd234580 con 0x55d5cd1a7430 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd2a3810 con 0x55d5cd34a370 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd29cca0 con 0x55d5cd35d900 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd1a7430 0x55d5cd349e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.412+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd1a7430 0x55d5cd349e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54628/0 (socket says 172.21.15.31:54628) 2023-12-14T18:28:34.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a294e0700 1 -- 172.21.15.31:0/2819684961 learned_addr learned my addr 172.21.15.31:0/2819684961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd34a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 -- 172.21.15.31:0/2819684961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 msgr2=0x55d5cd34a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd34a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 -- 172.21.15.31:0/2819684961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd1a7430 msgr2=0x55d5cd349e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd1a7430 0x55d5cd349e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 -- 172.21.15.31:0/2819684961 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5cd11e350 con 0x55d5cd35d900 2023-12-14T18:28:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd34a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8a100a1460 0x7f8a100a3910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a24197af0 tx=0x7f8a14013bf0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.413+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35fce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009c80 tx=0x7f8a1c005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d753fa9cb78835bf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a0f7fe700 1 -- 172.21.15.31:0/2819684961 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c00e040 con 0x55d5cd35d900 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a0f7fe700 1 -- 172.21.15.31:0/2819684961 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1c0074e0 con 0x55d5cd35d900 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a0f7fe700 1 -- 172.21.15.31:0/2819684961 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0050d0 con 0x55d5cd35d900 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a378d2b80 1 -- 172.21.15.31:0/2819684961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 msgr2=0x55d5cd35fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35fce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009c80 tx=0x7f8a1c005a80 comp rx=0 tx=0).stop 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a378d2b80 1 -- 172.21.15.31:0/2819684961 shutdown_connections 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35fce0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.415+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd1a7430 0x55d5cd349e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.416+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/2819684961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd34a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.416+0000 7f8a378d2b80 1 -- 172.21.15.31:0/2819684961 >> 172.21.15.31:0/2819684961 conn(0x55d5cd340890 msgr2=0x55d5cd346200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.416+0000 7f8a378d2b80 1 -- 172.21.15.31:0/2819684961 shutdown_connections 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.416+0000 7f8a378d2b80 1 -- 172.21.15.31:0/2819684961 wait complete. 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.416+0000 7f8a378d2b80 1 -- start start 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.416+0000 7f8a378d2b80 1 Processor -- start 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd35d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd35a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd35d900 0x55d5cd35aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd238f30 con 0x55d5cd35d900 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd238bd0 con 0x55d5cd34a370 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd120640 con 0x55d5cd1a7430 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a378d2b80 10 client.0 _ll_register_callbacks cb 0x55d5cd25c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:34.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd35d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd35d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:57666/0 (socket says 172.21.15.31:57666) 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a294e0700 1 -- 172.21.15.31:0/1961814708 learned_addr learned my addr 172.21.15.31:0/1961814708 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd35d900 0x55d5cd35aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.417+0000 7f8a294e0700 1 -- 172.21.15.31:0/1961814708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 msgr2=0x55d5cd35a520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a294e0700 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd35a520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a294e0700 1 -- 172.21.15.31:0/1961814708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd35d900 msgr2=0x55d5cd35aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a294e0700 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd35d900 0x55d5cd35aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a294e0700 1 -- 172.21.15.31:0/1961814708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a24068d70 con 0x55d5cd1a7430 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a294e0700 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd35d200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a2009d570 tx=0x7f8a2009d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e32c90d57c8e22 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a0cff9700 1 -- 172.21.15.31:0/1961814708 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200880b0 con 0x55d5cd1a7430 2023-12-14T18:28:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a0cff9700 1 -- 172.21.15.31:0/1961814708 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f8a20088210 con 0x55d5cd1a7430 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5cd33dea0 con 0x55d5cd1a7430 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a0cff9700 1 -- 172.21.15.31:0/1961814708 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a2009d840 con 0x55d5cd1a7430 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a378d2b80 10 client.25666 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.418+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5cd263110 con 0x55d5cd1a7430 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a0cff9700 1 -- 172.21.15.31:0/1961814708 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5cd33dea0 con 0x55d5cd1a7430 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a0cff9700 1 -- 172.21.15.31:0/1961814708 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a200a55a0 con 0x55d5cd1a7430 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a378d2b80 10 client.25666 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a378d2b80 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a378d2b80 20 client.25666 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd37e3b0 0x55d5cd3807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a378d2b80 4 client.25666 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5cd295c80 con 0x55d5cd37e3b0 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.419+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd37e3b0 0x55d5cd3807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.420+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd37e3b0 0x55d5cd3807c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9154c3a2629f1517 server_cookie=d6a712130a8e5f29 in_seq=0 out_seq=0 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.420+0000 7f8a0cff9700 10 client.25666 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.420+0000 7f8a0cff9700 1 -- 172.21.15.31:0/1961814708 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d5cd295c80 con 0x55d5cd37e3b0 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.420+0000 7f8a0cff9700 10 client.25666 handle_command_reply: tid=0 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.420+0000 7f8a378d2b80 1 client.25666 shutdown 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 msgr2=0x55d5cd35d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd35d200 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a2009d570 tx=0x7f8a2009d070 comp rx=0 tx=0).stop 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 shutdown_connections 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd37e3b0 0x55d5cd3807c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd35d900 0x55d5cd35aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd35d200 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a0cff9700 0 client.25666 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/1961814708 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd34a370 0x55d5cd35a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 >> 172.21.15.31:0/1961814708 conn(0x55d5cd340890 msgr2=0x55d5cd345df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 shutdown_connections 2023-12-14T18:28:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 1 -- 172.21.15.31:0/1961814708 wait complete. 2023-12-14T18:28:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.421+0000 7f8a378d2b80 20 client.25666 trim_cache size 0 max 16384 2023-12-14T18:28:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 Processor -- start 2023-12-14T18:28:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 -- start start 2023-12-14T18:28:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 0x55d5cd35cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd359ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd35bb00 con 0x55d5cd34a370 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd34b8c0 con 0x55d5cd1a7430 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.437+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd369380 con 0x55d5cd35d900 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 0x55d5cd35cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a28cdf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd359ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 0x55d5cd35cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:40582/0 (socket says 172.21.15.31:40582) 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 -- 172.21.15.31:0/4277063937 learned_addr learned my addr 172.21.15.31:0/4277063937 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a28cdf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd359ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54644/0 (socket says 172.21.15.31:54644) 2023-12-14T18:28:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 -- 172.21.15.31:0/4277063937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 msgr2=0x55d5cd35a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 -- 172.21.15.31:0/4277063937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 msgr2=0x55d5cd359ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd359ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 -- 172.21.15.31:0/4277063937 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5cd262e60 con 0x55d5cd1a7430 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.438+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 0x55d5cd35cd30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a2009b930 tx=0x7f8a2001cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2c20e8659b48dd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a0ffff700 1 -- 172.21.15.31:0/4277063937 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2009d3b0 con 0x55d5cd1a7430 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a0ffff700 1 -- 172.21.15.31:0/4277063937 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a20012100 con 0x55d5cd1a7430 2023-12-14T18:28:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a0ffff700 1 -- 172.21.15.31:0/4277063937 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200a55a0 con 0x55d5cd1a7430 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4277063937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 msgr2=0x55d5cd35cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 0x55d5cd35cd30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a2009b930 tx=0x7f8a2001cd50 comp rx=0 tx=0).stop 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4277063937 shutdown_connections 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd35d900 0x55d5cd35a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd1a7430 0x55d5cd35cd30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4277063937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd359ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.439+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4277063937 >> 172.21.15.31:0/4277063937 conn(0x55d5cd340890 msgr2=0x55d5cd368c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4277063937 shutdown_connections 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4277063937 wait complete. 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 -- start start 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 Processor -- start 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd3636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd363c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 0x55d5cd3d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd364540 con 0x55d5cd34a370 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd3d17b0 con 0x55d5cd35d900 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.440+0000 7f8a378d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5cd34d090 con 0x55d5cd1a7430 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a378d2b80 10 client.0 _ll_register_callbacks cb 0x55d5cd25c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 0x55d5cd3d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 0x55d5cd3d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:40594/0 (socket says 172.21.15.31:40594) 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 -- 172.21.15.31:0/4029549148 learned_addr learned my addr 172.21.15.31:0/4029549148 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd3636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd363c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 -- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 msgr2=0x55d5cd3636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd3636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 -- 172.21.15.31:0/4029549148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 msgr2=0x55d5cd363c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd363c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a29ce1700 1 -- 172.21.15.31:0/4029549148 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5cd263110 con 0x55d5cd35d900 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a28cdf700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd363c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.441+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd3636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.442+0000 7f8a29ce1700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 0x55d5cd3d1270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005af0 tx=0x7f8a1c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb38e88bbd76b04c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.442+0000 7f8a0cff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.442+0000 7f8a0cff9700 1 -- 172.21.15.31:0/4029549148 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c02b070 con 0x55d5cd35d900 2023-12-14T18:28:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.442+0000 7f8a0cff9700 1 -- 172.21.15.31:0/4029549148 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f8a1c00bdf0 con 0x55d5cd35d900 2023-12-14T18:28:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.442+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a1c0074e0 con 0x55d5cd35d900 2023-12-14T18:28:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a0cff9700 1 -- 172.21.15.31:0/4029549148 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1c00e040 con 0x55d5cd35d900 2023-12-14T18:28:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 10 client.25664 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d5cd249940 con 0x55d5cd35d900 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a0cff9700 1 -- 172.21.15.31:0/4029549148 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a1c0074e0 con 0x55d5cd35d900 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a0cff9700 1 -- 172.21.15.31:0/4029549148 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a1c00b100 con 0x55d5cd35d900 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 10 client.25664 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 10 client.25664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 20 client.25664 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd34e420 0x55d5cd350800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 4 client.25664 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5cd2959a0 con 0x55d5cd34e420 2023-12-14T18:28:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.443+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd34e420 0x55d5cd350800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.444+0000 7f8a0cff9700 10 client.25664 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.444+0000 7f8a294e0700 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd34e420 0x55d5cd350800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d1bd3cc478462f server_cookie=c51fc89d3edbaf12 in_seq=0 out_seq=0 2023-12-14T18:28:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a0cff9700 1 -- 172.21.15.31:0/4029549148 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5cd2959a0 con 0x55d5cd34e420 2023-12-14T18:28:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a0cff9700 10 client.25664 handle_command_reply: tid=0 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 client.25664 shutdown 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 msgr2=0x55d5cd3d1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 0x55d5cd3d1270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005af0 tx=0x7f8a1c00b760 comp rx=0 tx=0).stop 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 shutdown_connections 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d5cd34e420 0x55d5cd350800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d5cd35d900 0x55d5cd3d1270 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5cd1a7430 0x55d5cd3636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 --2- 172.21.15.31:0/4029549148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d5cd34a370 0x55d5cd363c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 >> 172.21.15.31:0/4029549148 conn(0x55d5cd340890 msgr2=0x55d5cd34bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a0cff9700 0 client.25664 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 shutdown_connections 2023-12-14T18:28:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 1 -- 172.21.15.31:0/4029549148 wait complete. 2023-12-14T18:28:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.452+0000 7f8a378d2b80 20 client.25664 trim_cache size 0 max 16384 2023-12-14T18:28:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:28:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:28:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:28:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8a100a1460 msgr2=0x7f8a100a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8a100a1460 0x7f8a100a3910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8a24197af0 tx=0x7f8a14013bf0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 msgr2=0x7f8a241964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 0x7f8a241964e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000be10 tx=0x7f8a2000dc30 comp rx=0 tx=0).stop 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 shutdown_connections 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a24104460 0x7f8a2419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8a100a1460 0x7f8a100a3910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a240ff420 0x7f8a241964e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 --2- 172.21.15.31:0/602029886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a240ffdc0 0x7f8a24196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 >> 172.21.15.31:0/602029886 conn(0x7f8a24075fc0 msgr2=0x7f8a24102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 shutdown_connections 2023-12-14T18:28:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:34.453+0000 7f8a2b744700 1 -- 172.21.15.31:0/602029886 wait complete. 2023-12-14T18:28:35.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:35 smithi031 ceph-mon[42085]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-12-14T18:28:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:35 smithi176 ceph-mon[43506]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-12-14T18:28:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:35 smithi204 ceph-mon[43866]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-12-14T18:28:37.521 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 95% (11492/12072) Updating files: 96% (11590/12072) Updating files: 97% (11710/12072) Updating files: 98% (11831/12072) Updating files: 99% (11952/12072) Updating files: 100% (12072/12072) Updating files: 100% (12072/12072), done. 2023-12-14T18:28:37.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:37 smithi031 ceph-mon[42085]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-12-14T18:28:37.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:37 smithi176 ceph-mon[43506]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-12-14T18:28:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:37 smithi204 ceph-mon[43866]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to 'bf026da9d69000f3d4a9d16360156095d6612a2d'. 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.269 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr: 2023-12-14T18:28:38.270 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at bf026da9d69 doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-12-14T18:28:38.272 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-14T18:28:38.298 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-14T18:28:38.300 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-14T18:28:38.300 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-14T18:28:38.862 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-14T18:28:38.963 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-14T18:28:39.006 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-14T18:28:39.008 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-14T18:28:39.008 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-14T18:28:39.049 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-14T18:28:39.053 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:28:39.053 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-14T18:28:39.115 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-12-14T18:28:39.116 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-12-14T18:28:39.116 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-12-14T18:28:39.183 INFO:tasks.workunit.client.0.smithi031.stderr:+ set -e 2023-12-14T18:28:39.184 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p testdir 2023-12-14T18:28:39.186 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd testdir 2023-12-14T18:28:39.186 INFO:tasks.workunit.client.0.smithi031.stderr:+ set +e 2023-12-14T18:28:39.186 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfacl -d -m u:nobody:rw . 2023-12-14T18:28:39.193 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.193 INFO:tasks.workunit.client.0.smithi031.stderr:+ set -e 2023-12-14T18:28:39.193 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=0 2023-12-14T18:28:39.193 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 0 -lt 100 ']' 2023-12-14T18:28:39.193 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 0 + 1 2023-12-14T18:28:39.194 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=1 2023-12-14T18:28:39.194 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:39.196 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:39.196 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:39.198 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:39.198 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:39.198 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:39.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:39 smithi031 ceph-mon[42085]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-12-14T18:28:39.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:39 smithi176 ceph-mon[43506]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-12-14T18:28:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:39 smithi204 ceph-mon[43866]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2023-12-14T18:28:40.064 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:40.065 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:40.260 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:40.260 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:40.269 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:40.269 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:40.269 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 1 -lt 100 ']' 2023-12-14T18:28:40.269 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 1 + 1 2023-12-14T18:28:40.301 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=2 2023-12-14T18:28:40.301 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:40.328 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:40.329 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:40.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:40.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:40.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:40.864 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:40.864 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:40.993 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:40.993 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:41.001 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:41.002 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:41.002 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -lt 100 ']' 2023-12-14T18:28:41.002 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 2 + 1 2023-12-14T18:28:41.021 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=3 2023-12-14T18:28:41.022 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:41.041 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:41.041 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:41.043 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:41.043 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:41.043 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:41.514 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:41.514 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:41.651 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:41.652 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:41.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:41.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:41.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 3 -lt 100 ']' 2023-12-14T18:28:41.661 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 3 + 1 2023-12-14T18:28:41.693 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=4 2023-12-14T18:28:41.693 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:41.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:41 smithi031 ceph-mon[42085]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-12-14T18:28:41.722 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:41.722 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:41.723 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:41.724 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:41.724 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:41.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:41 smithi176 ceph-mon[43506]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-12-14T18:28:41.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:41 smithi204 ceph-mon[43866]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2023-12-14T18:28:42.138 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:42.139 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:42.268 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:42.268 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:42.276 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:42.276 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:42.276 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 4 -lt 100 ']' 2023-12-14T18:28:42.276 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 4 + 1 2023-12-14T18:28:42.301 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=5 2023-12-14T18:28:42.301 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:42.327 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:42.327 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:42.328 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:42.329 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:42.329 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:42.772 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:42.772 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:42.943 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:42.943 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:42.952 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:42.952 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:42.952 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 5 -lt 100 ']' 2023-12-14T18:28:42.952 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 5 + 1 2023-12-14T18:28:42.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=6 2023-12-14T18:28:42.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:43.018 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:43.018 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:43.020 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:43.020 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:43.020 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:43.489 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:43.489 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:43.618 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:43.618 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:43.626 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:43.626 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:43.626 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 6 -lt 100 ']' 2023-12-14T18:28:43.626 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 6 + 1 2023-12-14T18:28:43.651 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=7 2023-12-14T18:28:43.651 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:43.674 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:43.674 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:43.676 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:43.676 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:43.676 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:43.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:43 smithi031 ceph-mon[42085]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2023-12-14T18:28:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:43 smithi176 ceph-mon[43506]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2023-12-14T18:28:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:43 smithi204 ceph-mon[43866]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2023-12-14T18:28:44.030 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:44.030 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:44.184 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:44.184 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:44.195 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:44.195 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:44.195 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 7 -lt 100 ']' 2023-12-14T18:28:44.195 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 7 + 1 2023-12-14T18:28:44.225 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=8 2023-12-14T18:28:44.226 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:44.265 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:44.266 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:44.267 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:44.267 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:44.267 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:45.547 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:45.547 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:45.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:45 smithi031 ceph-mon[42085]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2023-12-14T18:28:45.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:45 smithi176 ceph-mon[43506]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2023-12-14T18:28:45.759 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:45.759 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:45.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:45.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:45.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 8 -lt 100 ']' 2023-12-14T18:28:45.771 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 8 + 1 2023-12-14T18:28:45.789 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=9 2023-12-14T18:28:45.789 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:45.824 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:45.824 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:45.825 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:45.826 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:45.826 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:45 smithi204 ceph-mon[43866]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 4.1 KiB/s wr, 10 op/s 2023-12-14T18:28:46.546 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:46.547 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:46.717 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:46.717 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:46.726 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:46.726 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:46.727 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 9 -lt 100 ']' 2023-12-14T18:28:46.727 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 9 + 1 2023-12-14T18:28:46.759 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=10 2023-12-14T18:28:46.759 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:46.784 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:46.784 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:46.785 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:46.785 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:46.785 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:46.840 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:28:47.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:47 smithi031 ceph-mon[42085]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:28:47.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:47 smithi176 ceph-mon[43506]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:28:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:47 smithi204 ceph-mon[43866]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:28:47.996 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:47.996 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:48.149 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:48.149 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:48.158 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:48.158 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:48.158 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 10 -lt 100 ']' 2023-12-14T18:28:48.159 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 10 + 1 2023-12-14T18:28:48.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=11 2023-12-14T18:28:48.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:48.284 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:48.284 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:48.286 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:48.286 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:48.286 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:49.004 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:49.005 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:49.192 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:49.192 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:49.211 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:49.211 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:49.211 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 11 -lt 100 ']' 2023-12-14T18:28:49.211 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 11 + 1 2023-12-14T18:28:49.250 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=12 2023-12-14T18:28:49.250 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:49.296 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:49.296 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:49.298 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:49.298 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:49.298 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:49.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 -- 172.21.15.31:0/1078828622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2088104410 msgr2=0x7f208810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 --2- 172.21.15.31:0/1078828622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2088104410 0x7f208810bfe0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f208000b0d0 tx=0x7f208000b3e0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 -- 172.21.15.31:0/2690263455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 msgr2=0x7fc214102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 --2- 172.21.15.31:0/2690263455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc214102960 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009990 tx=0x7fc1fc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 -- 172.21.15.31:0/1078828622 shutdown_connections 2023-12-14T18:28:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 --2- 172.21.15.31:0/1078828622 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2088104410 0x7f208810bfe0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 --2- 172.21.15.31:0/1078828622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20880ff3d0 0x7f20880ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 --2- 172.21.15.31:0/1078828622 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ffd70 0x7f2088103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 -- 172.21.15.31:0/1078828622 >> 172.21.15.31:0/1078828622 conn(0x7f2088076230 msgr2=0x7f2088076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 -- 172.21.15.31:0/2690263455 shutdown_connections 2023-12-14T18:28:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 --2- 172.21.15.31:0/2690263455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214102ea0 0x7fc21410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 --2- 172.21.15.31:0/2690263455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214108500 0x7fc2141088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 --2- 172.21.15.31:0/2690263455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc214102960 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 -- 172.21.15.31:0/2690263455 >> 172.21.15.31:0/2690263455 conn(0x7fc2140fe030 msgr2=0x7fc214100440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 -- 172.21.15.31:0/1078828622 shutdown_connections 2023-12-14T18:28:49.623 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 -- 172.21.15.31:0/2690263455 shutdown_connections 2023-12-14T18:28:49.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.618+0000 7f208d703700 1 -- 172.21.15.31:0/1078828622 wait complete. 2023-12-14T18:28:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.618+0000 7fc2193d1700 1 -- 172.21.15.31:0/2690263455 wait complete. 2023-12-14T18:28:49.624 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.619+0000 7fc2193d1700 1 Processor -- start 2023-12-14T18:28:49.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.619+0000 7f208d703700 1 Processor -- start 2023-12-14T18:28:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 -- start start 2023-12-14T18:28:49.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 -- start start 2023-12-14T18:28:49.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ff3d0 0x7f2088198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20880ffd70 0x7f2088198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2088104410 0x7f208819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc2141985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214102ea0 0x7fc214198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214108500 0x7fc21419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419d5a0 con 0x7fc2141024f0 2023-12-14T18:28:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419d710 con 0x7fc214102ea0 2023-12-14T18:28:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2193d1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419d8d0 con 0x7fc214108500 2023-12-14T18:28:49.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208819c5f0 con 0x7f20880ffd70 2023-12-14T18:28:49.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208819c760 con 0x7f20880ff3d0 2023-12-14T18:28:49.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f208d703700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208819c920 con 0x7f2088104410 2023-12-14T18:28:49.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f20867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20880ffd70 0x7f2088198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ff3d0 0x7f2088198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc212ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc2141985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc212ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc2141985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36802/0 (socket says 172.21.15.31:36802) 2023-12-14T18:28:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc212ffd700 1 -- 172.21.15.31:0/3039894141 learned_addr learned my addr 172.21.15.31:0/3039894141 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214102ea0 0x7fc214198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f20867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20880ffd70 0x7f2088198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36794/0 (socket says 172.21.15.31:36794) 2023-12-14T18:28:49.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f20867fc700 1 -- 172.21.15.31:0/1592374384 learned_addr learned my addr 172.21.15.31:0/1592374384 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.620+0000 7f20877fe700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2088104410 0x7f208819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.620+0000 7fc2137fe700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214108500 0x7fc21419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f20877fe700 1 -- 172.21.15.31:0/1592374384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ff3d0 msgr2=0x7f2088198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f20877fe700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ff3d0 0x7f2088198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f20877fe700 1 -- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20880ffd70 msgr2=0x7f2088198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f20877fe700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20880ffd70 0x7f2088198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f20877fe700 1 -- 172.21.15.31:0/1592374384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009520 con 0x7f2088104410 2023-12-14T18:28:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ff3d0 0x7f2088198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f20877fe700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2088104410 0x7f208819bff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2080009fd0 tx=0x7f2080007860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc20a7fc700 1 -- 172.21.15.31:0/3039894141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214108500 msgr2=0x7fc21419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214108500 0x7fc21419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc20a7fc700 1 -- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 msgr2=0x7fc2141985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc2141985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc20a7fc700 1 -- 172.21.15.31:0/3039894141 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c009c40 con 0x7fc214102ea0 2023-12-14T18:28:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc2137fe700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214108500 0x7fc21419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.631 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214102ea0 0x7fc214198b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0059d0 tx=0x7fc1fc005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f206ffff700 1 -- 172.21.15.31:0/1592374384 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208000e040 con 0x7f2088104410 2023-12-14T18:28:49.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f206ffff700 1 -- 172.21.15.31:0/1592374384 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208000bcb0 con 0x7f2088104410 2023-12-14T18:28:49.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080009d00 con 0x7f2088104410 2023-12-14T18:28:49.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f206ffff700 1 -- 172.21.15.31:0/1592374384 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800058b0 con 0x7f2088104410 2023-12-14T18:28:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc210ff9700 1 -- 172.21.15.31:0/3039894141 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc01d070 con 0x7fc214102ea0 2023-12-14T18:28:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc210ff9700 1 -- 172.21.15.31:0/3039894141 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1fc022470 con 0x7fc214102ea0 2023-12-14T18:28:49.632 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.621+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc0095f0 con 0x7fc214102ea0 2023-12-14T18:28:49.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.621+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208810e190 con 0x7f2088104410 2023-12-14T18:28:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.622+0000 7fc210ff9700 1 -- 172.21.15.31:0/3039894141 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc017650 con 0x7fc214102ea0 2023-12-14T18:28:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.622+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2141124f0 con 0x7fc214102ea0 2023-12-14T18:28:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.624+0000 7f206ffff700 1 -- 172.21.15.31:0/1592374384 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f2080004a50 con 0x7f2088104410 2023-12-14T18:28:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.624+0000 7f206ffff700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f20700a1440 0x7f20700a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.625+0000 7f206ffff700 1 -- 172.21.15.31:0/1592374384 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f20800ce160 con 0x7f2088104410 2023-12-14T18:28:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.625+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f20700a1440 0x7f20700a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.625+0000 7fc210ff9700 1 -- 172.21.15.31:0/3039894141 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fc1fc00fad0 con 0x7fc214102ea0 2023-12-14T18:28:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.625+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f20700a1440 0x7f20700a38f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2078009fd0 tx=0x7f2078009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.626+0000 7fc210ff9700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc2000a1460 0x7fc2000a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.626+0000 7fc212ffd700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc2000a1460 0x7fc2000a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.626+0000 7fc210ff9700 1 -- 172.21.15.31:0/3039894141 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fc1fc0ce3c0 con 0x7fc214102ea0 2023-12-14T18:28:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.627+0000 7fc212ffd700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc2000a1460 0x7fc2000a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc204009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.697 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.695+0000 7fc22555fb80 1 Processor -- start 2023-12-14T18:28:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.695+0000 7f2099891b80 1 Processor -- start 2023-12-14T18:28:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.695+0000 7fc22555fb80 1 -- start start 2023-12-14T18:28:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.695+0000 7f2099891b80 1 -- start start 2023-12-14T18:28:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.695+0000 7f2099891b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 0x558403103750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.695+0000 7f2099891b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x55840310e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2099891b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x558403110f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2099891b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402fed5e0 con 0x558403103380 2023-12-14T18:28:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2099891b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840305c870 con 0x55840310eb50 2023-12-14T18:28:49.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2099891b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558403055d00 con 0x558403103c90 2023-12-14T18:28:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d6de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d6e370 0x560218d6e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d790f0 0x560218d7b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc22555fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218c58580 con 0x560218d6e370 2023-12-14T18:28:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc22555fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc7810 con 0x560218d790f0 2023-12-14T18:28:49.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc22555fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218cc0ca0 con 0x560218bcb430 2023-12-14T18:28:49.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 0x558403103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 0x558403103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36814/0 (socket says 172.21.15.31:36814) 2023-12-14T18:28:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f2086ffd700 1 -- 172.21.15.31:0/4111529407 learned_addr learned my addr 172.21.15.31:0/4111529407 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f20867fc700 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x55840310e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc212ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d6de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc212ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d6de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:58584/0 (socket says 172.21.15.31:58584) 2023-12-14T18:28:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc212ffd700 1 -- 172.21.15.31:0/2516239586 learned_addr learned my addr 172.21.15.31:0/2516239586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc2137fe700 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d790f0 0x560218d7b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.702 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.696+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d6e370 0x560218d6e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.696+0000 7f20877fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x558403110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc212ffd700 1 -- 172.21.15.31:0/2516239586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d790f0 msgr2=0x560218d7b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc212ffd700 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d790f0 0x560218d7b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f2086ffd700 1 -- 172.21.15.31:0/4111529407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 msgr2=0x55840310e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f2086ffd700 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x55840310e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f2086ffd700 1 -- 172.21.15.31:0/4111529407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 msgr2=0x558403110f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.703 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc212ffd700 1 -- 172.21.15.31:0/2516239586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d6e370 msgr2=0x560218d6e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f2086ffd700 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x558403110f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f2086ffd700 1 -- 172.21.15.31:0/4111529407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558402ed73b0 con 0x558403103380 2023-12-14T18:28:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc212ffd700 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d6e370 0x560218d6e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.704 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc212ffd700 1 -- 172.21.15.31:0/2516239586 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560218b42350 con 0x560218bcb430 2023-12-14T18:28:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc2137fe700 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d790f0 0x560218d7b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f20867fc700 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x55840310e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f2086ffd700 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 0x558403103750 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2078012fd0 tx=0x7f2078014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d743315f9c7f70 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc212ffd700 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d6de30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e4b0 tx=0x7fc204014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca16b9ac0b980f80 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f206d7fa700 1 -- 172.21.15.31:0/4111529407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207801b600 con 0x558403103380 2023-12-14T18:28:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc2097fa700 1 -- 172.21.15.31:0/2516239586 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20400ec50 con 0x560218bcb430 2023-12-14T18:28:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc2097fa700 1 -- 172.21.15.31:0/2516239586 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc204021070 con 0x560218bcb430 2023-12-14T18:28:49.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.697+0000 7fc2097fa700 1 -- 172.21.15.31:0/2516239586 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20401c670 con 0x560218bcb430 2023-12-14T18:28:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f206d7fa700 1 -- 172.21.15.31:0/4111529407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f207800ee40 con 0x558403103380 2023-12-14T18:28:49.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.697+0000 7f206d7fa700 1 -- 172.21.15.31:0/4111529407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078026b20 con 0x558403103380 2023-12-14T18:28:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 -- 172.21.15.31:0/2516239586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 msgr2=0x560218d6de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.707 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d6de30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e4b0 tx=0x7fc204014000 comp rx=0 tx=0).stop 2023-12-14T18:28:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 -- 172.21.15.31:0/4111529407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 msgr2=0x558403103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 0x558403103750 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2078012fd0 tx=0x7f2078014000 comp rx=0 tx=0).stop 2023-12-14T18:28:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 -- 172.21.15.31:0/2516239586 shutdown_connections 2023-12-14T18:28:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d790f0 0x560218d7b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d6de30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 --2- 172.21.15.31:0/2516239586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d6e370 0x560218d6e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 -- 172.21.15.31:0/2516239586 >> 172.21.15.31:0/2516239586 conn(0x560218d64890 msgr2=0x560218d6a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 -- 172.21.15.31:0/4111529407 shutdown_connections 2023-12-14T18:28:49.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x558403110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103380 0x558403103750 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 --2- 172.21.15.31:0/4111529407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x55840310e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.698+0000 7f2099891b80 1 -- 172.21.15.31:0/4111529407 >> 172.21.15.31:0/4111529407 conn(0x5584030f9ca0 msgr2=0x5584030ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.698+0000 7fc22555fb80 1 -- 172.21.15.31:0/2516239586 shutdown_connections 2023-12-14T18:28:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.699+0000 7fc22555fb80 1 -- 172.21.15.31:0/2516239586 wait complete. 2023-12-14T18:28:49.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.699+0000 7f2099891b80 1 -- 172.21.15.31:0/4111529407 shutdown_connections 2023-12-14T18:28:49.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.699+0000 7f2099891b80 1 -- 172.21.15.31:0/4111529407 wait complete. 2023-12-14T18:28:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.699+0000 7fc22555fb80 1 -- start start 2023-12-14T18:28:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.699+0000 7f2099891b80 1 -- start start 2023-12-14T18:28:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.699+0000 7fc22555fb80 1 Processor -- start 2023-12-14T18:28:49.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.699+0000 7f2099891b80 1 Processor -- start 2023-12-14T18:28:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103c90 0x55840310b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x55840310bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218c5cf30 con 0x560218d790f0 2023-12-14T18:28:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218c5cbd0 con 0x560218d6e370 2023-12-14T18:28:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218b44640 con 0x560218bcb430 2023-12-14T18:28:49.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc22555fb80 10 client.0 _ll_register_callbacks cb 0x560218c808b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402ff1f90 con 0x558403103c90 2023-12-14T18:28:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402ff1c30 con 0x55840310eb50 2023-12-14T18:28:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402ed96a0 con 0x558403103380 2023-12-14T18:28:49.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2099891b80 10 client.0 _ll_register_callbacks cb 0x558403015910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36822/0 (socket says 172.21.15.31:36822) 2023-12-14T18:28:49.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 -- 172.21.15.31:0/1643570571 learned_addr learned my addr 172.21.15.31:0/1643570571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc212ffd700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:58598/0 (socket says 172.21.15.31:58598) 2023-12-14T18:28:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f2086ffd700 1 -- 172.21.15.31:0/479586877 learned_addr learned my addr 172.21.15.31:0/479586877 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.700+0000 7f20867fc700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103c90 0x55840310b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 -- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 msgr2=0x560218d78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 -- 172.21.15.31:0/1643570571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 msgr2=0x560218d75d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.716 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d75d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc2137fe700 1 -- 172.21.15.31:0/1643570571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc204017070 con 0x560218d790f0 2023-12-14T18:28:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.700+0000 7fc212ffd700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2086ffd700 1 -- 172.21.15.31:0/479586877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 msgr2=0x55840310bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f20877fe700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x55840310bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2086ffd700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x55840310bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2086ffd700 1 -- 172.21.15.31:0/479586877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103c90 msgr2=0x55840310b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2086ffd700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103c90 0x55840310b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2086ffd700 1 -- 172.21.15.31:0/479586877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2078017050 con 0x558403103380 2023-12-14T18:28:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f20877fe700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x55840310bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f20867fc700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103c90 0x55840310b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc2137fe700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d762c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009410 tx=0x7fc20c00e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc246643a2dd7be6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc1fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2086ffd700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310e500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2078028040 tx=0x7f2078005500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a53eb016583cdf0c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f206aff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:28:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/1643570571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c004db0 con 0x560218d790f0 2023-12-14T18:28:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/1643570571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fc20c005020 con 0x560218d790f0 2023-12-14T18:28:49.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560218d61ea0 con 0x560218d790f0 2023-12-14T18:28:49.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f206aff5700 1 -- 172.21.15.31:0/479586877 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2078026bd0 con 0x558403103380 2023-12-14T18:28:49.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584030f72b0 con 0x558403103380 2023-12-14T18:28:49.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.701+0000 7f206aff5700 1 -- 172.21.15.31:0/479586877 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f2078026d30 con 0x558403103380 2023-12-14T18:28:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/1643570571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc20c00eec0 con 0x560218d790f0 2023-12-14T18:28:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc22555fb80 10 client.15834 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:49.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.701+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560218c87110 con 0x560218d790f0 2023-12-14T18:28:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f206aff5700 1 -- 172.21.15.31:0/479586877 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20780046d0 con 0x558403103380 2023-12-14T18:28:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 10 client.25693 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:49.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55840301c170 con 0x558403103380 2023-12-14T18:28:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/1643570571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560218d61ea0 con 0x560218d790f0 2023-12-14T18:28:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/1643570571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc20c0166f0 con 0x560218d790f0 2023-12-14T18:28:49.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f206aff5700 1 -- 172.21.15.31:0/479586877 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584030f72b0 con 0x558403103380 2023-12-14T18:28:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc22555fb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc22555fb80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc22555fb80 20 client.15834 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:49.722 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc22555fb80 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218d88b90 0x560218d8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc22555fb80 4 client.15834 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:49.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560218cb9c80 con 0x560218d88b90 2023-12-14T18:28:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f206aff5700 1 -- 172.21.15.31:0/479586877 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f207802f650 con 0x558403103380 2023-12-14T18:28:49.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 10 client.25693 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 20 client.25693 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55840311e3e0 0x5584031207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 4 client.25693 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:49.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.702+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55840304ece0 con 0x55840311e3e0 2023-12-14T18:28:49.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.702+0000 7fc212ffd700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218d88b90 0x560218d8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.703+0000 7f20867fc700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55840311e3e0 0x5584031207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.703+0000 7fc212ffd700 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218d88b90 0x560218d8afa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e47975f63290769 server_cookie=f48b3149ea78ea5e in_seq=0 out_seq=0 2023-12-14T18:28:49.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.703+0000 7fc1fa7fc700 10 client.15834 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.703+0000 7f20867fc700 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55840311e3e0 0x5584031207f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e50d5a880de618 server_cookie=cc6546dc688cb363 in_seq=0 out_seq=0 2023-12-14T18:28:49.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.703+0000 7f206aff5700 10 client.25693 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/1643570571 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560218cb9c80 con 0x560218d88b90 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc1fa7fc700 10 client.15834 handle_command_reply: tid=0 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 client.15834 shutdown 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 msgr2=0x560218d762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d762c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009410 tx=0x7fc20c00e7d0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 shutdown_connections 2023-12-14T18:28:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218d88b90 0x560218d8afa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f206aff5700 1 -- 172.21.15.31:0/479586877 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55840304ece0 con 0x55840311e3e0 2023-12-14T18:28:49.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f206aff5700 10 client.25693 handle_command_reply: tid=0 2023-12-14T18:28:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d762c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 --2- 172.21.15.31:0/1643570571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d75d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 >> 172.21.15.31:0/1643570571 conn(0x560218d64890 msgr2=0x560218d69d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc1fa7fc700 0 client.15834 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 client.25693 shutdown 2023-12-14T18:28:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 shutdown_connections 2023-12-14T18:28:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 1 -- 172.21.15.31:0/1643570571 wait complete. 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 msgr2=0x55840310e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310e500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2078028040 tx=0x7f2078005500 comp rx=0 tx=0).stop 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 shutdown_connections 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55840311e3e0 0x5584031207f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55840310eb50 0x55840310bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310e500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 --2- 172.21.15.31:0/479586877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558403103c90 0x55840310b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.704+0000 7fc22555fb80 20 client.15834 trim_cache size 0 max 16384 2023-12-14T18:28:49.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f206aff5700 0 client.25693 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 >> 172.21.15.31:0/479586877 conn(0x5584030f9ca0 msgr2=0x5584030ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.704+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 shutdown_connections 2023-12-14T18:28:49.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.705+0000 7f2099891b80 1 -- 172.21.15.31:0/479586877 wait complete. 2023-12-14T18:28:49.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.705+0000 7f2099891b80 20 client.25693 trim_cache size 0 max 16384 2023-12-14T18:28:49.731 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:49 smithi031 ceph-mon[42085]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:28:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.721+0000 7fc22555fb80 1 Processor -- start 2023-12-14T18:28:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.721+0000 7fc22555fb80 1 -- start start 2023-12-14T18:28:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d7fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d70f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d71470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc22555fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218d6f0a0 con 0x560218d790f0 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc22555fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218d6eda0 con 0x560218d6e370 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc22555fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218d7ba70 con 0x560218bcb430 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d71470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc2137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d71470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36844/0 (socket says 172.21.15.31:36844) 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc2137fe700 1 -- 172.21.15.31:0/4165635831 learned_addr learned my addr 172.21.15.31:0/4165635831 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d70f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.722+0000 7fc212ffd700 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d7fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc2137fe700 1 -- 172.21.15.31:0/4165635831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 msgr2=0x560218d7fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc2137fe700 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d7fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc2137fe700 1 -- 172.21.15.31:0/4165635831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 msgr2=0x560218d70f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc2137fe700 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d70f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc2137fe700 1 -- 172.21.15.31:0/4165635831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560218d660d0 con 0x560218d790f0 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d70f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc2137fe700 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d71470 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009390 tx=0x7fc20c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbef8a9e673101e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc209ffb700 1 -- 172.21.15.31:0/4165635831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c02f070 con 0x560218d790f0 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc209ffb700 1 -- 172.21.15.31:0/4165635831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc20c00f040 con 0x560218d790f0 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.723+0000 7fc209ffb700 1 -- 172.21.15.31:0/4165635831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c0050c0 con 0x560218d790f0 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 -- 172.21.15.31:0/4165635831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 msgr2=0x560218d71470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d71470 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009390 tx=0x7fc20c00eb20 comp rx=0 tx=0).stop 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 -- 172.21.15.31:0/4165635831 shutdown_connections 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218d790f0 0x560218d71470 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218bcb430 0x560218d7fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4165635831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218d70f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 -- 172.21.15.31:0/4165635831 >> 172.21.15.31:0/4165635831 conn(0x560218d64890 msgr2=0x560218d8aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 -- 172.21.15.31:0/4165635831 shutdown_connections 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.724+0000 7fc22555fb80 1 -- 172.21.15.31:0/4165635831 wait complete. 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 -- start start 2023-12-14T18:28:49.735 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 Processor -- start 2023-12-14T18:28:49.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.725+0000 7f2099891b80 1 Processor -- start 2023-12-14T18:28:49.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.725+0000 7f2099891b80 1 -- start start 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 0x560218dd41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218dd4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218d790f0 0x560218d7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218d7f800 con 0x560218bcb430 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218d7f970 con 0x560218d6e370 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560218d7fb30 con 0x560218d790f0 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc22555fb80 10 client.0 _ll_register_callbacks cb 0x7fc20c007980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.725+0000 7fc212ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 0x560218dd41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.725+0000 7f2099891b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 0x560218dd41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36860/0 (socket says 172.21.15.31:36860) 2023-12-14T18:28:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 -- 172.21.15.31:0/4167605900 learned_addr learned my addr 172.21.15.31:0/4167605900 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2099891b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103c90 0x55840310e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2099891b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2099891b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840310c770 con 0x55840310eb50 2023-12-14T18:28:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2099891b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840310cc40 con 0x558403103c90 2023-12-14T18:28:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2099891b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558403109d10 con 0x558403103380 2023-12-14T18:28:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218dd4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc2137fe700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218d790f0 0x560218d7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403106ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36862/0 (socket says 172.21.15.31:36862) 2023-12-14T18:28:49.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 -- 172.21.15.31:0/1358166826 learned_addr learned my addr 172.21.15.31:0/1358166826 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20867fc700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103c90 0x55840310e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 -- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218d790f0 msgr2=0x560218d7f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218d790f0 0x560218d7f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 -- 172.21.15.31:0/4167605900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 msgr2=0x560218dd4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218dd4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 -- 172.21.15.31:0/4167605900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2141124f0 con 0x560218bcb430 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 -- 172.21.15.31:0/1358166826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 msgr2=0x55840310dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 -- 172.21.15.31:0/1358166826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103c90 msgr2=0x55840310e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103c90 0x55840310e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20877fe700 1 -- 172.21.15.31:0/1358166826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840301bec0 con 0x55840310eb50 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f20867fc700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103c90 0x55840310e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:49.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.726+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc212ffd700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 0x560218dd41e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e480 tx=0x7fc204005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6cd2864731c851 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.726+0000 7fc1fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:49.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.727+0000 7f20877fe700 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403106ad0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f20800971f0 tx=0x7f208000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98039a4548fe9010 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/4167605900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204026d00 con 0x560218bcb430 2023-12-14T18:28:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/4167605900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7fc204031390 con 0x560218bcb430 2023-12-14T18:28:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc204021070 con 0x560218bcb430 2023-12-14T18:28:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.727+0000 7f206dffb700 1 -- 172.21.15.31:0/1358166826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080019320 con 0x55840310eb50 2023-12-14T18:28:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.727+0000 7f206dffb700 1 -- 172.21.15.31:0/1358166826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f208000be20 con 0x55840310eb50 2023-12-14T18:28:49.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.727+0000 7f206dffb700 1 -- 172.21.15.31:0/1358166826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208009af50 con 0x55840310eb50 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/4167605900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc204026e60 con 0x560218bcb430 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc22555fb80 10 client.15843 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560218c87110 con 0x560218bcb430 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/4167605900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc204021070 con 0x560218bcb430 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.727+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/4167605900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc2040264a0 con 0x560218bcb430 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc22555fb80 10 client.15843 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc22555fb80 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc22555fb80 20 client.15843 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218ddd020 0x560218ddf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc22555fb80 4 client.15843 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560218cb99a0 con 0x560218ddd020 2023-12-14T18:28:49.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 -- 172.21.15.31:0/1358166826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 msgr2=0x558403106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403106ad0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f20800971f0 tx=0x7f208000b3e0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218ddd020 0x560218ddf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 -- 172.21.15.31:0/1358166826 shutdown_connections 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403106ad0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103380 0x55840310dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 --2- 172.21.15.31:0/1358166826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103c90 0x55840310e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 -- 172.21.15.31:0/1358166826 >> 172.21.15.31:0/1358166826 conn(0x5584030f9ca0 msgr2=0x55840311a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 -- 172.21.15.31:0/1358166826 shutdown_connections 2023-12-14T18:28:49.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.728+0000 7f2099891b80 1 -- 172.21.15.31:0/1358166826 wait complete. 2023-12-14T18:28:49.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc20a7fc700 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218ddd020 0x560218ddf430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b9e77e7f33f329 server_cookie=57ba04980c207c85 in_seq=0 out_seq=0 2023-12-14T18:28:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.728+0000 7fc1fa7fc700 10 client.15843 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.749 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:49 smithi176 ceph-mon[43506]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:28:49.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 -- start start 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 Processor -- start 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103380 0x5584031134b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x5584031139f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403171260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584031717a0 con 0x55840310eb50 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584031718e0 con 0x558403103380 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558403171a20 con 0x558403103c90 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2099891b80 10 client.0 _ll_register_callbacks cb 0x7f2080005210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f20877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403171260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f2086ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103380 0x5584031134b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f20877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403171260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36874/0 (socket says 172.21.15.31:36874) 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.729+0000 7f20877fe700 1 -- 172.21.15.31:0/1381100398 learned_addr learned my addr 172.21.15.31:0/1381100398 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2086ffd700 1 -- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 msgr2=0x5584031139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x5584031139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2086ffd700 1 -- 172.21.15.31:0/1381100398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 msgr2=0x558403171260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403171260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2086ffd700 1 -- 172.21.15.31:0/1381100398 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55840301c170 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f20877fe700 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403171260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2086ffd700 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103380 0x5584031134b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f207802fa00 tx=0x7f207802fe30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96aaed2184c31646 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f206aff5700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f206aff5700 1 -- 172.21.15.31:0/1381100398 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207802f640 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f206aff5700 1 -- 172.21.15.31:0/1381100398 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9321+0+0 (secure 0 0 0) 0x7f20780269d0 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.730+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f207800ee40 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f206aff5700 1 -- 172.21.15.31:0/1381100398 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2078005620 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 10 client.25697 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f208810e190 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f206aff5700 1 -- 172.21.15.31:0/1381100398 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f207800ee40 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f206aff5700 1 -- 172.21.15.31:0/1381100398 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2078043480 con 0x558403103380 2023-12-14T18:28:49.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 10 client.25697 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 10 client.25697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 20 client.25697 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558403172da0 0x558403175180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 4 client.25697 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55840304ea00 con 0x558403172da0 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.731+0000 7f20867fc700 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558403172da0 0x558403175180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.732+0000 7f20867fc700 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558403172da0 0x558403175180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4f78ab03b9b854 server_cookie=631c591aa7cbff11 in_seq=0 out_seq=0 2023-12-14T18:28:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.732+0000 7f206aff5700 10 client.25697 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc1fa7fc700 1 -- 172.21.15.31:0/4167605900 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560218cb99a0 con 0x560218ddd020 2023-12-14T18:28:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc1fa7fc700 10 client.15843 handle_command_reply: tid=0 2023-12-14T18:28:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 client.15843 shutdown 2023-12-14T18:28:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f206aff5700 1 -- 172.21.15.31:0/1381100398 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55840304ea00 con 0x558403172da0 2023-12-14T18:28:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f206aff5700 10 client.25697 handle_command_reply: tid=0 2023-12-14T18:28:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 client.25697 shutdown 2023-12-14T18:28:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 msgr2=0x560218dd41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 0x560218dd41e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e480 tx=0x7fc204005d50 comp rx=0 tx=0).stop 2023-12-14T18:28:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103380 msgr2=0x5584031134b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103380 0x5584031134b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f207802fa00 tx=0x7f207802fe30 comp rx=0 tx=0).stop 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 shutdown_connections 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560218ddd020 0x560218ddf430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560218d790f0 0x560218d7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560218bcb430 0x560218dd41e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 --2- 172.21.15.31:0/4167605900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560218d6e370 0x560218dd4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc1fa7fc700 0 client.15843 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.745+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 >> 172.21.15.31:0/4167605900 conn(0x560218d64890 msgr2=0x560218d693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 shutdown_connections 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558403172da0 0x558403175180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55840310eb50 0x558403171260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558403103380 0x5584031134b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f206aff5700 0 client.25697 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 --2- 172.21.15.31:0/1381100398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558403103c90 0x5584031139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 >> 172.21.15.31:0/1381100398 conn(0x5584030f9ca0 msgr2=0x558403106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 shutdown_connections 2023-12-14T18:28:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.745+0000 7f2099891b80 1 -- 172.21.15.31:0/1381100398 wait complete. 2023-12-14T18:28:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.746+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 shutdown_connections 2023-12-14T18:28:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f2099891b80 20 client.25697 trim_cache size 0 max 16384 2023-12-14T18:28:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.746+0000 7fc22555fb80 1 -- 172.21.15.31:0/4167605900 wait complete. 2023-12-14T18:28:49.756 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:28:49.756 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:28:49.756 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:28:49.756 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:28:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.746+0000 7fc22555fb80 20 client.15843 trim_cache size 0 max 16384 2023-12-14T18:28:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:28:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:28:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:28:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f20700a1440 msgr2=0x7f20700a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f20700a1440 0x7f20700a38f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2078009fd0 tx=0x7f2078009260 comp rx=0 tx=0).stop 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2088104410 msgr2=0x7f208819bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2088104410 0x7f208819bff0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2080009fd0 tx=0x7f2080007860 comp rx=0 tx=0).stop 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 shutdown_connections 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f20700a1440 0x7f20700a38f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2088104410 0x7f208819bff0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f20880ff3d0 0x7f2088198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 --2- 172.21.15.31:0/1592374384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20880ffd70 0x7f2088198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.746+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 >> 172.21.15.31:0/1592374384 conn(0x7f2088076230 msgr2=0x7f20880fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.746+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc2000a1460 msgr2=0x7fc2000a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc2000a1460 0x7fc2000a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc204009ca0 tx=0x7fc204009260 comp rx=0 tx=0).stop 2023-12-14T18:28:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214102ea0 msgr2=0x7fc214198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214102ea0 0x7fc214198b00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0059d0 tx=0x7fc1fc005720 comp rx=0 tx=0).stop 2023-12-14T18:28:49.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.747+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 shutdown_connections 2023-12-14T18:28:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 shutdown_connections 2023-12-14T18:28:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc2000a1460 0x7fc2000a3910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc214108500 0x7fc21419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2141024f0 0x7fc2141985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 --2- 172.21.15.31:0/3039894141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc214102ea0 0x7fc214198b00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:49.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:49.747+0000 7f208d703700 1 -- 172.21.15.31:0/1592374384 wait complete. 2023-12-14T18:28:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 >> 172.21.15.31:0/3039894141 conn(0x7fc2140fe030 msgr2=0x7fc2140fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 shutdown_connections 2023-12-14T18:28:49.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:28:49.747+0000 7fc2193d1700 1 -- 172.21.15.31:0/3039894141 wait complete. 2023-12-14T18:28:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:49 smithi204 ceph-mon[43866]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:28:49.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:28:49.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:28:49.879 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:49.880 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:50.038 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:50.038 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:50.058 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:50.058 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:50.058 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 12 -lt 100 ']' 2023-12-14T18:28:50.058 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 12 + 1 2023-12-14T18:28:50.100 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=13 2023-12-14T18:28:50.100 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:50.111 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:50.111 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:50.113 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:50.113 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:50.113 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:50.504 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:50.504 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:50.600 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:50.600 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:50.609 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:50.609 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:50.609 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 13 -lt 100 ']' 2023-12-14T18:28:50.609 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 13 + 1 2023-12-14T18:28:50.641 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=14 2023-12-14T18:28:50.641 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:50.692 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:50.692 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:50.694 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:50.694 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:50.694 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:51.146 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:51.146 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:51.383 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:51.383 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:51.400 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:51.400 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:51.400 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 14 -lt 100 ']' 2023-12-14T18:28:51.400 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 14 + 1 2023-12-14T18:28:51.449 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=15 2023-12-14T18:28:51.450 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:51.500 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:51.500 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:51.501 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:51.502 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:51.502 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:51.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:51 smithi031 ceph-mon[42085]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2023-12-14T18:28:51.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:51 smithi176 ceph-mon[43506]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2023-12-14T18:28:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:51 smithi204 ceph-mon[43866]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2023-12-14T18:28:52.170 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:52.170 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:52.400 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:52.400 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:52.417 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:52.417 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:52.417 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 15 -lt 100 ']' 2023-12-14T18:28:52.417 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 15 + 1 2023-12-14T18:28:52.516 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=16 2023-12-14T18:28:52.517 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:52.633 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:52.633 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:52.634 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:52.635 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:52.635 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 -- 172.21.15.31:0/816853116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c0ff900 msgr2=0x7f5e5c10c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 --2- 172.21.15.31:0/816853116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c10c660 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009990 tx=0x7f5e4c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 -- 172.21.15.31:0/816853116 shutdown_connections 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 --2- 172.21.15.31:0/816853116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 --2- 172.21.15.31:0/816853116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e5c0feff0 0x7f5e5c0ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 --2- 172.21.15.31:0/816853116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c10c660 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 -- 172.21.15.31:0/816853116 >> 172.21.15.31:0/816853116 conn(0x7f5e5c076370 msgr2=0x7f5e5c076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:52.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 -- 172.21.15.31:0/816853116 shutdown_connections 2023-12-14T18:28:52.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.634+0000 7f5e64405700 1 -- 172.21.15.31:0/816853116 wait complete. 2023-12-14T18:28:52.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 Processor -- start 2023-12-14T18:28:52.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 -- start start 2023-12-14T18:28:52.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e5c0feff0 0x7f5e5c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:52.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c19d610 con 0x7f5e5c10cba0 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c19d780 con 0x7f5e5c0feff0 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.635+0000 7f5e64405700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c19d940 con 0x7f5e5c0ff900 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36894/0 (socket says 172.21.15.31:36894) 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 -- 172.21.15.31:0/1406966507 learned_addr learned my addr 172.21.15.31:0/1406966507 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e621a1700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e5c0feff0 0x7f5e5c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e619a0700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:52.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 -- 172.21.15.31:0/1406966507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c0ff900 msgr2=0x7f5e5c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 -- 172.21.15.31:0/1406966507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e5c0feff0 msgr2=0x7f5e5c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e5c0feff0 0x7f5e5c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 -- 172.21.15.31:0/1406966507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e54009c40 con 0x7f5e5c10cba0 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e619a0700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e629a2700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c19cfd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c0f0 tx=0x7f5e5400c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.636+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400cab0 con 0x7f5e5c10cba0 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.637+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5400e030 con 0x7f5e5c10cba0 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.637+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c0095f0 con 0x7f5e5c10cba0 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.637+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004670 con 0x7f5e5c10cba0 2023-12-14T18:28:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.637+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c1124f0 con 0x7f5e5c10cba0 2023-12-14T18:28:52.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.638+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c112360 con 0x7f5e5c10cba0 2023-12-14T18:28:52.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.640+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f5e5400cd10 con 0x7f5e5c10cba0 2023-12-14T18:28:52.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.641+0000 7f5e537fe700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5e480a1460 0x7f5e480a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:52.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.641+0000 7f5e621a1700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5e480a1460 0x7f5e480a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:52.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.641+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f5e540cd770 con 0x7f5e5c10cba0 2023-12-14T18:28:52.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.641+0000 7f5e621a1700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5e480a1460 0x7f5e480a3910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009c60 tx=0x7f5e58009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:52.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.658+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e54096630 con 0x7f5e5c10cba0 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-12-14T18:28:52.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:52 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2023-12-14T18:28:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-14T18:28:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-12-14T18:28:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-14T18:28:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-12-14T18:28:52.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2023-12-14T18:28:52.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-12-14T18:28:52.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-12-14T18:28:52.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:52 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2023-12-14T18:28:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-14T18:28:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-12-14T18:28:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2023-12-14T18:28:52.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-12-14T18:28:52.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2023-12-14T18:28:52.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-12-14T18:28:52.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2023-12-14T18:28:52.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:52 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2023-12-14T18:28:52.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.920+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e5c112680 con 0x7f5e5c10cba0 2023-12-14T18:28:52.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.921+0000 7f5e537fe700 1 -- 172.21.15.31:0/1406966507 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f5e54099c50 con 0x7f5e5c10cba0 2023-12-14T18:28:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:28:52.924 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:28:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.923+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5e480a1460 msgr2=0x7f5e480a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.923+0000 7f5e64405700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5e480a1460 0x7f5e480a3910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009c60 tx=0x7f5e58009260 comp rx=0 tx=0).stop 2023-12-14T18:28:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 msgr2=0x7f5e5c19cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:52.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c19cfd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400c0f0 tx=0x7f5e5400c400 comp rx=0 tx=0).stop 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 shutdown_connections 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e5c10cba0 0x7f5e5c19cfd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5e5c0feff0 0x7f5e5c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5e480a1460 0x7f5e480a3910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 --2- 172.21.15.31:0/1406966507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e5c0ff900 0x7f5e5c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 >> 172.21.15.31:0/1406966507 conn(0x7f5e5c076370 msgr2=0x7f5e5c0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 shutdown_connections 2023-12-14T18:28:52.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:52.924+0000 7f5e64405700 1 -- 172.21.15.31:0/1406966507 wait complete. 2023-12-14T18:28:52.928 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:28:52.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:28:53.220 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:53.221 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:53.276 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:53.276 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:53.292 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:53.292 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:53.292 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 16 -lt 100 ']' 2023-12-14T18:28:53.292 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 16 + 1 2023-12-14T18:28:53.349 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=17 2023-12-14T18:28:53.350 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:53.369 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:53.369 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:53.371 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:53.371 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:53.371 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.2 KiB/s wr, 5 op/s 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: osdmap e89: 12 total, 12 up, 12 in 2023-12-14T18:28:53.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:53 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1406966507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:28:53.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.2 KiB/s wr, 5 op/s 2023-12-14T18:28:53.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-14T18:28:53.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-12-14T18:28:53.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2023-12-14T18:28:53.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2023-12-14T18:28:53.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: osdmap e89: 12 total, 12 up, 12 in 2023-12-14T18:28:53.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:53 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1406966507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:28:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.2 KiB/s wr, 5 op/s 2023-12-14T18:28:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2023-12-14T18:28:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-12-14T18:28:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2023-12-14T18:28:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2023-12-14T18:28:53.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: osdmap e89: 12 total, 12 up, 12 in 2023-12-14T18:28:53.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:53 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1406966507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:28:54.120 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:54.121 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:54.318 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:54.318 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:54.342 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:54.342 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:54.342 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 17 -lt 100 ']' 2023-12-14T18:28:54.342 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 17 + 1 2023-12-14T18:28:54.391 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=18 2023-12-14T18:28:54.391 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:54.452 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:54.452 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:54.453 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:54.453 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:54.453 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:54.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:54 smithi031 ceph-mon[42085]: osdmap e90: 12 total, 12 up, 12 in 2023-12-14T18:28:54.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:54 smithi176 ceph-mon[43506]: osdmap e90: 12 total, 12 up, 12 in 2023-12-14T18:28:54.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:54 smithi204 ceph-mon[43866]: osdmap e90: 12 total, 12 up, 12 in 2023-12-14T18:28:55.028 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:55.029 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:55.199 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:55.199 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:55.233 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:55.234 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:55.234 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 18 -lt 100 ']' 2023-12-14T18:28:55.234 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 18 + 1 2023-12-14T18:28:55.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=19 2023-12-14T18:28:55.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:55.351 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:55.351 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:55.352 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:55.352 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:55.352 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:55.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:55 smithi031 ceph-mon[42085]: pgmap v36: 129 pgs: 4 peering, 125 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:28:55.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:55 smithi176 ceph-mon[43506]: pgmap v36: 129 pgs: 4 peering, 125 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:28:55.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:55 smithi204 ceph-mon[43866]: pgmap v36: 129 pgs: 4 peering, 125 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:28:55.920 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:55.920 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:56.074 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:56.074 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:56.090 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:56.090 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:56.090 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 19 -lt 100 ']' 2023-12-14T18:28:56.090 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 19 + 1 2023-12-14T18:28:56.124 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=20 2023-12-14T18:28:56.124 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:56.155 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:56.155 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:56.156 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:56.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:56.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:56.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 -- 172.21.15.31:0/3501939385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078105550 msgr2=0x7ff07810d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 --2- 172.21.15.31:0/3501939385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078105550 0x7ff07810d120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff070009f90 tx=0x7ff07000b330 comp rx=0 tx=0).stop 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 -- 172.21.15.31:0/3501939385 shutdown_connections 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 --2- 172.21.15.31:0/3501939385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078105550 0x7ff07810d120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 --2- 172.21.15.31:0/3501939385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff078100510 0x7ff0781008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 --2- 172.21.15.31:0/3501939385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100eb0 0x7ff078104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 -- 172.21.15.31:0/3501939385 >> 172.21.15.31:0/3501939385 conn(0x7ff0780fbd80 msgr2=0x7ff0780fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 -- 172.21.15.31:0/3501939385 shutdown_connections 2023-12-14T18:28:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.525+0000 7ff07eb17700 1 -- 172.21.15.31:0/3501939385 wait complete. 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 Processor -- start 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 -- start start 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 0x7ff0781986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078100eb0 0x7ff078198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff078105550 0x7ff07819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07819c5c0 con 0x7ff078100510 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07819c730 con 0x7ff078100eb0 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.526+0000 7ff07eb17700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07819c8f0 con 0x7ff078105550 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 0x7ff0781986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 0x7ff0781986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34602/0 (socket says 172.21.15.31:34602) 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 -- 172.21.15.31:0/2765916478 learned_addr learned my addr 172.21.15.31:0/2765916478 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff077fff700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078100eb0 0x7ff078198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff078105550 0x7ff07819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 -- 172.21.15.31:0/2765916478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff078105550 msgr2=0x7ff07819bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff078105550 0x7ff07819bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 -- 172.21.15.31:0/2765916478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078100eb0 msgr2=0x7ff078198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078100eb0 0x7ff078198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 -- 172.21.15.31:0/2765916478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0600095f0 con 0x7ff078100510 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.527+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 0x7ff0781986d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9a0 tx=0x7ff06800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.528+0000 7ff075ffb700 1 -- 172.21.15.31:0/2765916478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068009790 con 0x7ff078100510 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.528+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff070009c40 con 0x7ff078100510 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.528+0000 7ff075ffb700 1 -- 172.21.15.31:0/2765916478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff068010430 con 0x7ff078100510 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.528+0000 7ff075ffb700 1 -- 172.21.15.31:0/2765916478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068018680 con 0x7ff078100510 2023-12-14T18:28:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.528+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff078110370 con 0x7ff078100510 2023-12-14T18:28:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.531+0000 7ff075ffb700 1 -- 172.21.15.31:0/2765916478 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff0680105a0 con 0x7ff078100510 2023-12-14T18:28:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.531+0000 7ff075ffb700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff0640a13e0 0x7ff0640a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.531+0000 7ff077fff700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff0640a13e0 0x7ff0640a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.532+0000 7ff075ffb700 1 -- 172.21.15.31:0/2765916478 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff068014070 con 0x7ff078100510 2023-12-14T18:28:56.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.532+0000 7ff077fff700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff0640a13e0 0x7ff0640a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff078199ce0 tx=0x7ff060009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.635+0000 7ff08aca5b80 1 Processor -- start 2023-12-14T18:28:56.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.635+0000 7ff08aca5b80 1 -- start start 2023-12-14T18:28:56.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c2fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c305610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c307f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c1e45e0 con 0x56551c2fa380 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c253870 con 0x56551c2fac90 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c24cd00 con 0x56551c305b50 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c2fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c2fa750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34614/0 (socket says 172.21.15.31:34614) 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07c8b3700 1 -- 172.21.15.31:0/1824107619 learned_addr learned my addr 172.21.15.31:0/1824107619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff077fff700 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c305610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c307f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07c8b3700 1 -- 172.21.15.31:0/1824107619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 msgr2=0x56551c307f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c307f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.636+0000 7ff07c8b3700 1 -- 172.21.15.31:0/1824107619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 msgr2=0x56551c305610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.637+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c305610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.637+0000 7ff07c8b3700 1 -- 172.21.15.31:0/1824107619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551c0ce3b0 con 0x56551c2fa380 2023-12-14T18:28:56.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.637+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c2fa750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff068099fa0 tx=0x7ff068099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa5a060a6f4d0d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.637+0000 7ff06effd700 1 -- 172.21.15.31:0/1824107619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06809e040 con 0x56551c2fa380 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.637+0000 7ff06effd700 1 -- 172.21.15.31:0/1824107619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0680a2460 con 0x56551c2fa380 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.637+0000 7ff06effd700 1 -- 172.21.15.31:0/1824107619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0680a1fd0 con 0x56551c2fa380 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1824107619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 msgr2=0x56551c2fa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c2fa750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff068099fa0 tx=0x7ff068099a60 comp rx=0 tx=0).stop 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1824107619 shutdown_connections 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c307f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c2fa750 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1824107619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c305610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1824107619 >> 172.21.15.31:0/1824107619 conn(0x56551c2f0ca0 msgr2=0x56551c2f67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1824107619 shutdown_connections 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.638+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1824107619 wait complete. 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.639+0000 7ff08aca5b80 1 -- start start 2023-12-14T18:28:56.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.639+0000 7ff08aca5b80 1 Processor -- start 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c3054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c302810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c302e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c1e8f90 con 0x56551c305b50 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c1e8c30 con 0x56551c2fa380 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c0d06a0 con 0x56551c2fac90 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff08aca5b80 10 client.0 _ll_register_callbacks cb 0x56551c20c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:56.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff07d0b4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c302e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff07d0b4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c302e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34630/0 (socket says 172.21.15.31:34630) 2023-12-14T18:28:56.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1801373884 learned_addr learned my addr 172.21.15.31:0/1801373884 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:56.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff077fff700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c302810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.640+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c3054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 msgr2=0x56551c302810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c302810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1801373884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 msgr2=0x56551c3054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c3054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1801373884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff078110370 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff077fff700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c302810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c302e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff070007680 tx=0x7ff0700076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec5b88a9296da66 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff057fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff057fff700 1 -- 172.21.15.31:0/1801373884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07000e040 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff057fff700 1 -- 172.21.15.31:0/1801373884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff070003c00 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.641+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56551c2ee2b0 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff057fff700 1 -- 172.21.15.31:0/1801373884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff070007850 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 10 client.15867 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56551c213170 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff057fff700 1 -- 172.21.15.31:0/1801373884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56551c2ee2b0 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff057fff700 1 -- 172.21.15.31:0/1801373884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff070019070 con 0x56551c305b50 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:56.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 10 client.15867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:28:56.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 20 client.15867 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:56.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c315500 0x56551c317910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 4 client.15867 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:56.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.642+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56551c245ce0 con 0x56551c315500 2023-12-14T18:28:56.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.643+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c315500 0x56551c317910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.643+0000 7ff057fff700 10 client.15867 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.643+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c315500 0x56551c317910 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc4fcf4dc629a4e6 server_cookie=42b8efd2ec2f5562 in_seq=0 out_seq=0 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff057fff700 1 -- 172.21.15.31:0/1801373884 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56551c245ce0 con 0x56551c315500 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff057fff700 10 client.15867 handle_command_reply: tid=0 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 client.15867 shutdown 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 msgr2=0x56551c302e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c302e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff070007680 tx=0x7ff0700076b0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 shutdown_connections 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c315500 0x56551c317910 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c302e40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c3054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff057fff700 0 client.15867 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1801373884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c302810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 >> 172.21.15.31:0/1801373884 conn(0x56551c2f0ca0 msgr2=0x56551c2f63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 shutdown_connections 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.644+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1801373884 wait complete. 2023-12-14T18:28:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.645+0000 7ff08aca5b80 20 client.15867 trim_cache size 0 max 16384 2023-12-14T18:28:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.659+0000 7ff08aca5b80 1 Processor -- start 2023-12-14T18:28:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.659+0000 7ff08aca5b80 1 -- start start 2023-12-14T18:28:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c305120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c2fdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c2fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c2fb5b0 con 0x56551c305b50 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c2f73b0 con 0x56551c2fa380 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c303420 con 0x56551c2fac90 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c2fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c2fe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34632/0 (socket says 172.21.15.31:34632) 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1603574247 learned_addr learned my addr 172.21.15.31:0/1603574247 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff077fff700 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c2fdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c305120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1603574247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 msgr2=0x56551c2fdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c2fdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1603574247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 msgr2=0x56551c305120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c305120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.660+0000 7ff07d0b4700 1 -- 172.21.15.31:0/1603574247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551c308180 con 0x56551c305b50 2023-12-14T18:28:56.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.661+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c2fe090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b900 tx=0x7ff070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a0de317236ea04 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.661+0000 7ff06f7fe700 1 -- 172.21.15.31:0/1603574247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07000e070 con 0x56551c305b50 2023-12-14T18:28:56.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.661+0000 7ff06f7fe700 1 -- 172.21.15.31:0/1603574247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff070021070 con 0x56551c305b50 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.661+0000 7ff06f7fe700 1 -- 172.21.15.31:0/1603574247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0700052a0 con 0x56551c305b50 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1603574247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 msgr2=0x56551c2fe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c2fe090 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b900 tx=0x7ff070009bb0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1603574247 shutdown_connections 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c305b50 0x56551c2fe090 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fa380 0x56551c305120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/1603574247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c2fac90 0x56551c2fdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1603574247 >> 172.21.15.31:0/1603574247 conn(0x56551c2f0ca0 msgr2=0x56551c311800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1603574247 shutdown_connections 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 -- 172.21.15.31:0/1603574247 wait complete. 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 -- start start 2023-12-14T18:28:56.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.662+0000 7ff08aca5b80 1 Processor -- start 2023-12-14T18:28:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c30a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c30ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c30b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c368540 con 0x56551c2fa380 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c3686b0 con 0x56551c2fac90 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551c368870 con 0x56551c305b50 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff08aca5b80 10 client.0 _ll_register_callbacks cb 0x7ff070007e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff07c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c30a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff07c8b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c30a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34648/0 (socket says 172.21.15.31:34648) 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff07c8b3700 1 -- 172.21.15.31:0/3648681825 learned_addr learned my addr 172.21.15.31:0/3648681825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff077fff700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c30ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.663+0000 7ff07d0b4700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c30b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff07c8b3700 1 -- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 msgr2=0x56551c30b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c30b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff07c8b3700 1 -- 172.21.15.31:0/3648681825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 msgr2=0x56551c30ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c30ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff07c8b3700 1 -- 172.21.15.31:0/3648681825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56551c213170 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff077fff700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c30ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff07c8b3700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c30a610 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff068098360 tx=0x7ff068010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120d57058b634c42 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff057fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff057fff700 1 -- 172.21.15.31:0/3648681825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068099500 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff057fff700 1 -- 172.21.15.31:0/3648681825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff068099660 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0680a2460 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff057fff700 1 -- 172.21.15.31:0/3648681825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0680b82a0 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff08aca5b80 10 client.15873 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.664+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56551c07ab60 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff057fff700 1 -- 172.21.15.31:0/3648681825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0680c8080 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff057fff700 1 -- 172.21.15.31:0/3648681825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff0680a2060 con 0x56551c2fa380 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff08aca5b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff08aca5b80 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff08aca5b80 20 client.15873 populate_metadata read hostname 'smithi031' 2023-12-14T18:28:56.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c369d90 0x56551c36c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff08aca5b80 4 client.15873 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56551c245a00 con 0x56551c369d90 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.665+0000 7ff077fff700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c369d90 0x56551c36c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff057fff700 10 client.15873 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff077fff700 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c369d90 0x56551c36c1a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78e42606b16a5746 server_cookie=2338037fd7fbe8c7 in_seq=0 out_seq=0 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff057fff700 1 -- 172.21.15.31:0/3648681825 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56551c245a00 con 0x56551c369d90 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff057fff700 10 client.15873 handle_command_reply: tid=0 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 client.15873 shutdown 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 msgr2=0x56551c30a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c30a610 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff068098360 tx=0x7ff068010880 comp rx=0 tx=0).stop 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 shutdown_connections 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56551c369d90 0x56551c36c1a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56551c305b50 0x56551c30b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56551c2fa380 0x56551c30a610 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 --2- 172.21.15.31:0/3648681825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56551c2fac90 0x56551c30ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 >> 172.21.15.31:0/3648681825 conn(0x56551c2f0ca0 msgr2=0x56551c3050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff057fff700 0 client.15873 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:28:56.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 shutdown_connections 2023-12-14T18:28:56.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.666+0000 7ff08aca5b80 1 -- 172.21.15.31:0/3648681825 wait complete. 2023-12-14T18:28:56.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff08aca5b80 20 client.15873 trim_cache size 0 max 16384 2023-12-14T18:28:56.672 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff0640a13e0 msgr2=0x7ff0640a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff0640a13e0 0x7ff0640a3890 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff078199ce0 tx=0x7ff060009310 comp rx=0 tx=0).stop 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 msgr2=0x7ff0781986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 0x7ff0781986d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e9a0 tx=0x7ff06800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 shutdown_connections 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff0640a13e0 0x7ff0640a3890 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff078105550 0x7ff07819bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff078100510 0x7ff0781986d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 --2- 172.21.15.31:0/2765916478 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff078100eb0 0x7ff078198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 >> 172.21.15.31:0/2765916478 conn(0x7ff0780fbd80 msgr2=0x7ff0780fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.667+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 shutdown_connections 2023-12-14T18:28:56.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:28:56.668+0000 7ff07eb17700 1 -- 172.21.15.31:0/2765916478 wait complete. 2023-12-14T18:28:56.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:28:57.036 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:57.036 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:57.151 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:57.151 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:57.166 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:57.166 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:57.166 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 20 -lt 100 ']' 2023-12-14T18:28:57.166 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 20 + 1 2023-12-14T18:28:57.215 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=21 2023-12-14T18:28:57.215 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:57.251 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:57.251 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:57.253 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:57.253 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:57.253 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:57.695 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:57.695 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:57 smithi031 ceph-mon[42085]: pgmap v37: 129 pgs: 4 peering, 125 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:28:57.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:57 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:28:57.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:57 smithi176 ceph-mon[43506]: pgmap v37: 129 pgs: 4 peering, 125 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:28:57.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:57 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:28:57.826 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:57.826 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:57.832 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:57.832 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:57.832 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 21 -lt 100 ']' 2023-12-14T18:28:57.832 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 21 + 1 2023-12-14T18:28:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:57 smithi204 ceph-mon[43866]: pgmap v37: 129 pgs: 4 peering, 125 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:28:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:57 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:28:57.927 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=22 2023-12-14T18:28:57.927 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:57.990 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:57.990 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:57.992 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:57.992 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:57.992 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:59.194 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:59.194 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:59.440 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:28:59.441 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:28:59.448 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:59.448 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:59.449 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 22 -lt 100 ']' 2023-12-14T18:28:59.449 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 22 + 1 2023-12-14T18:28:59.490 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=23 2023-12-14T18:28:59.490 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:28:59.583 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:28:59.583 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:28:59.584 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:28:59.584 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:28:59.584 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:59.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:28:59 smithi031 ceph-mon[42085]: pgmap v38: 129 pgs: 3 peering, 126 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-12-14T18:28:59.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:28:59 smithi176 ceph-mon[43506]: pgmap v38: 129 pgs: 3 peering, 126 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-12-14T18:28:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:28:59 smithi204 ceph-mon[43866]: pgmap v38: 129 pgs: 3 peering, 126 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 13 op/s 2023-12-14T18:29:00.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 -- 172.21.15.31:0/1639752244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741026b0 msgr2=0x7f3f74102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 --2- 172.21.15.31:0/1639752244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741026b0 0x7f3f74102a80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3f700099c0 tx=0x7f3f70009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 -- 172.21.15.31:0/1639752244 shutdown_connections 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 --2- 172.21.15.31:0/1639752244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f741076f0 0x7f3f7410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 --2- 172.21.15.31:0/1639752244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741026b0 0x7f3f74102a80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 --2- 172.21.15.31:0/1639752244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f74103050 0x7f3f74107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 -- 172.21.15.31:0/1639752244 >> 172.21.15.31:0/1639752244 conn(0x7f3f740fdf60 msgr2=0x7f3f74100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 -- 172.21.15.31:0/1639752244 shutdown_connections 2023-12-14T18:29:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.650+0000 7f3f7c519700 1 -- 172.21.15.31:0/1639752244 wait complete. 2023-12-14T18:29:00.660 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:00.660 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 Processor -- start 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 -- start start 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f741026b0 0x7f3f741985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74103050 0x7f3f74198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 0x7f3f7419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7419d580 con 0x7f3f741076f0 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7419d6f0 con 0x7f3f741026b0 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.667+0000 7f3f7c519700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7419d8b0 con 0x7f3f74103050 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 0x7f3f7419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f79ab4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74103050 0x7f3f74198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 0x7f3f7419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34658/0 (socket says 172.21.15.31:34658) 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 -- 172.21.15.31:0/769682812 learned_addr learned my addr 172.21.15.31:0/769682812 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f741026b0 0x7f3f741985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 -- 172.21.15.31:0/769682812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74103050 msgr2=0x7f3f74198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74103050 0x7f3f74198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 -- 172.21.15.31:0/769682812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f741026b0 msgr2=0x7f3f741985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f741026b0 0x7f3f741985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 -- 172.21.15.31:0/769682812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f700095f0 con 0x7f3f741076f0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.668+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 0x7f3f7419cf40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00ef30 tx=0x7f3f6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.669+0000 7f3f677fe700 1 -- 172.21.15.31:0/769682812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c018070 con 0x7f3f741076f0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.669+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f741124b0 con 0x7f3f741076f0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.669+0000 7f3f677fe700 1 -- 172.21.15.31:0/769682812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6c00f040 con 0x7f3f741076f0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.669+0000 7f3f677fe700 1 -- 172.21.15.31:0/769682812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c00cd00 con 0x7f3f741076f0 2023-12-14T18:29:00.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.669+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f74112320 con 0x7f3f741076f0 2023-12-14T18:29:00.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.672+0000 7f3f677fe700 1 -- 172.21.15.31:0/769682812 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f3f6c003a00 con 0x7f3f741076f0 2023-12-14T18:29:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.672+0000 7f3f677fe700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f600a13e0 0x7f3f600a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.672+0000 7f3f677fe700 1 -- 172.21.15.31:0/769682812 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f6c0cce10 con 0x7f3f741076f0 2023-12-14T18:29:00.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.673+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f600a13e0 0x7f3f600a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.673+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f600a13e0 0x7f3f600a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:00.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.723+0000 7f3f886a7b80 1 Processor -- start 2023-12-14T18:29:00.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 -- start start 2023-12-14T18:29:00.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6380 0x55c6dddd6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddcc05e0 con 0x55c6ddde1b50 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddd2f870 con 0x55c6dddd6380 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddd28d00 con 0x55c6dddd6c90 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34660/0 (socket says 172.21.15.31:34660) 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f7aab6700 1 -- 172.21.15.31:0/2115619972 learned_addr learned my addr 172.21.15.31:0/2115619972 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f79ab4700 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.724+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6380 0x55c6dddd6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f7aab6700 1 -- 172.21.15.31:0/2115619972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 msgr2=0x55c6ddde1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f7aab6700 1 -- 172.21.15.31:0/2115619972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6380 msgr2=0x55c6dddd6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6380 0x55c6dddd6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f7aab6700 1 -- 172.21.15.31:0/2115619972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ddbaa3b0 con 0x55c6ddde1b50 2023-12-14T18:29:00.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde3f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c099690 tx=0x7f3f6c0994d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d490990b72dde7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f64ff9700 1 -- 172.21.15.31:0/2115619972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c004ad0 con 0x55c6ddde1b50 2023-12-14T18:29:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f64ff9700 1 -- 172.21.15.31:0/2115619972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6c099990 con 0x55c6ddde1b50 2023-12-14T18:29:00.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.725+0000 7f3f64ff9700 1 -- 172.21.15.31:0/2115619972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c098e80 con 0x55c6ddde1b50 2023-12-14T18:29:00.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 -- 172.21.15.31:0/2115619972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 msgr2=0x55c6ddde3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde3f30 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c099690 tx=0x7f3f6c0994d0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 -- 172.21.15.31:0/2115619972 shutdown_connections 2023-12-14T18:29:00.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde3f30 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6380 0x55c6dddd6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/2115619972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 -- 172.21.15.31:0/2115619972 >> 172.21.15.31:0/2115619972 conn(0x55c6dddccca0 msgr2=0x55c6dddd27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 -- 172.21.15.31:0/2115619972 shutdown_connections 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.726+0000 7f3f886a7b80 1 -- 172.21.15.31:0/2115619972 wait complete. 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 -- start start 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 Processor -- start 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6dddde840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddddef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddcc4f90 con 0x55c6ddde1b50 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddcc4c30 con 0x55c6dddd6c90 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddbac6a0 con 0x55c6dddd6380 2023-12-14T18:29:00.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.727+0000 7f3f886a7b80 10 client.0 _ll_register_callbacks cb 0x55c6ddce8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddddef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddddef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34676/0 (socket says 172.21.15.31:34676) 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 -- 172.21.15.31:0/1502910807 learned_addr learned my addr 172.21.15.31:0/1502910807 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f79ab4700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6dddde840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 -- 172.21.15.31:0/1502910807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 msgr2=0x55c6ddde1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 -- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 msgr2=0x55c6dddde840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6dddde840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 -- 172.21.15.31:0/1502910807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f74112320 con 0x55c6ddde1b50 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddddef80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00ef00 tx=0x7f3f6c009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdb558854695e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f5dffb700 1 -- 172.21.15.31:0/1502910807 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0c5060 con 0x55c6ddde1b50 2023-12-14T18:29:00.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.728+0000 7f3f5dffb700 1 -- 172.21.15.31:0/1502910807 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f6c084040 con 0x55c6ddde1b50 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6dddca2b0 con 0x55c6ddde1b50 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f5dffb700 1 -- 172.21.15.31:0/1502910807 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f6c004ad0 con 0x55c6ddde1b50 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 10 client.15885 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c6ddcef170 con 0x55c6ddde1b50 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f5dffb700 1 -- 172.21.15.31:0/1502910807 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6dddca2b0 con 0x55c6ddde1b50 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f5dffb700 1 -- 172.21.15.31:0/1502910807 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3f6c0ca330 con 0x55c6ddde1b50 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 10 client.15885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 20 client.15885 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dddf10c0 0x55c6dddf34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 4 client.15885 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.729+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6ddd21ce0 con 0x55c6dddf10c0 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.730+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dddf10c0 0x55c6dddf34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.730+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dddf10c0 0x55c6dddf34d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa258be2857329b2 server_cookie=6bc897e0cdb57f32 in_seq=0 out_seq=0 2023-12-14T18:29:00.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.730+0000 7f3f5dffb700 10 client.15885 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f5dffb700 1 -- 172.21.15.31:0/1502910807 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6ddd21ce0 con 0x55c6dddf10c0 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f5dffb700 10 client.15885 handle_command_reply: tid=0 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 client.15885 shutdown 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 msgr2=0x55c6ddddef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddddef80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00ef00 tx=0x7f3f6c009420 comp rx=0 tx=0).stop 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 shutdown_connections 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddddef80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dddf10c0 0x55c6dddf34d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/1502910807 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6dddde840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f5dffb700 0 client.15885 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 >> 172.21.15.31:0/1502910807 conn(0x55c6dddccca0 msgr2=0x55c6dddd23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 shutdown_connections 2023-12-14T18:29:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.731+0000 7f3f886a7b80 1 -- 172.21.15.31:0/1502910807 wait complete. 2023-12-14T18:29:00.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.732+0000 7f3f886a7b80 20 client.15885 trim_cache size 0 max 16384 2023-12-14T18:29:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 Processor -- start 2023-12-14T18:29:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 -- start start 2023-12-14T18:29:00.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 0x55c6ddde1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6ddde1b50 0x55c6dddd9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dddd75e0 con 0x55c6dddd6380 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dddda6b0 con 0x55c6ddde1b50 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dddd7d30 con 0x55c6dddd6c90 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.746+0000 7f3f7a2b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 0x55c6ddde1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 0x55c6ddde1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34682/0 (socket says 172.21.15.31:34682) 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/471485559 learned_addr learned my addr 172.21.15.31:0/471485559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f79ab4700 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6ddde1b50 0x55c6dddd9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/471485559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 msgr2=0x55c6ddde1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/471485559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6ddde1b50 msgr2=0x55c6dddd9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6ddde1b50 0x55c6dddd9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/471485559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6ddceeec0 con 0x55c6dddd6380 2023-12-14T18:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 0x55c6ddde1050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f700058e0 tx=0x7f3f70022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae792c2270d12c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f657fa700 1 -- 172.21.15.31:0/471485559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7002c040 con 0x55c6dddd6380 2023-12-14T18:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f657fa700 1 -- 172.21.15.31:0/471485559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7002d040 con 0x55c6dddd6380 2023-12-14T18:29:00.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.747+0000 7f3f657fa700 1 -- 172.21.15.31:0/471485559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70004260 con 0x55c6dddd6380 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 -- 172.21.15.31:0/471485559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 msgr2=0x55c6ddde1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 0x55c6ddde1050 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f700058e0 tx=0x7f3f70022000 comp rx=0 tx=0).stop 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 -- 172.21.15.31:0/471485559 shutdown_connections 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6ddde1b50 0x55c6dddd9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6dddd6380 0x55c6ddde1050 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/471485559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6c90 0x55c6ddde1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 -- 172.21.15.31:0/471485559 >> 172.21.15.31:0/471485559 conn(0x55c6dddccca0 msgr2=0x55c6ddded8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 -- 172.21.15.31:0/471485559 shutdown_connections 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.748+0000 7f3f886a7b80 1 -- 172.21.15.31:0/471485559 wait complete. 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 -- start start 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 Processor -- start 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6ddde6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddde7750 con 0x55c6ddde1b50 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6ddde78c0 con 0x55c6dddd6c90 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dde44f50 con 0x55c6dddd6380 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f886a7b80 10 client.0 _ll_register_callbacks cb 0x7f3f700037f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f7aab6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34684/0 (socket says 172.21.15.31:34684) 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f7aab6700 1 -- 172.21.15.31:0/300703201 learned_addr learned my addr 172.21.15.31:0/300703201 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.749+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f79ab4700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6ddde6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 msgr2=0x55c6ddde6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6ddde6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/300703201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 msgr2=0x55c6ddde7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7a2b5700 1 -- 172.21.15.31:0/300703201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7410fb00 con 0x55c6dddd6380 2023-12-14T18:29:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7aab6700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f7a2b5700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde8750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f70037040 tx=0x7f3f70003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18ffa6e80c7b2c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f5dffb700 1 -- 172.21.15.31:0/300703201 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7000b990 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f5dffb700 1 -- 172.21.15.31:0/300703201 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f7000baf0 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.750+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f6c099990 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f5dffb700 1 -- 172.21.15.31:0/300703201 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f7000bed0 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 10 client.25747 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c6ddcd59a0 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f5dffb700 1 -- 172.21.15.31:0/300703201 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6c099990 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f5dffb700 1 -- 172.21.15.31:0/300703201 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3f70046080 con 0x55c6dddd6380 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 10 client.25747 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 10 client.25747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 20 client.25747 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dde46150 0x55c6dde48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 4 client.25747 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6ddd21a00 con 0x55c6dde46150 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.751+0000 7f3f79ab4700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dde46150 0x55c6dde48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f79ab4700 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dde46150 0x55c6dde48530 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b878938a865d4ff2 server_cookie=a7cb5fa612be514a in_seq=0 out_seq=0 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f5dffb700 10 client.25747 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f5dffb700 1 -- 172.21.15.31:0/300703201 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6ddd21a00 con 0x55c6dde46150 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f5dffb700 10 client.25747 handle_command_reply: tid=0 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 client.25747 shutdown 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 msgr2=0x55c6ddde8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde8750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f70037040 tx=0x7f3f70003c70 comp rx=0 tx=0).stop 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 shutdown_connections 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c6ddde1b50 0x55c6ddde7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55c6dde46150 0x55c6dde48530 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c6dddd6380 0x55c6ddde8750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 --2- 172.21.15.31:0/300703201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c6dddd6c90 0x55c6ddde6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f5dffb700 0 client.25747 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 >> 172.21.15.31:0/300703201 conn(0x55c6dddccca0 msgr2=0x55c6dddd9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 shutdown_connections 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.752+0000 7f3f886a7b80 1 -- 172.21.15.31:0/300703201 wait complete. 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f886a7b80 20 client.25747 trim_cache size 0 max 16384 2023-12-14T18:29:00.759 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f600a13e0 msgr2=0x7f3f600a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f600a13e0 0x7f3f600a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70005fb0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 msgr2=0x7f3f7419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 0x7f3f7419cf40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00ef30 tx=0x7f3f6c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 shutdown_connections 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f741076f0 0x7f3f7419cf40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f741026b0 0x7f3f741985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f600a13e0 0x7f3f600a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 --2- 172.21.15.31:0/769682812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f74103050 0x7f3f74198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 >> 172.21.15.31:0/769682812 conn(0x7f3f740fdf60 msgr2=0x7f3f740feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:00.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 shutdown_connections 2023-12-14T18:29:00.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:00.753+0000 7f3f7c519700 1 -- 172.21.15.31:0/769682812 wait complete. 2023-12-14T18:29:00.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:29:00.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:00.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:00.933 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:00.933 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:00.933 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 23 -lt 100 ']' 2023-12-14T18:29:00.933 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 23 + 1 2023-12-14T18:29:00.998 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=24 2023-12-14T18:29:00.998 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:01.054 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:01.054 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:01.055 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:01.055 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:01.056 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:01.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:01 smithi031 ceph-mon[42085]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:01.727 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:01.727 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:01.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:01 smithi176 ceph-mon[43506]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:01 smithi204 ceph-mon[43866]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 18 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:01.851 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:01.851 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:01.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:01.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:01.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 24 -lt 100 ']' 2023-12-14T18:29:01.873 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 24 + 1 2023-12-14T18:29:01.931 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=25 2023-12-14T18:29:01.931 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:01.982 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:01.982 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:01.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:01.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:01.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:02.619 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:02.619 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:02.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:02 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:29:02.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:02 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:29:02.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:02 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:29:02.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:02 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:29:02.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:02 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:29:02.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:02 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:29:02.750 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:02.750 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:02.791 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:02.791 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:02.791 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 25 -lt 100 ']' 2023-12-14T18:29:02.792 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 25 + 1 2023-12-14T18:29:02.822 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=26 2023-12-14T18:29:02.822 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:02.828 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:02.828 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:02.830 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:02.830 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:02.830 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:02.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:02 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:29:02.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:02 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:29:02.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:02 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:29:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:03 smithi176 ceph-mon[43506]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:03 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:29:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:03 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:29:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:03 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:29:03.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:03 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:29:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:03 smithi204 ceph-mon[43866]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:03 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:29:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:03 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:29:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:03 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:29:03.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:03 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:29:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:03 smithi031 ceph-mon[42085]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:03 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:29:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:03 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:29:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:03 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:29:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:03 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:29:04.060 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:04.060 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:04.306 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:04.306 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:04.340 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:04.341 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:04.341 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 26 -lt 100 ']' 2023-12-14T18:29:04.341 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 26 + 1 2023-12-14T18:29:04.390 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=27 2023-12-14T18:29:04.390 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:04.442 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:04.443 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:04.444 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:04.444 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:04.444 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:04.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.741+0000 7f13b8a40700 1 -- 172.21.15.31:0/3005540322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40677b0 msgr2=0x7f13b40ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.741+0000 7f13b8a40700 1 --2- 172.21.15.31:0/3005540322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40677b0 0x7f13b40ff470 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009990 tx=0x7f13a4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 -- 172.21.15.31:0/143607508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 msgr2=0x7f3eec103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/143607508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 0x7f3eec103d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009920 tx=0x7f3ee0009c30 comp rx=0 tx=0).stop 2023-12-14T18:29:04.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 -- 172.21.15.31:0/3005540322 shutdown_connections 2023-12-14T18:29:04.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 --2- 172.21.15.31:0/3005540322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b410cbe0 0x7f13b410efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 --2- 172.21.15.31:0/3005540322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b40677b0 0x7f13b40ff470 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 --2- 172.21.15.31:0/3005540322 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b40ff9b0 0x7f13b40ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 -- 172.21.15.31:0/3005540322 >> 172.21.15.31:0/3005540322 conn(0x7f13b4074a60 msgr2=0x7f13b4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.745 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 -- 172.21.15.31:0/143607508 shutdown_connections 2023-12-14T18:29:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/143607508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec104440 0x7f3eec10c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/143607508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec0ff400 0x7f3eec0ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/143607508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 0x7f3eec103d60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.746 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 -- 172.21.15.31:0/143607508 >> 172.21.15.31:0/143607508 conn(0x7f3eec075fa0 msgr2=0x7f3eec0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 -- 172.21.15.31:0/3005540322 shutdown_connections 2023-12-14T18:29:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 -- 172.21.15.31:0/143607508 shutdown_connections 2023-12-14T18:29:04.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f13b8a40700 1 -- 172.21.15.31:0/3005540322 wait complete. 2023-12-14T18:29:04.747 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.742+0000 7f3ef12ef700 1 -- 172.21.15.31:0/143607508 wait complete. 2023-12-14T18:29:04.748 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.743+0000 7f3ef12ef700 1 Processor -- start 2023-12-14T18:29:04.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.743+0000 7f13b8a40700 1 Processor -- start 2023-12-14T18:29:04.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.743+0000 7f13b8a40700 1 -- start start 2023-12-14T18:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.743+0000 7f3ef12ef700 1 -- start start 2023-12-14T18:29:04.749 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3ef12ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 0x7f3eec194320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b8a40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 0x7f13b4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.750 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3ef12ef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 0x7f3eec194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b8a40700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b40ff9b0 0x7f13b4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3ef12ef700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec104440 0x7f3eec198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3ef12ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec199300 con 0x7f3eec0ff400 2023-12-14T18:29:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3ef12ef700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec199470 con 0x7f3eec104440 2023-12-14T18:29:04.751 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3ef12ef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec199630 con 0x7f3eec0ffda0 2023-12-14T18:29:04.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b8a40700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b410cbe0 0x7f13b419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b8a40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d4e0 con 0x7f13b40677b0 2023-12-14T18:29:04.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b8a40700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d650 con 0x7f13b40ff9b0 2023-12-14T18:29:04.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b8a40700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419d810 con 0x7f13b410cbe0 2023-12-14T18:29:04.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 0x7f13b4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 0x7f13b4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53888/0 (socket says 172.21.15.31:53888) 2023-12-14T18:29:04.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b259c700 1 -- 172.21.15.31:0/1806768149 learned_addr learned my addr 172.21.15.31:0/1806768149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 0x7f3eec194320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 0x7f3eec194320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53886/0 (socket says 172.21.15.31:53886) 2023-12-14T18:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec104440 0x7f3eec198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3eeaffd700 1 -- 172.21.15.31:0/1757358629 learned_addr learned my addr 172.21.15.31:0/1757358629 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 0x7f3eec194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.744+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b410cbe0 0x7f13b419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b40ff9b0 0x7f13b4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeb7fe700 1 -- 172.21.15.31:0/1757358629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 msgr2=0x7f3eec194860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 0x7f3eec194860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeb7fe700 1 -- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 msgr2=0x7f3eec194320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 0x7f3eec194320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeb7fe700 1 -- 172.21.15.31:0/1757358629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0009520 con 0x7f3eec104440 2023-12-14T18:29:04.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 0x7f3eec194320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:04.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b2d9d700 1 -- 172.21.15.31:0/1806768149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b40ff9b0 msgr2=0x7f13b4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b40ff9b0 0x7f13b4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b2d9d700 1 -- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 msgr2=0x7f13b4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 0x7f13b4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b2d9d700 1 -- 172.21.15.31:0/1806768149 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40095f0 con 0x7f13b410cbe0 2023-12-14T18:29:04.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b259c700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 0x7f13b4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:04.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec104440 0x7f3eec198cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400e940 tx=0x7f3ee400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b410cbe0 0x7f13b419cea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00ef90 tx=0x7f13ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3ed3fff700 1 -- 172.21.15.31:0/1757358629 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004220 con 0x7f3eec104440 2023-12-14T18:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3ed3fff700 1 -- 172.21.15.31:0/1757358629 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee4004cf0 con 0x7f3eec104440 2023-12-14T18:29:04.758 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec10e1c0 con 0x7f3eec104440 2023-12-14T18:29:04.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13a37fe700 1 -- 172.21.15.31:0/1806768149 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac018070 con 0x7f13b410cbe0 2023-12-14T18:29:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3ed3fff700 1 -- 172.21.15.31:0/1757358629 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4005700 con 0x7f3eec104440 2023-12-14T18:29:04.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13a37fe700 1 -- 172.21.15.31:0/1806768149 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac00f040 con 0x7f13b410cbe0 2023-12-14T18:29:04.760 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec10e030 con 0x7f3eec104440 2023-12-14T18:29:04.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.745+0000 7f13a37fe700 1 -- 172.21.15.31:0/1806768149 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac00cc30 con 0x7f13b410cbe0 2023-12-14T18:29:04.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.746+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b4112490 con 0x7f13b410cbe0 2023-12-14T18:29:04.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.746+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4112300 con 0x7f13b410cbe0 2023-12-14T18:29:04.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.749+0000 7f13a37fe700 1 -- 172.21.15.31:0/1806768149 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f13ac003a00 con 0x7f13b410cbe0 2023-12-14T18:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.749+0000 7f3ed3fff700 1 -- 172.21.15.31:0/1757358629 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f3ee4004380 con 0x7f3eec104440 2023-12-14T18:29:04.761 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.750+0000 7f3ed3fff700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3ed40a13e0 0x7f3ed40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.750+0000 7f13a37fe700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f139c0a1460 0x7f139c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.750+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3ed40a13e0 0x7f3ed40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.762 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.750+0000 7f3ed3fff700 1 -- 172.21.15.31:0/1757358629 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3ee4014070 con 0x7f3eec104440 2023-12-14T18:29:04.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.750+0000 7f13b259c700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f139c0a1460 0x7f139c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.750+0000 7f13a37fe700 1 -- 172.21.15.31:0/1806768149 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f13ac0cce40 con 0x7f13b410cbe0 2023-12-14T18:29:04.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.751+0000 7f13b259c700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f139c0a1460 0x7f139c0a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13a400b2e0 tx=0x7f13a4005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.751+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3ed40a13e0 0x7f3ed40a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0095a0 tx=0x7f3edc00d040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 Processor -- start 2023-12-14T18:29:04.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f13c4bceb80 1 Processor -- start 2023-12-14T18:29:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 -- start start 2023-12-14T18:29:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f5be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f5c370 0x55ca23f5c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f670f0 0x55ca23f694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23e46580 con 0x55ca23db9430 2023-12-14T18:29:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23eb5810 con 0x55ca23f670f0 2023-12-14T18:29:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3efd47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23eaeca0 con 0x55ca23f5c370 2023-12-14T18:29:04.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f13c4bceb80 1 -- start start 2023-12-14T18:29:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f5be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f5be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53904/0 (socket says 172.21.15.31:53904) 2023-12-14T18:29:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3eeaffd700 1 -- 172.21.15.31:0/1079755901 learned_addr learned my addr 172.21.15.31:0/1079755901 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.811+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f5c370 0x55ca23f5c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 0x55ed6b960750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f670f0 0x55ca23f694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960c90 0x55ed6b96b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b96df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b84a5e0 con 0x55ed6b96bb50 2023-12-14T18:29:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b8b9870 con 0x55ed6b960380 2023-12-14T18:29:04.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b8b2d00 con 0x55ed6b960c90 2023-12-14T18:29:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeaffd700 1 -- 172.21.15.31:0/1079755901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f5c370 msgr2=0x55ca23f5c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f5c370 0x55ca23f5c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeaffd700 1 -- 172.21.15.31:0/1079755901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f670f0 msgr2=0x55ca23f694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f670f0 0x55ca23f694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.819 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeaffd700 1 -- 172.21.15.31:0/1079755901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca23d30350 con 0x55ca23db9430 2023-12-14T18:29:04.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b259c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 0x55ed6b960750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b96df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b259c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 0x55ed6b960750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34872/0 (socket says 172.21.15.31:34872) 2023-12-14T18:29:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b259c700 1 -- 172.21.15.31:0/192888611 learned_addr learned my addr 172.21.15.31:0/192888611 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b96df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53914/0 (socket says 172.21.15.31:53914) 2023-12-14T18:29:04.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960c90 0x55ed6b96b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f5be30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0108a0 tx=0x7f3edc00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f143d9a776016a88 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 -- 172.21.15.31:0/192888611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960c90 msgr2=0x55ed6b96b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960c90 0x55ed6b96b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 -- 172.21.15.31:0/192888611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 msgr2=0x55ed6b960750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 0x55ed6b960750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b2d9d700 1 -- 172.21.15.31:0/192888611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed6b7343b0 con 0x55ed6b96bb50 2023-12-14T18:29:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.812+0000 7f13b259c700 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 0x55ed6b960750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:04.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b96df30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ca080 tx=0x7f13ac009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1eac2fc8fe2999 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f3ed17fa700 1 -- 172.21.15.31:0/1079755901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00b150 con 0x55ca23db9430 2023-12-14T18:29:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f3ed17fa700 1 -- 172.21.15.31:0/1079755901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3edc011040 con 0x55ca23db9430 2023-12-14T18:29:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f3ed17fa700 1 -- 172.21.15.31:0/1079755901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc016d20 con 0x55ca23db9430 2023-12-14T18:29:04.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f13a0ff9700 1 -- 172.21.15.31:0/192888611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac09d9f0 con 0x55ed6b96bb50 2023-12-14T18:29:04.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f13a0ff9700 1 -- 172.21.15.31:0/192888611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac0a1d80 con 0x55ed6b96bb50 2023-12-14T18:29:04.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.813+0000 7f13a0ff9700 1 -- 172.21.15.31:0/192888611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac099110 con 0x55ed6b96bb50 2023-12-14T18:29:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 -- 172.21.15.31:0/1079755901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 msgr2=0x55ca23f5be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f5be30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0108a0 tx=0x7f3edc00b050 comp rx=0 tx=0).stop 2023-12-14T18:29:04.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 -- 172.21.15.31:0/192888611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 msgr2=0x55ed6b96df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b96df30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ca080 tx=0x7f13ac009f90 comp rx=0 tx=0).stop 2023-12-14T18:29:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 -- 172.21.15.31:0/1079755901 shutdown_connections 2023-12-14T18:29:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f670f0 0x55ca23f694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f5be30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 --2- 172.21.15.31:0/1079755901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f5c370 0x55ca23f5c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 -- 172.21.15.31:0/1079755901 >> 172.21.15.31:0/1079755901 conn(0x55ca23f52890 msgr2=0x55ca23f58200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 -- 172.21.15.31:0/192888611 shutdown_connections 2023-12-14T18:29:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b96df30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960380 0x55ed6b960750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/192888611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960c90 0x55ed6b96b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 -- 172.21.15.31:0/192888611 >> 172.21.15.31:0/192888611 conn(0x55ed6b956ca0 msgr2=0x55ed6b95c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 -- 172.21.15.31:0/1079755901 shutdown_connections 2023-12-14T18:29:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 -- 172.21.15.31:0/1079755901 wait complete. 2023-12-14T18:29:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 -- 172.21.15.31:0/192888611 shutdown_connections 2023-12-14T18:29:04.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 -- 172.21.15.31:0/192888611 wait complete. 2023-12-14T18:29:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f3efd47db80 1 -- start start 2023-12-14T18:29:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.814+0000 7f13c4bceb80 1 -- start start 2023-12-14T18:29:04.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 Processor -- start 2023-12-14T18:29:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 Processor -- start 2023-12-14T18:29:04.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b96b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 0x55ed6b968810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f66a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f63dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23e4af30 con 0x55ca23db9430 2023-12-14T18:29:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23e4abd0 con 0x55ca23f5c370 2023-12-14T18:29:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23d32640 con 0x55ca23f670f0 2023-12-14T18:29:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3efd47db80 10 client.0 _ll_register_callbacks cb 0x55ca23e6e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b968e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b84ef90 con 0x55ed6b96bb50 2023-12-14T18:29:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b84ec30 con 0x55ed6b960c90 2023-12-14T18:29:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b7366a0 con 0x55ed6b960380 2023-12-14T18:29:04.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13c4bceb80 10 client.0 _ll_register_callbacks cb 0x55ed6b872910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f66a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f66a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53926/0 (socket says 172.21.15.31:53926) 2023-12-14T18:29:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeaffd700 1 -- 172.21.15.31:0/2481277474 learned_addr learned my addr 172.21.15.31:0/2481277474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f63dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13b1d9b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 0x55ed6b968810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13b2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b968e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13b1d9b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 0x55ed6b968810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34884/0 (socket says 172.21.15.31:34884) 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13b1d9b700 1 -- 172.21.15.31:0/3234871039 learned_addr learned my addr 172.21.15.31:0/3234871039 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.815+0000 7f13b259c700 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b96b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13b1d9b700 1 -- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 msgr2=0x55ed6b96b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b96b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13b1d9b700 1 -- 172.21.15.31:0/3234871039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 msgr2=0x55ed6b968e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b968e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13b1d9b700 1 -- 172.21.15.31:0/3234871039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4112300 con 0x55ed6b960c90 2023-12-14T18:29:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeb7fe700 1 -- 172.21.15.31:0/2481277474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 msgr2=0x55ca23f63dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f63dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeb7fe700 1 -- 172.21.15.31:0/2481277474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 msgr2=0x55ca23f66a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f66a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeb7fe700 1 -- 172.21.15.31:0/2481277474 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc010610 con 0x55ca23f670f0 2023-12-14T18:29:04.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 0x55ed6b968810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13a80127d0 tx=0x7f13a8012ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7e7ca97e5afb57b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f1399ffb700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:29:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f643b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40c9920 tx=0x7f3ee4099710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6fd67173127d51 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eceff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:29:04.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f1399ffb700 1 -- 172.21.15.31:0/3234871039 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8017070 con 0x55ed6b960c90 2023-12-14T18:29:04.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f1399ffb700 1 -- 172.21.15.31:0/3234871039 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f13a8004830 con 0x55ed6b960c90 2023-12-14T18:29:04.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed6b9542b0 con 0x55ed6b960c90 2023-12-14T18:29:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.816+0000 7f3eceff5700 1 -- 172.21.15.31:0/2481277474 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40a1390 con 0x55ca23f670f0 2023-12-14T18:29:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3eceff5700 1 -- 172.21.15.31:0/2481277474 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3ee409d200 con 0x55ca23f670f0 2023-12-14T18:29:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca23f4fea0 con 0x55ca23f670f0 2023-12-14T18:29:04.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f1399ffb700 1 -- 172.21.15.31:0/3234871039 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a80049f0 con 0x55ed6b960c90 2023-12-14T18:29:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 10 client.25760 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:04.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ed6b879170 con 0x55ed6b960c90 2023-12-14T18:29:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3eceff5700 1 -- 172.21.15.31:0/2481277474 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee40a1620 con 0x55ca23f670f0 2023-12-14T18:29:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 10 client.25768 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ca23e75110 con 0x55ca23f670f0 2023-12-14T18:29:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f1399ffb700 1 -- 172.21.15.31:0/3234871039 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ed6b9542b0 con 0x55ed6b960c90 2023-12-14T18:29:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3eceff5700 1 -- 172.21.15.31:0/2481277474 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca23f4fea0 con 0x55ca23f670f0 2023-12-14T18:29:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f1399ffb700 1 -- 172.21.15.31:0/3234871039 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13a801c5e0 con 0x55ed6b960c90 2023-12-14T18:29:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 20 client.25760 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3eceff5700 1 -- 172.21.15.31:0/2481277474 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ee40c9c90 con 0x55ca23f670f0 2023-12-14T18:29:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b97b600 0x55ed6b97da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 4 client.25760 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:04.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed6b8abce0 con 0x55ed6b97b600 2023-12-14T18:29:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 10 client.25768 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 10 client.25768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 20 client.25768 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23f76b30 0x55ca23f78f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 4 client.25768 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.817+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca23ea7c80 con 0x55ca23f76b30 2023-12-14T18:29:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.818+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23f76b30 0x55ca23f78f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.818+0000 7f13b259c700 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b97b600 0x55ed6b97da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.818+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23f76b30 0x55ca23f78f10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf41244f1e6bfeda server_cookie=fdf751997842f983 in_seq=0 out_seq=0 2023-12-14T18:29:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.818+0000 7f3eceff5700 10 client.25768 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f1399ffb700 10 client.25760 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f13b259c700 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b97b600 0x55ed6b97da10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae16dd2585c8dd1a server_cookie=6d8580c5797e022d in_seq=0 out_seq=0 2023-12-14T18:29:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f3eceff5700 1 -- 172.21.15.31:0/2481277474 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca23ea7c80 con 0x55ca23f76b30 2023-12-14T18:29:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f3eceff5700 10 client.25768 handle_command_reply: tid=0 2023-12-14T18:29:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f3efd47db80 1 client.25768 shutdown 2023-12-14T18:29:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f1399ffb700 1 -- 172.21.15.31:0/3234871039 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed6b8abce0 con 0x55ed6b97b600 2023-12-14T18:29:04.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f1399ffb700 10 client.25760 handle_command_reply: tid=0 2023-12-14T18:29:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 msgr2=0x55ca23f643b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f643b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ee40c9920 tx=0x7f3ee4099710 comp rx=0 tx=0).stop 2023-12-14T18:29:04.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.819+0000 7f13c4bceb80 1 client.25760 shutdown 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 shutdown_connections 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23f76b30 0x55ca23f78f10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f643b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23db9430 0x55ca23f66a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2481277474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f63dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3eceff5700 0 client.25768 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 >> 172.21.15.31:0/2481277474 conn(0x55ca23f52890 msgr2=0x55ca23f57d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 shutdown_connections 2023-12-14T18:29:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 msgr2=0x55ed6b968810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 0x55ed6b968810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13a80127d0 tx=0x7f13a8012ae0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 1 -- 172.21.15.31:0/2481277474 wait complete. 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 shutdown_connections 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b97b600 0x55ed6b97da10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b96bb50 0x55ed6b968e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b96b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/3234871039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b960c90 0x55ed6b968810 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 >> 172.21.15.31:0/3234871039 conn(0x55ed6b956ca0 msgr2=0x55ed6b95c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f1399ffb700 0 client.25760 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f3efd47db80 20 client.25768 trim_cache size 0 max 16384 2023-12-14T18:29:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 shutdown_connections 2023-12-14T18:29:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 1 -- 172.21.15.31:0/3234871039 wait complete. 2023-12-14T18:29:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.820+0000 7f13c4bceb80 20 client.25760 trim_cache size 0 max 16384 2023-12-14T18:29:04.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.835+0000 7f13c4bceb80 1 Processor -- start 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.835+0000 7f13c4bceb80 1 -- start start 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.835+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b968010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.835+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b97d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b96b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b9613c0 con 0x55ed6b956ca0 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b9615b0 con 0x55ed6b96bb50 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b963730 con 0x55ed6b960380 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b968010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b968010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53932/0 (socket says 172.21.15.31:53932) 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 -- 172.21.15.31:0/2420050172 learned_addr learned my addr 172.21.15.31:0/2420050172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b96b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b97d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 -- 172.21.15.31:0/2420050172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 msgr2=0x55ed6b97d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b97d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 -- 172.21.15.31:0/2420050172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 msgr2=0x55ed6b96b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b96b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 -- 172.21.15.31:0/2420050172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed6b878ec0 con 0x55ed6b956ca0 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b97d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.836+0000 7f13b259c700 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b968010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f13a40252d0 tx=0x7f13a4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c443c019692a41 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.837+0000 7f13a17fa700 1 -- 172.21.15.31:0/2420050172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a402d030 con 0x55ed6b956ca0 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.837+0000 7f13a17fa700 1 -- 172.21.15.31:0/2420050172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13a402e040 con 0x55ed6b956ca0 2023-12-14T18:29:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.837+0000 7f13a17fa700 1 -- 172.21.15.31:0/2420050172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004b60 con 0x55ed6b956ca0 2023-12-14T18:29:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.837+0000 7f3efd47db80 1 Processor -- start 2023-12-14T18:29:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.837+0000 7f13c4bceb80 1 -- 172.21.15.31:0/2420050172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 msgr2=0x55ed6b968010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.837+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b968010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f13a40252d0 tx=0x7f13a4023000 comp rx=0 tx=0).stop 2023-12-14T18:29:04.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 -- 172.21.15.31:0/2420050172 shutdown_connections 2023-12-14T18:29:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b96b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b968010 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/2420050172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b97d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 -- 172.21.15.31:0/2420050172 >> 172.21.15.31:0/2420050172 conn(0x55ed6b96e200 msgr2=0x55ed6b976e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 -- start start 2023-12-14T18:29:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 -- 172.21.15.31:0/2420050172 shutdown_connections 2023-12-14T18:29:04.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 -- 172.21.15.31:0/2420050172 wait complete. 2023-12-14T18:29:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23db9430 0x55ca23f63590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f670f0 0x55ca23f63d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23f5dd00 con 0x55ca23f670f0 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23f4fa70 con 0x55ca23f5c370 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3efd47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23f524d0 con 0x55ca23db9430 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eeb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f670f0 0x55ca23f63d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f66670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34900/0 (socket says 172.21.15.31:34900) 2023-12-14T18:29:04.859 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 -- 172.21.15.31:0/230560358 learned_addr learned my addr 172.21.15.31:0/230560358 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 -- start start 2023-12-14T18:29:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23db9430 0x55ca23f63590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f13c4bceb80 1 Processor -- start 2023-12-14T18:29:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 -- 172.21.15.31:0/230560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23db9430 msgr2=0x55ca23f63590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23db9430 0x55ca23f63590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 -- 172.21.15.31:0/230560358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f670f0 msgr2=0x55ca23f63d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.838+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f670f0 0x55ca23f63d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f3eea7fc700 1 -- 172.21.15.31:0/230560358 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca23e74e60 con 0x55ca23f5c370 2023-12-14T18:29:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f670f0 0x55ca23f63d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b970670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b970bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b9ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b9713c0 con 0x55ed6b956ca0 2023-12-14T18:29:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b971530 con 0x55ed6b96bb50 2023-12-14T18:29:04.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed6b9cea40 con 0x55ed6b960380 2023-12-14T18:29:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f66670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009920 tx=0x7f3ee000f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6bbb53860011476 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13c4bceb80 10 client.0 _ll_register_callbacks cb 0x7f13a8009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b970670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b970bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b970bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:44858/0 (socket says 172.21.15.31:44858) 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b970670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53940/0 (socket says 172.21.15.31:53940) 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 -- 172.21.15.31:0/829255879 learned_addr learned my addr 172.21.15.31:0/829255879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b2d9d700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b9ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f3ed1ffb700 1 -- 172.21.15.31:0/230560358 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000f940 con 0x55ca23f5c370 2023-12-14T18:29:04.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 -- 172.21.15.31:0/829255879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 msgr2=0x55ed6b9ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b9ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 -- 172.21.15.31:0/829255879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 msgr2=0x55ed6b970670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b970670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b1d9b700 1 -- 172.21.15.31:0/829255879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a4025040 con 0x55ed6b960380 2023-12-14T18:29:04.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f13b259c700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b970670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f3ed1ffb700 1 -- 172.21.15.31:0/230560358 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ee0021040 con 0x55ca23f5c370 2023-12-14T18:29:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.839+0000 7f3ed1ffb700 1 -- 172.21.15.31:0/230560358 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0005670 con 0x55ca23f5c370 2023-12-14T18:29:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f13b1d9b700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b970bb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13a8000c00 tx=0x7f13a8005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c240fd2de85acf65 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f1399ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:29:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f1399ffb700 1 -- 172.21.15.31:0/829255879 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a801cea0 con 0x55ed6b960380 2023-12-14T18:29:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f1399ffb700 1 -- 172.21.15.31:0/829255879 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f13a8000e40 con 0x55ed6b960380 2023-12-14T18:29:04.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13ac0a1d80 con 0x55ed6b960380 2023-12-14T18:29:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f1399ffb700 1 -- 172.21.15.31:0/829255879 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a8017040 con 0x55ed6b960380 2023-12-14T18:29:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f13c4bceb80 10 client.25780 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:04.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ed6b85f9a0 con 0x55ed6b960380 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 -- 172.21.15.31:0/230560358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 msgr2=0x55ca23f66670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f66670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009920 tx=0x7f3ee000f510 comp rx=0 tx=0).stop 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 -- 172.21.15.31:0/230560358 shutdown_connections 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f670f0 0x55ca23f63d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23db9430 0x55ca23f63590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 --2- 172.21.15.31:0/230560358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23f5c370 0x55ca23f66670 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 -- 172.21.15.31:0/230560358 >> 172.21.15.31:0/230560358 conn(0x55ca23f52890 msgr2=0x55ca23f72e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 -- 172.21.15.31:0/230560358 shutdown_connections 2023-12-14T18:29:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f3efd47db80 1 -- 172.21.15.31:0/230560358 wait complete. 2023-12-14T18:29:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.840+0000 7f1399ffb700 1 -- 172.21.15.31:0/829255879 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13ac0a1d80 con 0x55ed6b960380 2023-12-14T18:29:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f1399ffb700 1 -- 172.21.15.31:0/829255879 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13a8032070 con 0x55ed6b960380 2023-12-14T18:29:04.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13c4bceb80 10 client.25780 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13c4bceb80 10 client.25780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13c4bceb80 20 client.25780 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b9d0100 0x55ed6b9d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13c4bceb80 4 client.25780 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:04.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed6b8aba00 con 0x55ed6b9d0100 2023-12-14T18:29:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 -- start start 2023-12-14T18:29:04.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13b259c700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b9d0100 0x55ed6b9d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 Processor -- start 2023-12-14T18:29:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23db9430 0x55ca23f6ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 0x55ca23f6c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f6c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23f6cdd0 con 0x55ca23f5c370 2023-12-14T18:29:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23f6cf40 con 0x55ca23db9430 2023-12-14T18:29:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f3efd47db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca23fd31b0 con 0x55ca23f670f0 2023-12-14T18:29:04.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f13b259c700 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b9d0100 0x55ed6b9d2510 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e8845deef62ea0 server_cookie=ea09ef823c7283c7 in_seq=0 out_seq=0 2023-12-14T18:29:04.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.841+0000 7f1399ffb700 10 client.25780 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3efd47db80 10 client.0 _ll_register_callbacks cb 0x7f3ee4098b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 0x55ca23f6c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 0x55ca23f6c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53956/0 (socket says 172.21.15.31:53956) 2023-12-14T18:29:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 -- 172.21.15.31:0/2701346263 learned_addr learned my addr 172.21.15.31:0/2701346263 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eeaffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23db9430 0x55ca23f6ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f1399ffb700 1 -- 172.21.15.31:0/829255879 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed6b8aba00 con 0x55ed6b9d0100 2023-12-14T18:29:04.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f1399ffb700 10 client.25780 handle_command_reply: tid=0 2023-12-14T18:29:04.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f13c4bceb80 1 client.25780 shutdown 2023-12-14T18:29:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f6c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 -- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 msgr2=0x55ca23f6c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.875 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f6c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 -- 172.21.15.31:0/2701346263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23db9430 msgr2=0x55ca23f6ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23db9430 0x55ca23f6ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 -- 172.21.15.31:0/2701346263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca23e75110 con 0x55ca23f5c370 2023-12-14T18:29:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eeb7fe700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f6c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:29:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 msgr2=0x55ed6b970bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b970bb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13a8000c00 tx=0x7f13a8005e30 comp rx=0 tx=0).stop 2023-12-14T18:29:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 shutdown_connections 2023-12-14T18:29:04.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ed6b9d0100 0x55ed6b9d2510 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ed6b96bb50 0x55ed6b9ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ed6b956ca0 0x55ed6b970670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f1399ffb700 0 client.25780 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 1 --2- 172.21.15.31:0/829255879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ed6b960380 0x55ed6b970bb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 >> 172.21.15.31:0/829255879 conn(0x55ed6b96e200 msgr2=0x55ed6b97d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eea7fc700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 0x55ca23f6c100 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009c70 tx=0x7f3ee0005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a068f50613bb9777 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.842+0000 7f3eceff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:04.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 shutdown_connections 2023-12-14T18:29:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3eceff5700 1 -- 172.21.15.31:0/2701346263 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000b970 con 0x55ca23f5c370 2023-12-14T18:29:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3eceff5700 1 -- 172.21.15.31:0/2701346263 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3ee0005040 con 0x55ca23f5c370 2023-12-14T18:29:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3edc011040 con 0x55ca23f5c370 2023-12-14T18:29:04.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 1 -- 172.21.15.31:0/829255879 wait complete. 2023-12-14T18:29:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3eceff5700 1 -- 172.21.15.31:0/2701346263 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee0005690 con 0x55ca23f5c370 2023-12-14T18:29:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3efd47db80 10 client.15927 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f3eec10c850 con 0x55ca23f5c370 2023-12-14T18:29:04.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f13c4bceb80 20 client.25780 trim_cache size 0 max 16384 2023-12-14T18:29:04.880 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.843+0000 7f3eceff5700 1 -- 172.21.15.31:0/2701346263 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3edc011040 con 0x55ca23f5c370 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3eceff5700 1 -- 172.21.15.31:0/2701346263 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ee002d030 con 0x55ca23f5c370 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3efd47db80 10 client.15927 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3efd47db80 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3efd47db80 20 client.15927 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23fd44b0 0x55ca23fd68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3efd47db80 4 client.15927 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca23ea79a0 con 0x55ca23fd44b0 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f139c0a1460 msgr2=0x7f139c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f139c0a1460 0x7f139c0a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f13a400b2e0 tx=0x7f13a4005fd0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b410cbe0 msgr2=0x7f13b419cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b410cbe0 0x7f13b419cea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13ac00ef90 tx=0x7f13ac00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 shutdown_connections 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f139c0a1460 0x7f139c0a3910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13b410cbe0 0x7f13b419cea0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13b40677b0 0x7f13b4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 --2- 172.21.15.31:0/1806768149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13b40ff9b0 0x7f13b4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 >> 172.21.15.31:0/1806768149 conn(0x7f13b4074a60 msgr2=0x7f13b410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 shutdown_connections 2023-12-14T18:29:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23fd44b0 0x55ca23fd68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:04.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:04.844+0000 7f13b8a40700 1 -- 172.21.15.31:0/1806768149 wait complete. 2023-12-14T18:29:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.845+0000 7f3eeaffd700 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23fd44b0 0x55ca23fd68c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10703fcab117628c server_cookie=8a0eddbc83c5e725 in_seq=0 out_seq=0 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.845+0000 7f3eceff5700 10 client.15927 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.863+0000 7f3eceff5700 1 -- 172.21.15.31:0/2701346263 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca23ea79a0 con 0x55ca23fd44b0 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.863+0000 7f3eceff5700 10 client.15927 handle_command_reply: tid=0 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.863+0000 7f3efd47db80 1 client.15927 shutdown 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 msgr2=0x55ca23f6c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 0x55ca23f6c100 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009c70 tx=0x7f3ee0005520 comp rx=0 tx=0).stop 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 shutdown_connections 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca23fd44b0 0x55ca23fd68c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca23f670f0 0x55ca23f6c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3eceff5700 0 client.15927 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca23db9430 0x55ca23f6ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 --2- 172.21.15.31:0/2701346263 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca23f5c370 0x55ca23f6c100 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 >> 172.21.15.31:0/2701346263 conn(0x55ca23f52890 msgr2=0x55ca23f665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 shutdown_connections 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 1 -- 172.21.15.31:0/2701346263 wait complete. 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.864+0000 7f3efd47db80 20 client.15927 trim_cache size 0 max 16384 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:29:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:29:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3ed40a13e0 msgr2=0x7f3ed40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3ed40a13e0 0x7f3ed40a3890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0095a0 tx=0x7f3edc00d040 comp rx=0 tx=0).stop 2023-12-14T18:29:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec104440 msgr2=0x7f3eec198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec104440 0x7f3eec198cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400e940 tx=0x7f3ee400ec50 comp rx=0 tx=0).stop 2023-12-14T18:29:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 shutdown_connections 2023-12-14T18:29:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3eec104440 0x7f3eec198cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eec0ff400 0x7f3eec194320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3ed40a13e0 0x7f3ed40a3890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 --2- 172.21.15.31:0/1757358629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec0ffda0 0x7f3eec194860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 >> 172.21.15.31:0/1757358629 conn(0x7f3eec075fa0 msgr2=0x7f3eec0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 shutdown_connections 2023-12-14T18:29:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:04.865+0000 7f3ef12ef700 1 -- 172.21.15.31:0/1757358629 wait complete. 2023-12-14T18:29:05.101 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:05.101 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:05.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:05.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:05.223 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:05.223 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:05.223 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 27 -lt 100 ']' 2023-12-14T18:29:05.223 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 27 + 1 2023-12-14T18:29:05.247 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=28 2023-12-14T18:29:05.247 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:05.281 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:05.281 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:05.283 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:05.283 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:05.283 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:05.710 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:05.710 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:05.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:05 smithi176 ceph-mon[43506]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:05.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:05 smithi204 ceph-mon[43866]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:05.889 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:05.889 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:05.900 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:05.900 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:05.900 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 28 -lt 100 ']' 2023-12-14T18:29:05.900 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 28 + 1 2023-12-14T18:29:05.930 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=29 2023-12-14T18:29:05.930 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:05.955 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:05.955 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:05.956 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:05.956 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:05.956 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:05.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:05 smithi031 ceph-mon[42085]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 38 KiB/s wr, 22 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:05.999 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:29:05.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:29:06.368 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:06.368 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:06.433 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:06.433 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:06.444 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:06.444 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:06.444 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 29 -lt 100 ']' 2023-12-14T18:29:06.444 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 29 + 1 2023-12-14T18:29:06.484 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=30 2023-12-14T18:29:06.484 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:06.531 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:06.531 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:06.532 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:06.532 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:06.532 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:07.026 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:07.027 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:07.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:07.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:07.248 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:07.248 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:07.248 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 30 -lt 100 ']' 2023-12-14T18:29:07.249 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 30 + 1 2023-12-14T18:29:07.305 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=31 2023-12-14T18:29:07.306 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:07.340 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:07.340 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:07.341 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:07.341 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:07.342 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:07.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:07 smithi176 ceph-mon[43506]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:07 smithi204 ceph-mon[43866]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:07.845 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:07 smithi031 ceph-mon[42085]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:08.035 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:08.035 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:08.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:08.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:08.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:08.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:08.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 31 -lt 100 ']' 2023-12-14T18:29:08.214 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 31 + 1 2023-12-14T18:29:08.247 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=32 2023-12-14T18:29:08.247 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:08.298 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:08.298 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:08.299 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:08.299 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:08.299 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 -- 172.21.15.31:0/1722701122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c1043d0 msgr2=0x7f709c10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 --2- 172.21.15.31:0/1722701122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c1043d0 0x7f709c10bfa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f70940099c0 tx=0x7f7094009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 -- 172.21.15.31:0/1722701122 shutdown_connections 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 --2- 172.21.15.31:0/1722701122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c1043d0 0x7f709c10bfa0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 --2- 172.21.15.31:0/1722701122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f709c0ff390 0x7f709c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 --2- 172.21.15.31:0/1722701122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f709c0ffd30 0x7f709c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 -- 172.21.15.31:0/1722701122 >> 172.21.15.31:0/1722701122 conn(0x7f709c075fd0 msgr2=0x7f709c0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 -- 172.21.15.31:0/1722701122 shutdown_connections 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.599+0000 7f70a42a0700 1 -- 172.21.15.31:0/1722701122 wait complete. 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 Processor -- start 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 -- start start 2023-12-14T18:29:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f709c0ff390 0x7f709c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 0x7f709c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f709c1043d0 0x7f709c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19b3c0 con 0x7f709c0ffd30 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19b530 con 0x7f709c0ff390 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a42a0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c19b6f0 con 0x7f709c1043d0 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.600+0000 7f70a183b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 0x7f709c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 0x7f709c196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53978/0 (socket says 172.21.15.31:53978) 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 -- 172.21.15.31:0/2148504585 learned_addr learned my addr 172.21.15.31:0/2148504585 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a283d700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f709c1043d0 0x7f709c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a203c700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f709c0ff390 0x7f709c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 -- 172.21.15.31:0/2148504585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f709c1043d0 msgr2=0x7f709c19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f709c1043d0 0x7f709c19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 -- 172.21.15.31:0/2148504585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f709c0ff390 msgr2=0x7f709c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f709c0ff390 0x7f709c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 -- 172.21.15.31:0/2148504585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70940095f0 con 0x7f709c0ffd30 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70a183b700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 0x7f709c196920 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f708c00e940 tx=0x7f708c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.601+0000 7f70937fe700 1 -- 172.21.15.31:0/2148504585 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c009730 con 0x7f709c0ffd30 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.602+0000 7f70937fe700 1 -- 172.21.15.31:0/2148504585 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f708c010430 con 0x7f709c0ffd30 2023-12-14T18:29:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.602+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c10f1a0 con 0x7f709c0ffd30 2023-12-14T18:29:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.602+0000 7f70937fe700 1 -- 172.21.15.31:0/2148504585 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c00cea0 con 0x7f709c0ffd30 2023-12-14T18:29:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.602+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c10f010 con 0x7f709c0ffd30 2023-12-14T18:29:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.604+0000 7f70937fe700 1 -- 172.21.15.31:0/2148504585 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f708c010810 con 0x7f709c0ffd30 2023-12-14T18:29:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.605+0000 7f70937fe700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f70880a13e0 0x7f70880a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.605+0000 7f70a203c700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f70880a13e0 0x7f70880a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.605+0000 7f70937fe700 1 -- 172.21.15.31:0/2148504585 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f708c014070 con 0x7f709c0ffd30 2023-12-14T18:29:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.606+0000 7f70a203c700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f70880a13e0 0x7f70880a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7098009550 tx=0x7f70980091f0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.688+0000 7f70b042eb80 1 Processor -- start 2023-12-14T18:29:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 -- start start 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2400 0x55f11d8d27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 0x55f11d8dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d7bc660 con 0x55f11d8ddbd0 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d82b8f0 con 0x55f11d8d2400 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70b042eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d824d80 con 0x55f11d8d2d10 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70a283d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 0x55f11d8dffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70a283d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 0x55f11d8dffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53982/0 (socket says 172.21.15.31:53982) 2023-12-14T18:29:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70a283d700 1 -- 172.21.15.31:0/2041215758 learned_addr learned my addr 172.21.15.31:0/2041215758 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70a203c700 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2400 0x55f11d8d27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.689+0000 7f70a183b700 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a203c700 1 -- 172.21.15.31:0/2041215758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2d10 msgr2=0x55f11d8dd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a203c700 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a203c700 1 -- 172.21.15.31:0/2041215758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 msgr2=0x55f11d8dffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a203c700 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 0x55f11d8dffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a203c700 1 -- 172.21.15.31:0/2041215758 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11d6a6430 con 0x55f11d8d2400 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a283d700 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 0x55f11d8dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f70a203c700 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2400 0x55f11d8d27d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7098004600 tx=0x7f7098013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d982814aa9bd03 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f7090dac700 1 -- 172.21.15.31:0/2041215758 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098007da0 con 0x55f11d8d2400 2023-12-14T18:29:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f7090dac700 1 -- 172.21.15.31:0/2041215758 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709801a070 con 0x55f11d8d2400 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.690+0000 7f7090dac700 1 -- 172.21.15.31:0/2041215758 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980156a0 con 0x55f11d8d2400 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 -- 172.21.15.31:0/2041215758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2400 msgr2=0x55f11d8d27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2400 0x55f11d8d27d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7098004600 tx=0x7f7098013000 comp rx=0 tx=0).stop 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 -- 172.21.15.31:0/2041215758 shutdown_connections 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8ddbd0 0x55f11d8dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2400 0x55f11d8d27d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2041215758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 -- 172.21.15.31:0/2041215758 >> 172.21.15.31:0/2041215758 conn(0x55f11d8c8d20 msgr2=0x55f11d8ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 -- 172.21.15.31:0/2041215758 shutdown_connections 2023-12-14T18:29:08.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.691+0000 7f70b042eb80 1 -- 172.21.15.31:0/2041215758 wait complete. 2023-12-14T18:29:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 -- start start 2023-12-14T18:29:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 Processor -- start 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 0x55f11d8dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8ddbd0 0x55f11d8daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d7c1010 con 0x55f11d8d2400 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d7c0cb0 con 0x55f11d8d2d10 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d6a8720 con 0x55f11d8ddbd0 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70b042eb80 10 client.0 _ll_register_callbacks cb 0x55f11d7e4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.692+0000 7f70a203c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 0x55f11d8dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 0x55f11d8dd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53994/0 (socket says 172.21.15.31:53994) 2023-12-14T18:29:08.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 -- 172.21.15.31:0/389790691 learned_addr learned my addr 172.21.15.31:0/389790691 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a283d700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8ddbd0 0x55f11d8daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a183b700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 -- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8ddbd0 msgr2=0x55f11d8daff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8ddbd0 0x55f11d8daff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 -- 172.21.15.31:0/389790691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 msgr2=0x55f11d8da8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8da8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 -- 172.21.15.31:0/389790691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980042e0 con 0x55f11d8d2400 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a183b700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f70a203c700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 0x55f11d8dd600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f70980216f0 tx=0x7f7098021720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938827ddd1c4830c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f7085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:08.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f7085ffb700 1 -- 172.21.15.31:0/389790691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098004c90 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.693+0000 7f7085ffb700 1 -- 172.21.15.31:0/389790691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7098004df0 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f11d8c6330 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f7085ffb700 1 -- 172.21.15.31:0/389790691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7098005050 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 10 client.15939 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f11d7eb1f0 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f7085ffb700 1 -- 172.21.15.31:0/389790691 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f11d8c6330 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f7085ffb700 1 -- 172.21.15.31:0/389790691 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7098031020 con 0x55f11d8d2400 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 10 client.15939 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 20 client.15939 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ed510 0x55f11d8ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 4 client.15939 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.694+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f11d81dd60 con 0x55f11d8ed510 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.695+0000 7f70a183b700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ed510 0x55f11d8ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.695+0000 7f70a183b700 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ed510 0x55f11d8ef8f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c9f5d8b93182f8 server_cookie=302839be812de143 in_seq=0 out_seq=0 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.695+0000 7f7085ffb700 10 client.15939 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f7085ffb700 1 -- 172.21.15.31:0/389790691 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f11d81dd60 con 0x55f11d8ed510 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f7085ffb700 10 client.15939 handle_command_reply: tid=0 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 client.15939 shutdown 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 msgr2=0x55f11d8dd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 0x55f11d8dd600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f70980216f0 tx=0x7f7098021720 comp rx=0 tx=0).stop 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 shutdown_connections 2023-12-14T18:29:08.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ed510 0x55f11d8ef8f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8ddbd0 0x55f11d8daff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8d2400 0x55f11d8dd600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 --2- 172.21.15.31:0/389790691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f7085ffb700 0 client.15939 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 >> 172.21.15.31:0/389790691 conn(0x55f11d8c8d20 msgr2=0x55f11d8ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 shutdown_connections 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 1 -- 172.21.15.31:0/389790691 wait complete. 2023-12-14T18:29:08.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.696+0000 7f70b042eb80 20 client.15939 trim_cache size 0 max 16384 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 Processor -- start 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 -- start start 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d8db610 con 0x55f11d8c8d20 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d8d3db0 con 0x55f11d8d2d10 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.715+0000 7f70b042eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d8e4c50 con 0x55f11d8d2400 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54006/0 (socket says 172.21.15.31:54006) 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 -- 172.21.15.31:0/2326165309 learned_addr learned my addr 172.21.15.31:0/2326165309 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a183b700 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a283d700 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 -- 172.21.15.31:0/2326165309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 msgr2=0x55f11d8dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8dd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 -- 172.21.15.31:0/2326165309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 msgr2=0x55f11d8dd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 -- 172.21.15.31:0/2326165309 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11d7eaf40 con 0x55f11d8c8d20 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70a203c700 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e4710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f70980216c0 tx=0x7f7098015660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5d772146fc5bff server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70915ad700 1 -- 172.21.15.31:0/2326165309 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980040c0 con 0x55f11d8c8d20 2023-12-14T18:29:08.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70915ad700 1 -- 172.21.15.31:0/2326165309 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709800f040 con 0x55f11d8c8d20 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.716+0000 7f70915ad700 1 -- 172.21.15.31:0/2326165309 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098005040 con 0x55f11d8c8d20 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 -- 172.21.15.31:0/2326165309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 msgr2=0x55f11d8e4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e4710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f70980216c0 tx=0x7f7098015660 comp rx=0 tx=0).stop 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 -- 172.21.15.31:0/2326165309 shutdown_connections 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d8dd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e4710 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 --2- 172.21.15.31:0/2326165309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 -- 172.21.15.31:0/2326165309 >> 172.21.15.31:0/2326165309 conn(0x55f11d8e0280 msgr2=0x55f11d8d5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 -- 172.21.15.31:0/2326165309 shutdown_connections 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.717+0000 7f70b042eb80 1 -- 172.21.15.31:0/2326165309 wait complete. 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 -- start start 2023-12-14T18:29:08.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 Processor -- start 2023-12-14T18:29:08.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d941380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d9418c0 con 0x55f11d8c8d20 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d941a00 con 0x55f11d8d2d10 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11d941bc0 con 0x55f11d8d2400 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70b042eb80 10 client.0 _ll_register_callbacks cb 0x7f708c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70a203c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70a203c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54012/0 (socket says 172.21.15.31:54012) 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.718+0000 7f70a203c700 1 -- 172.21.15.31:0/1018327050 learned_addr learned my addr 172.21.15.31:0/1018327050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:08.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a183b700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a283d700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d941380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a203c700 1 -- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 msgr2=0x55f11d8e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a203c700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8e3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a203c700 1 -- 172.21.15.31:0/1018327050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 msgr2=0x55f11d941380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a203c700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d941380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a203c700 1 -- 172.21.15.31:0/1018327050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f11d7eb1f0 con 0x55f11d8c8d20 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70a203c700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e3350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f70980154c0 tx=0x7f7098007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568cd72096cd3045 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f7085ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f7085ffb700 1 -- 172.21.15.31:0/1018327050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980040c0 con 0x55f11d8c8d20 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f7085ffb700 1 -- 172.21.15.31:0/1018327050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7098015e90 con 0x55f11d8c8d20 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f709801a070 con 0x55f11d8c8d20 2023-12-14T18:29:08.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f7085ffb700 1 -- 172.21.15.31:0/1018327050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709802b400 con 0x55f11d8c8d20 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.719+0000 7f70b042eb80 10 client.15945 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f11d7d1a20 con 0x55f11d8c8d20 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f7085ffb700 1 -- 172.21.15.31:0/1018327050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7098052080 con 0x55f11d8c8d20 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f7085ffb700 1 -- 172.21.15.31:0/1018327050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7098031070 con 0x55f11d8c8d20 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 10 client.15945 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 20 client.15945 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ddbd0 0x55f11d944e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 4 client.15945 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:29:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f11d81da80 con 0x55f11d8ddbd0 2023-12-14T18:29:08.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.720+0000 7f70a183b700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ddbd0 0x55f11d944e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:08.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.721+0000 7f70a183b700 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ddbd0 0x55f11d944e60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86866d59acfb426 server_cookie=7b04cada0331f772 in_seq=0 out_seq=0 2023-12-14T18:29:08.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.721+0000 7f7085ffb700 10 client.15945 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:08.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f7085ffb700 1 -- 172.21.15.31:0/1018327050 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f11d81da80 con 0x55f11d8ddbd0 2023-12-14T18:29:08.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f7085ffb700 10 client.15945 handle_command_reply: tid=0 2023-12-14T18:29:08.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 client.15945 shutdown 2023-12-14T18:29:08.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 msgr2=0x55f11d8e3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e3350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f70980154c0 tx=0x7f7098007f60 comp rx=0 tx=0).stop 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 shutdown_connections 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f11d8ddbd0 0x55f11d944e60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f11d8d2d10 0x55f11d941380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f11d8c8d20 0x55f11d8e3350 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f7085ffb700 0 client.15945 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 --2- 172.21.15.31:0/1018327050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f11d8d2400 0x55f11d8e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 >> 172.21.15.31:0/1018327050 conn(0x55f11d8e0280 msgr2=0x55f11d8ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.724+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 shutdown_connections 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70b042eb80 1 -- 172.21.15.31:0/1018327050 wait complete. 2023-12-14T18:29:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70b042eb80 20 client.15945 trim_cache size 0 max 16384 2023-12-14T18:29:08.731 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:29:08.731 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:29:08.731 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "4a49fd00-e9c3-4333-92a1-0bcc288df058", 2023-12-14T18:29:08.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:29:08.731 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f70880a13e0 msgr2=0x7f70880a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f70880a13e0 0x7f70880a3890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7098009550 tx=0x7f70980091f0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 msgr2=0x7f709c196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 0x7f709c196920 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f708c00e940 tx=0x7f708c00ec50 comp rx=0 tx=0).stop 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 shutdown_connections 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f70880a13e0 0x7f70880a3890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f709c1043d0 0x7f709c19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f709c0ff390 0x7f709c1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 --2- 172.21.15.31:0/2148504585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f709c0ffd30 0x7f709c196920 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 >> 172.21.15.31:0/2148504585 conn(0x7f709c075fd0 msgr2=0x7f709c067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.725+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 shutdown_connections 2023-12-14T18:29:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:08.726+0000 7f70a42a0700 1 -- 172.21.15.31:0/2148504585 wait complete. 2023-12-14T18:29:08.773 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:29:09.084 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:09.084 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:09.190 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:09.190 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:09.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:09.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:09.214 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 32 -lt 100 ']' 2023-12-14T18:29:09.214 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 32 + 1 2023-12-14T18:29:09.246 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=33 2023-12-14T18:29:09.246 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:09.253 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:09.253 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:09.254 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:09.254 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:09.254 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:09.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:09 smithi176 ceph-mon[43506]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:09.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:09 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:29:09.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:09 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:09.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:09 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:29:09.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:09 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:09 smithi204 ceph-mon[43866]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:09 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:29:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:09 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:09 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:29:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:09 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:09.880 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:09 smithi031 ceph-mon[42085]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 19 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:09.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:09 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:29:09.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:09 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:09.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:09 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:29:09.881 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:09 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:10.084 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:10.084 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:10.275 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:10.275 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:10.297 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:10.298 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:10.298 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 33 -lt 100 ']' 2023-12-14T18:29:10.298 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 33 + 1 2023-12-14T18:29:10.354 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=34 2023-12-14T18:29:10.354 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:10.422 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:10.422 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:10.424 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:10.424 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:10.424 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:11.551 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:11.551 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:11.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:11 smithi176 ceph-mon[43506]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 28 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:11 smithi204 ceph-mon[43866]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 28 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:11.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:11 smithi031 ceph-mon[42085]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 KiB/s wr, 28 op/s; 0 B/s, 0 objects/s recovering 2023-12-14T18:29:11.972 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:11.972 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:11.981 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:11.981 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:11.981 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 34 -lt 100 ']' 2023-12-14T18:29:11.981 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 34 + 1 2023-12-14T18:29:12.030 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=35 2023-12-14T18:29:12.030 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:12.098 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:12.098 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:12.099 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:12.099 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:12.099 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:13.775 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:13 smithi031 ceph-mon[42085]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 41 KiB/s wr, 24 op/s 2023-12-14T18:29:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:13 smithi204 ceph-mon[43866]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 41 KiB/s wr, 24 op/s 2023-12-14T18:29:13.983 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:13.983 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:13 smithi176 ceph-mon[43506]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 41 KiB/s wr, 24 op/s 2023-12-14T18:29:14.379 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:14.380 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:14.413 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:14.413 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:14.413 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 35 -lt 100 ']' 2023-12-14T18:29:14.413 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 35 + 1 2023-12-14T18:29:14.470 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=36 2023-12-14T18:29:14.471 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:14.514 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:14.514 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:14.515 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:14.515 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:14.515 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:14 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:29:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:14 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:29:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:14 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:29:15.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 -- 172.21.15.31:0/2947004337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 msgr2=0x7f434810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 --2- 172.21.15.31:0/2947004337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f434810a230 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f433800b0a0 tx=0x7f433800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 -- 172.21.15.31:0/2947004337 shutdown_connections 2023-12-14T18:29:15.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 --2- 172.21.15.31:0/2947004337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f434810a800 0x7f434810cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 --2- 172.21.15.31:0/2947004337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4348101430 0x7f4348101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 --2- 172.21.15.31:0/2947004337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f434810a230 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 -- 172.21.15.31:0/2947004337 >> 172.21.15.31:0/2947004337 conn(0x7f4348076260 msgr2=0x7f4348076660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 -- 172.21.15.31:0/2947004337 shutdown_connections 2023-12-14T18:29:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.706+0000 7f434ef82700 1 -- 172.21.15.31:0/2947004337 wait complete. 2023-12-14T18:29:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.707+0000 7f434ef82700 1 Processor -- start 2023-12-14T18:29:15.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.707+0000 7f434ef82700 1 -- start start 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434ef82700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4348101430 0x7f4348198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434ef82700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f4348198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434ef82700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f434810a800 0x7f434819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434ef82700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434819c580 con 0x7f4348101d40 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434ef82700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434819c6f0 con 0x7f434810a800 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434ef82700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434819c8b0 con 0x7f4348101430 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f4348198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f4348198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54720/0 (socket says 172.21.15.31:54720) 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 -- 172.21.15.31:0/3273200394 learned_addr learned my addr 172.21.15.31:0/3273200394 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434cd1e700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4348101430 0x7f4348198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f434d51f700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f434810a800 0x7f434819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 -- 172.21.15.31:0/3273200394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4348101430 msgr2=0x7f4348198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4348101430 0x7f4348198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 -- 172.21.15.31:0/3273200394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f434810a800 msgr2=0x7f434819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f434810a800 0x7f434819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.708+0000 7f4347fff700 1 -- 172.21.15.31:0/3273200394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338009d00 con 0x7f4348101d40 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.709+0000 7f4347fff700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f4348198bd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f433800b070 tx=0x7f433800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.709+0000 7f4345ffb700 1 -- 172.21.15.31:0/3273200394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800e040 con 0x7f4348101d40 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.709+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43481102e0 con 0x7f4348101d40 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.709+0000 7f4345ffb700 1 -- 172.21.15.31:0/3273200394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4338007930 con 0x7f4348101d40 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.709+0000 7f4345ffb700 1 -- 172.21.15.31:0/3273200394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433801b6c0 con 0x7f4348101d40 2023-12-14T18:29:15.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.709+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4348110150 con 0x7f4348101d40 2023-12-14T18:29:15.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.712+0000 7f4345ffb700 1 -- 172.21.15.31:0/3273200394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f433801d470 con 0x7f4348101d40 2023-12-14T18:29:15.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.713+0000 7f4345ffb700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f43340a13e0 0x7f43340a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.713+0000 7f434cd1e700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f43340a13e0 0x7f43340a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.713+0000 7f4345ffb700 1 -- 172.21.15.31:0/3273200394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f43380ce4d0 con 0x7f4348101d40 2023-12-14T18:29:15.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.713+0000 7f434cd1e700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f43340a13e0 0x7f43340a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4330009c60 tx=0x7f4330009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:15.753 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:15 smithi031 ceph-mon[42085]: pgmap v46: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 KiB/s wr, 29 op/s 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.761+0000 7f435b110b80 1 Processor -- start 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.761+0000 7f435b110b80 1 -- start start 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f435b110b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 0x556fc98ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f435b110b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f435b110b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98f5b50 0x556fc98f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f435b110b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc97d45e0 con 0x556fc98ea380 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f435b110b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc9843870 con 0x556fc98f5b50 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f435b110b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc983cd00 con 0x556fc98eac90 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 0x556fc98ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 0x556fc98ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54732/0 (socket says 172.21.15.31:54732) 2023-12-14T18:29:15.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 -- 172.21.15.31:0/3459186607 learned_addr learned my addr 172.21.15.31:0/3459186607 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f4347fff700 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434d51f700 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98f5b50 0x556fc98f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 -- 172.21.15.31:0/3459186607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 msgr2=0x556fc98f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 -- 172.21.15.31:0/3459186607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98f5b50 msgr2=0x556fc98f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98f5b50 0x556fc98f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434cd1e700 1 -- 172.21.15.31:0/3459186607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fc96be3b0 con 0x556fc98ea380 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.762+0000 7f434d51f700 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98f5b50 0x556fc98f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.763+0000 7f434cd1e700 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 0x556fc98ea750 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f433000e4e0 tx=0x7f4330014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bea8723b4f3f1b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.763+0000 7f433effd700 1 -- 172.21.15.31:0/3459186607 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000ee00 con 0x556fc98ea380 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.763+0000 7f433effd700 1 -- 172.21.15.31:0/3459186607 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4330021070 con 0x556fc98ea380 2023-12-14T18:29:15.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.763+0000 7f433effd700 1 -- 172.21.15.31:0/3459186607 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433001d7b0 con 0x556fc98ea380 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 -- 172.21.15.31:0/3459186607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 msgr2=0x556fc98ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 0x556fc98ea750 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f433000e4e0 tx=0x7f4330014000 comp rx=0 tx=0).stop 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 -- 172.21.15.31:0/3459186607 shutdown_connections 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98f5b50 0x556fc98f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98ea380 0x556fc98ea750 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 --2- 172.21.15.31:0/3459186607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 -- 172.21.15.31:0/3459186607 >> 172.21.15.31:0/3459186607 conn(0x556fc98e0ca0 msgr2=0x556fc98e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:15.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 -- 172.21.15.31:0/3459186607 shutdown_connections 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 -- 172.21.15.31:0/3459186607 wait complete. 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.764+0000 7f435b110b80 1 -- start start 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.765+0000 7f435b110b80 1 Processor -- start 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.765+0000 7f435b110b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98f5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f435b110b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f435b110b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 0x556fc98f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f435b110b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc97d8f90 con 0x556fc98f5b50 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f435b110b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc97d8c30 con 0x556fc98ea380 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f435b110b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc96c06a0 con 0x556fc98eac90 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f435b110b80 10 client.0 _ll_register_callbacks cb 0x556fc97fc910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 0x556fc98f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 0x556fc98f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54736/0 (socket says 172.21.15.31:54736) 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 -- 172.21.15.31:0/1675658534 learned_addr learned my addr 172.21.15.31:0/1675658534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:15.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434cd1e700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f4347fff700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 -- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 msgr2=0x556fc98f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 -- 172.21.15.31:0/1675658534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 msgr2=0x556fc98f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 -- 172.21.15.31:0/1675658534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4330017070 con 0x556fc98f5b50 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f4347fff700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f434d51f700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 0x556fc98f2d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4340007ec0 tx=0x7f4340012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e336bcd8b457fd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.766+0000 7f4327fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f4327fff700 1 -- 172.21.15.31:0/1675658534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340010670 con 0x556fc98f5b50 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f4327fff700 1 -- 172.21.15.31:0/1675658534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f43400107d0 con 0x556fc98f5b50 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fc98de2b0 con 0x556fc98f5b50 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f4327fff700 1 -- 172.21.15.31:0/1675658534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4340010990 con 0x556fc98f5b50 2023-12-14T18:29:15.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f435b110b80 10 client.15957 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556fc9803170 con 0x556fc98f5b50 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f4327fff700 1 -- 172.21.15.31:0/1675658534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556fc98de2b0 con 0x556fc98f5b50 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.767+0000 7f4327fff700 1 -- 172.21.15.31:0/1675658534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f434001bbb0 con 0x556fc98f5b50 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f435b110b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f435b110b80 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f435b110b80 20 client.15957 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f435b110b80 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc9905580 0x556fc9907990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f435b110b80 4 client.15957 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fc9835ce0 con 0x556fc9905580 2023-12-14T18:29:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.768+0000 7f434cd1e700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc9905580 0x556fc9907990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f4327fff700 10 client.15957 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:15.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f434cd1e700 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc9905580 0x556fc9907990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d506ada001793a server_cookie=4fefba04c77dad0a in_seq=0 out_seq=0 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f4327fff700 1 -- 172.21.15.31:0/1675658534 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556fc9835ce0 con 0x556fc9905580 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f4327fff700 10 client.15957 handle_command_reply: tid=0 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f435b110b80 1 client.15957 shutdown 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 msgr2=0x556fc98f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f435b110b80 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 0x556fc98f2d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4340007ec0 tx=0x7f4340012730 comp rx=0 tx=0).stop 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.769+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 shutdown_connections 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc9905580 0x556fc9907990 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98f5b50 0x556fc98f2d70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 --2- 172.21.15.31:0/1675658534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f4327fff700 0 client.15957 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 >> 172.21.15.31:0/1675658534 conn(0x556fc98e0ca0 msgr2=0x556fc98e63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 shutdown_connections 2023-12-14T18:29:15.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 1 -- 172.21.15.31:0/1675658534 wait complete. 2023-12-14T18:29:15.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.770+0000 7f435b110b80 20 client.15957 trim_cache size 0 max 16384 2023-12-14T18:29:15.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 Processor -- start 2023-12-14T18:29:15.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 -- start start 2023-12-14T18:29:15.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98eca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc98eb5b0 con 0x556fc98e0ca0 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc98f0d10 con 0x556fc98ea380 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f435b110b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc98e55b0 con 0x556fc98eac90 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f434cd1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f434d51f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f434cd1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54740/0 (socket says 172.21.15.31:54740) 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f434cd1e700 1 -- 172.21.15.31:0/1490139826 learned_addr learned my addr 172.21.15.31:0/1490139826 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.788+0000 7f4347fff700 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434d51f700 1 -- 172.21.15.31:0/1490139826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 msgr2=0x556fc98eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434d51f700 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98eca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434d51f700 1 -- 172.21.15.31:0/1490139826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 msgr2=0x556fc98f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434d51f700 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434d51f700 1 -- 172.21.15.31:0/1490139826 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fc9802ec0 con 0x556fc98eac90 2023-12-14T18:29:15.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434cd1e700 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f4347fff700 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f434d51f700 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f4fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f434001a3e0 tx=0x7f4340012bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8a47d5eebeb684 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f433f7fe700 1 -- 172.21.15.31:0/1490139826 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43400050c0 con 0x556fc98eac90 2023-12-14T18:29:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f433f7fe700 1 -- 172.21.15.31:0/1490139826 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43400056e0 con 0x556fc98eac90 2023-12-14T18:29:15.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.789+0000 7f433f7fe700 1 -- 172.21.15.31:0/1490139826 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43400108d0 con 0x556fc98eac90 2023-12-14T18:29:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 -- 172.21.15.31:0/1490139826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 msgr2=0x556fc98f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f4fc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f434001a3e0 tx=0x7f4340012bb0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 -- 172.21.15.31:0/1490139826 shutdown_connections 2023-12-14T18:29:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98f4fc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 --2- 172.21.15.31:0/1490139826 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 -- 172.21.15.31:0/1490139826 >> 172.21.15.31:0/1490139826 conn(0x556fc98f8200 msgr2=0x556fc98ec350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 -- 172.21.15.31:0/1490139826 shutdown_connections 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 -- 172.21.15.31:0/1490139826 wait complete. 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.790+0000 7f435b110b80 1 -- start start 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 Processor -- start 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc98fb250 con 0x556fc98e0ca0 2023-12-14T18:29:15.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc98fb3c0 con 0x556fc98ea380 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fc98fb580 con 0x556fc98eac90 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f435b110b80 10 client.0 _ll_register_callbacks cb 0x7f4340007920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f434cd1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f434cd1e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98fc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54746/0 (socket says 172.21.15.31:54746) 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f434cd1e700 1 -- 172.21.15.31:0/2305462825 learned_addr learned my addr 172.21.15.31:0/2305462825 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f434d51f700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.791+0000 7f4347fff700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f434cd1e700 1 -- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 msgr2=0x556fc98fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f434cd1e700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98fad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f434cd1e700 1 -- 172.21.15.31:0/2305462825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 msgr2=0x556fc98fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f434cd1e700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f434cd1e700 1 -- 172.21.15.31:0/2305462825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433002a040 con 0x556fc98e0ca0 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f434cd1e700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98fc4a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4330011ab0 tx=0x7f433000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1796ed298cfd2458 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:15.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f4327fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f4327fff700 1 -- 172.21.15.31:0/2305462825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330038070 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4330021070 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f4327fff700 1 -- 172.21.15.31:0/2305462825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4330033ae0 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f4327fff700 1 -- 172.21.15.31:0/2305462825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4330033ca0 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f435b110b80 10 client.15963 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f4348110150 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.792+0000 7f4327fff700 1 -- 172.21.15.31:0/2305462825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4330021070 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f4327fff700 1 -- 172.21.15.31:0/2305462825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f433003b030 con 0x556fc98e0ca0 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f435b110b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f435b110b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f435b110b80 20 client.15963 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f435b110b80 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc98f5b50 0x556fc995c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:15.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f435b110b80 4 client.15963 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556fc9835a00 con 0x556fc98f5b50 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f4347fff700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc98f5b50 0x556fc995c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f4327fff700 10 client.15963 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.793+0000 7f4347fff700 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc98f5b50 0x556fc995c7c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2028d50db0d422b0 server_cookie=6a98e41f0cef838c in_seq=0 out_seq=0 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f4327fff700 1 -- 172.21.15.31:0/2305462825 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556fc9835a00 con 0x556fc98f5b50 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f4327fff700 10 client.15963 handle_command_reply: tid=0 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 client.15963 shutdown 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 msgr2=0x556fc98fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98fc4a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4330011ab0 tx=0x7f433000ec70 comp rx=0 tx=0).stop 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 shutdown_connections 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556fc98f5b50 0x556fc995c7c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556fc98eac90 0x556fc98fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556fc98e0ca0 0x556fc98fc4a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 --2- 172.21.15.31:0/2305462825 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556fc98ea380 0x556fc98fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f4327fff700 0 client.15963 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 >> 172.21.15.31:0/2305462825 conn(0x556fc98f8200 msgr2=0x556fc98ecbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 shutdown_connections 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 1 -- 172.21.15.31:0/2305462825 wait complete. 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.794+0000 7f435b110b80 20 client.15963 trim_cache size 0 max 16384 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:29:15.801 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f43340a13e0 msgr2=0x7f43340a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f43340a13e0 0x7f43340a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4330009c60 tx=0x7f4330009260 comp rx=0 tx=0).stop 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 msgr2=0x7f4348198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f4348198bd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f433800b070 tx=0x7f433800bce0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 shutdown_connections 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f434810a800 0x7f434819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f43340a13e0 0x7f43340a3890 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4330009c60 tx=0x7f4330009260 comp rx=0 tx=0).stop 2023-12-14T18:29:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4348101430 0x7f4348198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 --2- 172.21.15.31:0/3273200394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4348101d40 0x7f4348198bd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 >> 172.21.15.31:0/3273200394 conn(0x7f4348076260 msgr2=0x7f43480ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 shutdown_connections 2023-12-14T18:29:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:15.795+0000 7f434ef82700 1 -- 172.21.15.31:0/3273200394 wait complete. 2023-12-14T18:29:15.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:15 smithi204 ceph-mon[43866]: pgmap v46: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 KiB/s wr, 29 op/s 2023-12-14T18:29:15.941 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:15.942 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:15.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:29:15.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:29:15.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:15 smithi176 ceph-mon[43506]: pgmap v46: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 48 KiB/s wr, 29 op/s 2023-12-14T18:29:16.139 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:16.139 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:16.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:16.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:16.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 36 -lt 100 ']' 2023-12-14T18:29:16.155 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 36 + 1 2023-12-14T18:29:16.195 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=37 2023-12-14T18:29:16.195 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:16.207 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:16.207 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:16.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:16.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:16.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:16.617 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:16.617 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:16.722 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:16.722 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:16.730 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:16.731 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:16.731 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 37 -lt 100 ']' 2023-12-14T18:29:16.731 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 37 + 1 2023-12-14T18:29:16.762 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=38 2023-12-14T18:29:16.762 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:16.818 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:16.818 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:16.820 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:16.820 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:16.820 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:17.400 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:17.400 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:17.704 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:17.704 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:17.722 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:17.722 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:17.723 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 38 -lt 100 ']' 2023-12-14T18:29:17.723 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 38 + 1 2023-12-14T18:29:17.762 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=39 2023-12-14T18:29:17.762 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:17.820 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:17.821 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:17.822 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:17.822 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:17.822 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:17.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:17 smithi204 ceph-mon[43866]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:29:17.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:17 smithi031 ceph-mon[42085]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:29:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:17 smithi176 ceph-mon[43506]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:29:18.374 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:18.374 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:18.621 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:18.621 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:18.628 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:18.628 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:18.628 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 39 -lt 100 ']' 2023-12-14T18:29:18.629 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 39 + 1 2023-12-14T18:29:18.782 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=40 2023-12-14T18:29:18.782 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:18.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 -- 172.21.15.31:0/2282967453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14102570 msgr2=0x7f5b141029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 --2- 172.21.15.31:0/2282967453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14102570 0x7f5b141029c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009990 tx=0x7f5b08009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 -- 172.21.15.31:0/2282967453 shutdown_connections 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 --2- 172.21.15.31:0/2282967453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b14102f00 0x7f5b1410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 --2- 172.21.15.31:0/2282967453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b14108670 0x7f5b14108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 --2- 172.21.15.31:0/2282967453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14102570 0x7f5b141029c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 -- 172.21.15.31:0/2282967453 >> 172.21.15.31:0/2282967453 conn(0x7f5b140fe090 msgr2=0x7f5b141004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 -- 172.21.15.31:0/2282967453 shutdown_connections 2023-12-14T18:29:18.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.922+0000 7f5b18b64700 1 -- 172.21.15.31:0/2282967453 wait complete. 2023-12-14T18:29:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 Processor -- start 2023-12-14T18:29:18.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 -- start start 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b14102570 0x7f5b141985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b14102f00 0x7f5b14198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 0x7f5b1419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1419d580 con 0x7f5b14108670 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1419d6f0 con 0x7f5b14102570 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.924+0000 7f5b18b64700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1419d8b0 con 0x7f5b14102f00 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 0x7f5b1419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 0x7f5b1419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54760/0 (socket says 172.21.15.31:54760) 2023-12-14T18:29:18.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 -- 172.21.15.31:0/802361597 learned_addr learned my addr 172.21.15.31:0/802361597 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b1259c700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b14102570 0x7f5b141985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b11d9b700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b14102f00 0x7f5b14198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 -- 172.21.15.31:0/802361597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b14102f00 msgr2=0x7f5b14198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b14102f00 0x7f5b14198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 -- 172.21.15.31:0/802361597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b14102570 msgr2=0x7f5b141985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b14102570 0x7f5b141985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 -- 172.21.15.31:0/802361597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080095f0 con 0x7f5b14108670 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.925+0000 7f5b12d9d700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 0x7f5b1419cf40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c007ee0 tx=0x7f5b0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.926+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00dd20 con 0x7f5b14108670 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.926+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0c00f040 con 0x7f5b14108670 2023-12-14T18:29:18.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.926+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14112580 con 0x7f5b14108670 2023-12-14T18:29:18.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.926+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0048a0 con 0x7f5b14108670 2023-12-14T18:29:18.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.926+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b141123f0 con 0x7f5b14108670 2023-12-14T18:29:18.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.927+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14112710 con 0x7f5b14108670 2023-12-14T18:29:18.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.928+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f5b0c003a00 con 0x7f5b14108670 2023-12-14T18:29:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.929+0000 7f5b037fe700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5afc0a1460 0x7f5afc0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.929+0000 7f5b1259c700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5afc0a1460 0x7f5afc0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:18.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.929+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f5b0c0ccc20 con 0x7f5b14108670 2023-12-14T18:29:18.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.930+0000 7f5b1259c700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5afc0a1460 0x7f5afc0a3910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009be0 tx=0x7f5b04009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:18.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:18.934+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5b0c04e730 con 0x7f5b14108670 2023-12-14T18:29:18.937 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:18.937 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:18.939 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:18.939 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:18.939 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.200+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b141998f0 con 0x7f5b14108670 2023-12-14T18:29:19.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.201+0000 7f5b037fe700 1 -- 172.21.15.31:0/802361597 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f5b0c0959f0 con 0x7f5b14108670 2023-12-14T18:29:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:29:19.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:29:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5afc0a1460 msgr2=0x7f5afc0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5afc0a1460 0x7f5afc0a3910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009be0 tx=0x7f5b04009260 comp rx=0 tx=0).stop 2023-12-14T18:29:19.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 msgr2=0x7f5b1419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 0x7f5b1419cf40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c007ee0 tx=0x7f5b0c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 shutdown_connections 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b14108670 0x7f5b1419cf40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b14102570 0x7f5b141985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5afc0a1460 0x7f5afc0a3910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 --2- 172.21.15.31:0/802361597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b14102f00 0x7f5b14198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 >> 172.21.15.31:0/802361597 conn(0x7f5b140fe090 msgr2=0x7f5b140ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 shutdown_connections 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:19.203+0000 7f5b18b64700 1 -- 172.21.15.31:0/802361597 wait complete. 2023-12-14T18:29:19.205 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:29:19.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:29:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:19 smithi204 ceph-mon[43866]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:29:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:19 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/802361597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:29:19.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:19 smithi031 ceph-mon[42085]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:29:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:19 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/802361597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:29:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:19 smithi176 ceph-mon[43506]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:29:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:19 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/802361597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:29:20.166 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:20.166 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:20.399 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:20.399 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:20.415 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:20.415 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:20.415 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 40 -lt 100 ']' 2023-12-14T18:29:20.415 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 40 + 1 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 -- 172.21.15.31:0/758642317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e80fecb0 msgr2=0x7f55e80ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 --2- 172.21.15.31:0/758642317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e80fecb0 0x7f55e80ff120 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009990 tx=0x7f55e4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 -- 172.21.15.31:0/758642317 shutdown_connections 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 --2- 172.21.15.31:0/758642317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e810cef0 0x7f55e810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 --2- 172.21.15.31:0/758642317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e81012e0 0x7f55e80fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 --2- 172.21.15.31:0/758642317 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e80fecb0 0x7f55e80ff120 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 -- 172.21.15.31:0/758642317 >> 172.21.15.31:0/758642317 conn(0x7f55e8075810 msgr2=0x7f55e8075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 -- 172.21.15.31:0/758642317 shutdown_connections 2023-12-14T18:29:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.465+0000 7f55efc67700 1 -- 172.21.15.31:0/758642317 wait complete. 2023-12-14T18:29:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 Processor -- start 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 -- start start 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e80fecb0 0x7f55e819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 0x7f55e819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e810cef0 0x7f55e8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8198cb0 con 0x7f55e81012e0 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8198e20 con 0x7f55e810cef0 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55efc67700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8198fe0 con 0x7f55e80fecb0 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55ed202700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 0x7f55e819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55ed202700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 0x7f55e819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54782/0 (socket says 172.21.15.31:54782) 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55ed202700 1 -- 172.21.15.31:0/462683347 learned_addr learned my addr 172.21.15.31:0/462683347 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55ee204700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e810cef0 0x7f55e8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.466+0000 7f55eda03700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e80fecb0 0x7f55e819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ed202700 1 -- 172.21.15.31:0/462683347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e80fecb0 msgr2=0x7f55e819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ed202700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e80fecb0 0x7f55e819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ed202700 1 -- 172.21.15.31:0/462683347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e810cef0 msgr2=0x7f55e8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ed202700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e810cef0 0x7f55e8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ed202700 1 -- 172.21.15.31:0/462683347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e40095f0 con 0x7f55e81012e0 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ee204700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e810cef0 0x7f55e8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55ed202700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 0x7f55e819eb10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55e40095c0 tx=0x7f55e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55daffd700 1 -- 172.21.15.31:0/462683347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e401d070 con 0x7f55e81012e0 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55daffd700 1 -- 172.21.15.31:0/462683347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e4005050 con 0x7f55e81012e0 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e8112730 con 0x7f55e81012e0 2023-12-14T18:29:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55daffd700 1 -- 172.21.15.31:0/462683347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4004a90 con 0x7f55e81012e0 2023-12-14T18:29:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.467+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e81125a0 con 0x7f55e81012e0 2023-12-14T18:29:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.470+0000 7f55daffd700 1 -- 172.21.15.31:0/462683347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f55e4005430 con 0x7f55e81012e0 2023-12-14T18:29:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.470+0000 7f55daffd700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55d40a13e0 0x7f55d40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.471+0000 7f55daffd700 1 -- 172.21.15.31:0/462683347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f55e40ce210 con 0x7f55e81012e0 2023-12-14T18:29:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.471+0000 7f55eda03700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55d40a13e0 0x7f55d40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.471+0000 7f55eda03700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55d40a13e0 0x7f55d40a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f55dc007be0 tx=0x7f55dc0073d0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:20.481 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=41 2023-12-14T18:29:20.482 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:20.506 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:20.506 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:20.507 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:20.507 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:20.507 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:20.554 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 Processor -- start 2023-12-14T18:29:20.555 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 -- start start 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5977e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a5978370 0x5568a59787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a59854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a5862580 con 0x5568a59830f0 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a58d1810 con 0x5568a57d5430 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.554+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a58caca0 con 0x5568a5978370 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a59854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a59854d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54786/0 (socket says 172.21.15.31:54786) 2023-12-14T18:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 -- 172.21.15.31:0/3090829732 learned_addr learned my addr 172.21.15.31:0/3090829732 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ed202700 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a5978370 0x5568a59787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55eda03700 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5977e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 -- 172.21.15.31:0/3090829732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a5978370 msgr2=0x5568a59787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a5978370 0x5568a59787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 -- 172.21.15.31:0/3090829732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 msgr2=0x5568a5977e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5977e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 -- 172.21.15.31:0/3090829732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a574c350 con 0x5568a59830f0 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55eda03700 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5977e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.555+0000 7f55ee204700 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a59854d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009fd0 tx=0x7f55e0007960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e04adb03245f7b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.556+0000 7f55d3fff700 1 -- 172.21.15.31:0/3090829732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004bb0 con 0x5568a59830f0 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.556+0000 7f55d3fff700 1 -- 172.21.15.31:0/3090829732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e000f040 con 0x5568a59830f0 2023-12-14T18:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.556+0000 7f55d3fff700 1 -- 172.21.15.31:0/3090829732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0015cd0 con 0x5568a59830f0 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/3090829732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 msgr2=0x5568a59854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a59854d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009fd0 tx=0x7f55e0007960 comp rx=0 tx=0).stop 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/3090829732 shutdown_connections 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a59854d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5977e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/3090829732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a5978370 0x5568a59787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/3090829732 >> 172.21.15.31:0/3090829732 conn(0x5568a596e890 msgr2=0x5568a5974200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/3090829732 shutdown_connections 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/3090829732 wait complete. 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.557+0000 7f55fbdf5b80 1 -- start start 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 Processor -- start 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5982a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 0x5568a597fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a59830f0 0x5568a5980300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a5866f30 con 0x5568a5978370 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a5866bd0 con 0x5568a57d5430 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a574e640 con 0x5568a59830f0 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55fbdf5b80 10 client.0 _ll_register_callbacks cb 0x5568a588a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.558+0000 7f55ed202700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 0x5568a597fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 0x5568a597fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54800/0 (socket says 172.21.15.31:54800) 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 -- 172.21.15.31:0/351689907 learned_addr learned my addr 172.21.15.31:0/351689907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ee204700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a59830f0 0x5568a5980300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55eda03700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5982a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 -- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a59830f0 msgr2=0x5568a5980300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a59830f0 0x5568a5980300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 -- 172.21.15.31:0/351689907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 msgr2=0x5568a5982a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5982a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 -- 172.21.15.31:0/351689907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e0009db0 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55eda03700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5982a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ee204700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a59830f0 0x5568a5980300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55ed202700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 0x5568a597fd80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f55e40971e0 tx=0x7f55e4004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a8a40056af29cf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55d17fa700 1 -- 172.21.15.31:0/351689907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e409a420 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.559+0000 7f55d17fa700 1 -- 172.21.15.31:0/351689907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f55e409d070 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568a596bea0 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55d17fa700 1 -- 172.21.15.31:0/351689907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e40a30f0 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 10 client.15981 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5568a5891110 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55d17fa700 1 -- 172.21.15.31:0/351689907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e40b9080 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55d17fa700 1 -- 172.21.15.31:0/351689907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55e40cb6d0 con 0x5568a5978370 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 20 client.15981 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59926c0 0x5568a5994ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 4 client.15981 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.560+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568a58c3c80 con 0x5568a59926c0 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.561+0000 7f55eda03700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59926c0 0x5568a5994ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.561+0000 7f55eda03700 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59926c0 0x5568a5994ad0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4473410ec39c8680 server_cookie=495e0405513d0a4c in_seq=0 out_seq=0 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.561+0000 7f55d17fa700 10 client.15981 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55d17fa700 1 -- 172.21.15.31:0/351689907 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5568a58c3c80 con 0x5568a59926c0 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55d17fa700 10 client.15981 handle_command_reply: tid=0 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 client.15981 shutdown 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 msgr2=0x5568a597fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 0x5568a597fd80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f55e40971e0 tx=0x7f55e4004ed0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 shutdown_connections 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59926c0 0x5568a5994ad0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a59830f0 0x5568a5980300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.567 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a57d5430 0x5568a5982a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/351689907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a5978370 0x5568a597fd80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55d17fa700 0 client.15981 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 >> 172.21.15.31:0/351689907 conn(0x5568a596e890 msgr2=0x5568a5973d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 shutdown_connections 2023-12-14T18:29:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.562+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/351689907 wait complete. 2023-12-14T18:29:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.563+0000 7f55fbdf5b80 20 client.15981 trim_cache size 0 max 16384 2023-12-14T18:29:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 Processor -- start 2023-12-14T18:29:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 -- start start 2023-12-14T18:29:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5982520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5982a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a597fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a5985a70 con 0x5568a59830f0 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a597ba10 con 0x5568a5978370 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a59790d0 con 0x5568a57d5430 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55ee204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a597fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55ee204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a597fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54812/0 (socket says 172.21.15.31:54812) 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55ee204700 1 -- 172.21.15.31:0/4108140014 learned_addr learned my addr 172.21.15.31:0/4108140014 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55ed202700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5982a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.580+0000 7f55eda03700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5982520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ee204700 1 -- 172.21.15.31:0/4108140014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 msgr2=0x5568a5982520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ee204700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5982520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ee204700 1 -- 172.21.15.31:0/4108140014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 msgr2=0x5568a5982a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ee204700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5982a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ee204700 1 -- 172.21.15.31:0/4108140014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a59700d0 con 0x5568a59830f0 2023-12-14T18:29:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55eda03700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5982520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ed202700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5982a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55ee204700 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a597fd40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009fd0 tx=0x7f55e0007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebae8a0171166cd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55d8dac700 1 -- 172.21.15.31:0/4108140014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0003a00 con 0x5568a59830f0 2023-12-14T18:29:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55d8dac700 1 -- 172.21.15.31:0/4108140014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e0023070 con 0x5568a59830f0 2023-12-14T18:29:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.581+0000 7f55d8dac700 1 -- 172.21.15.31:0/4108140014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004290 con 0x5568a59830f0 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/4108140014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 msgr2=0x5568a597fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a597fd40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009fd0 tx=0x7f55e0007f20 comp rx=0 tx=0).stop 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/4108140014 shutdown_connections 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a597fd40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5982520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/4108140014 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5982a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/4108140014 >> 172.21.15.31:0/4108140014 conn(0x5568a596e890 msgr2=0x5568a598edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/4108140014 shutdown_connections 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/4108140014 wait complete. 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 -- start start 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.582+0000 7f55fbdf5b80 1 Processor -- start 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5989650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5989b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a5987ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a59882b0 con 0x5568a59830f0 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a5988420 con 0x5568a5978370 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a59885e0 con 0x5568a57d5430 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55fbdf5b80 10 client.0 _ll_register_callbacks cb 0x7f55e409a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55eda03700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5989650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55ee204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a5987ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55eda03700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5989650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:49158/0 (socket says 172.21.15.31:49158) 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55eda03700 1 -- 172.21.15.31:0/1741809303 learned_addr learned my addr 172.21.15.31:0/1741809303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.583+0000 7f55ed202700 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5989b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55eda03700 1 -- 172.21.15.31:0/1741809303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 msgr2=0x5568a5989b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55eda03700 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5989b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55eda03700 1 -- 172.21.15.31:0/1741809303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 msgr2=0x5568a5987ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55eda03700 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a5987ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55eda03700 1 -- 172.21.15.31:0/1741809303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568a5891110 con 0x5568a57d5430 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55eda03700 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5989650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5568a5995370 tx=0x7f55dc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d2c281aa76cfab server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55d17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55d17fa700 1 -- 172.21.15.31:0/1741809303 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc026030 con 0x5568a57d5430 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55d17fa700 1 -- 172.21.15.31:0/1741809303 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f55dc017070 con 0x5568a57d5430 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55e000f040 con 0x5568a57d5430 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55d17fa700 1 -- 172.21.15.31:0/1741809303 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55dc0287f0 con 0x5568a57d5430 2023-12-14T18:29:20.589 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55fbdf5b80 10 client.25840 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.584+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5568a574c350 con 0x5568a57d5430 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55d17fa700 1 -- 172.21.15.31:0/1741809303 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55e000f040 con 0x5568a57d5430 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55d17fa700 1 -- 172.21.15.31:0/1741809303 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55dc003b70 con 0x5568a57d5430 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55fbdf5b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55fbdf5b80 10 client.25840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55fbdf5b80 20 client.25840 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59e7740 0x5568a59e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55fbdf5b80 4 client.25840 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5568a58c39a0 con 0x5568a59e7740 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.585+0000 7f55ed202700 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59e7740 0x5568a59e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.586+0000 7f55ed202700 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59e7740 0x5568a59e9b50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd197591f089257c server_cookie=3f0c7ddc9e053cdb in_seq=0 out_seq=0 2023-12-14T18:29:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.586+0000 7f55d17fa700 10 client.25840 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55d17fa700 1 -- 172.21.15.31:0/1741809303 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5568a58c39a0 con 0x5568a59e7740 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55d17fa700 10 client.25840 handle_command_reply: tid=0 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 client.25840 shutdown 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 msgr2=0x5568a5989650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5989650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5568a5995370 tx=0x7f55dc0047c0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 shutdown_connections 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5568a59e7740 0x5568a59e9b50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5568a59830f0 0x5568a5987ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55d17fa700 0 client.25840 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568a57d5430 0x5568a5989650 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 --2- 172.21.15.31:0/1741809303 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5568a5978370 0x5568a5989b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.605+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 >> 172.21.15.31:0/1741809303 conn(0x5568a596e890 msgr2=0x5568a597fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 shutdown_connections 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55fbdf5b80 1 -- 172.21.15.31:0/1741809303 wait complete. 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55fbdf5b80 20 client.25840 trim_cache size 0 max 16384 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:29:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55d40a13e0 msgr2=0x7f55d40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55d40a13e0 0x7f55d40a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f55dc007be0 tx=0x7f55dc0073d0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 msgr2=0x7f55e819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 0x7f55e819eb10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f55e40095c0 tx=0x7f55e400f690 comp rx=0 tx=0).stop 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 shutdown_connections 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55e810cef0 0x7f55e8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55d40a13e0 0x7f55d40a3890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e80fecb0 0x7f55e819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 --2- 172.21.15.31:0/462683347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55e81012e0 0x7f55e819eb10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.606+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 >> 172.21.15.31:0/462683347 conn(0x7f55e8075810 msgr2=0x7f55e80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.607+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 shutdown_connections 2023-12-14T18:29:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:20.607+0000 7f55efc67700 1 -- 172.21.15.31:0/462683347 wait complete. 2023-12-14T18:29:20.932 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:20.932 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:21.105 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:21.105 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:21.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:21.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:21.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 41 -lt 100 ']' 2023-12-14T18:29:21.119 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 41 + 1 2023-12-14T18:29:21.169 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=42 2023-12-14T18:29:21.169 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:21.204 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:21.204 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:21.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:21.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:21.206 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:21.806 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:21.806 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:21 smithi204 ceph-mon[43866]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-12-14T18:29:21.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:21 smithi031 ceph-mon[42085]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-12-14T18:29:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:21 smithi176 ceph-mon[43506]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-12-14T18:29:22.053 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:22.053 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:22.062 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:22.062 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:22.062 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 42 -lt 100 ']' 2023-12-14T18:29:22.062 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 42 + 1 2023-12-14T18:29:22.111 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=43 2023-12-14T18:29:22.111 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:22.178 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:22.178 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:22.180 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:22.180 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:22.180 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:22.782 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:22.782 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:22.953 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:22.953 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:22.961 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:22.962 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:22.962 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 43 -lt 100 ']' 2023-12-14T18:29:22.962 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 43 + 1 2023-12-14T18:29:23.006 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=44 2023-12-14T18:29:23.006 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:23.051 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:23.051 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:23.053 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:23.053 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:23.053 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 -- 172.21.15.31:0/2229023570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 msgr2=0x7fd22010cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 --2- 172.21.15.31:0/2229023570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd22010cc20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd21800b0a0 tx=0x7fd21800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 -- 172.21.15.31:0/2229023570 shutdown_connections 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 --2- 172.21.15.31:0/2229023570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd22010cc20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 --2- 172.21.15.31:0/2229023570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220101470 0x7fd220101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 --2- 172.21.15.31:0/2229023570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd220101d80 0x7fd22010a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 -- 172.21.15.31:0/2229023570 >> 172.21.15.31:0/2229023570 conn(0x7fd2200763a0 msgr2=0x7fd2200767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 -- 172.21.15.31:0/2229023570 shutdown_connections 2023-12-14T18:29:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.430+0000 7fd224be0700 1 -- 172.21.15.31:0/2229023570 wait complete. 2023-12-14T18:29:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.431+0000 7fd224be0700 1 Processor -- start 2023-12-14T18:29:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.431+0000 7fd224be0700 1 -- start start 2023-12-14T18:29:23.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd224be0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220101470 0x7fd2200ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd224be0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd220101d80 0x7fd2200ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd224be0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd2201a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd224be0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2200ffe30 con 0x7fd22010a7b0 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd224be0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201a4c30 con 0x7fd220101d80 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd224be0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201a4d70 con 0x7fd220101470 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd2201a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd2201a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54834/0 (socket says 172.21.15.31:54834) 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 -- 172.21.15.31:0/3113800877 learned_addr learned my addr 172.21.15.31:0/3113800877 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd220101d80 0x7fd2200ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21e59c700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220101470 0x7fd2200ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 -- 172.21.15.31:0/3113800877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220101470 msgr2=0x7fd2200ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220101470 0x7fd2200ff0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 -- 172.21.15.31:0/3113800877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd220101d80 msgr2=0x7fd2200ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd220101d80 0x7fd2200ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.432+0000 7fd21ed9d700 1 -- 172.21.15.31:0/3113800877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218009d00 con 0x7fd22010a7b0 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.433+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd2201a46f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd218009fd0 tx=0x7fd2180094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.433+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3113800877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800e040 con 0x7fd22010a7b0 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.433+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2201102a0 con 0x7fd22010a7b0 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.433+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3113800877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd21801d470 con 0x7fd22010a7b0 2023-12-14T18:29:23.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.433+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3113800877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180052a0 con 0x7fd22010a7b0 2023-12-14T18:29:23.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.433+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd220110110 con 0x7fd22010a7b0 2023-12-14T18:29:23.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.435+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3113800877 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fd21801d5e0 con 0x7fd22010a7b0 2023-12-14T18:29:23.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.436+0000 7fd20f7fe700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd2080a13e0 0x7fd2080a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.436+0000 7fd21e59c700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd2080a13e0 0x7fd2080a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.436+0000 7fd20f7fe700 1 -- 172.21.15.31:0/3113800877 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd2180ce450 con 0x7fd22010a7b0 2023-12-14T18:29:23.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.437+0000 7fd21e59c700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd2080a13e0 0x7fd2080a3890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd210009c60 tx=0x7fd210009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.507+0000 7fd230d6eb80 1 Processor -- start 2023-12-14T18:29:23.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.507+0000 7fd230d6eb80 1 -- start start 2023-12-14T18:29:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261c7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e260b15e0 con 0x563e261c7380 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26120870 con 0x563e261d2b50 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e26119d00 con 0x563e261c7c90 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54848/0 (socket says 172.21.15.31:54848) 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 -- 172.21.15.31:0/271845517 learned_addr learned my addr 172.21.15.31:0/271845517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 -- 172.21.15.31:0/271845517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 msgr2=0x563e261d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 -- 172.21.15.31:0/271845517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 msgr2=0x563e261d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.508+0000 7fd21e59c700 1 -- 172.21.15.31:0/271845517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e25f9b3b0 con 0x563e261c7380 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.509+0000 7fd21e59c700 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261c7750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e4e0 tx=0x7fd210014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1ad15a8ee43e4d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.509+0000 7fd20cff9700 1 -- 172.21.15.31:0/271845517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000ee00 con 0x563e261c7380 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.509+0000 7fd20cff9700 1 -- 172.21.15.31:0/271845517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd210021070 con 0x563e261c7380 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.509+0000 7fd20cff9700 1 -- 172.21.15.31:0/271845517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21001d7f0 con 0x563e261c7380 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 -- 172.21.15.31:0/271845517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 msgr2=0x563e261c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261c7750 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e4e0 tx=0x7fd210014000 comp rx=0 tx=0).stop 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 -- 172.21.15.31:0/271845517 shutdown_connections 2023-12-14T18:29:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261c7750 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/271845517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 -- 172.21.15.31:0/271845517 >> 172.21.15.31:0/271845517 conn(0x563e261bdca0 msgr2=0x563e261c37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 -- 172.21.15.31:0/271845517 shutdown_connections 2023-12-14T18:29:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.510+0000 7fd230d6eb80 1 -- 172.21.15.31:0/271845517 wait complete. 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 -- start start 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 Processor -- start 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7380 0x563e261d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 0x563e261cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e260b5f90 con 0x563e261c7c90 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e260b5c30 con 0x563e261d2b50 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e25f9d6a0 con 0x563e261c7380 2023-12-14T18:29:23.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.511+0000 7fd230d6eb80 10 client.0 _ll_register_callbacks cb 0x563e260d9910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 0x563e261cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 0x563e261cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54864/0 (socket says 172.21.15.31:54864) 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 -- 172.21.15.31:0/4088824253 learned_addr learned my addr 172.21.15.31:0/4088824253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21e59c700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7380 0x563e261d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 -- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7380 msgr2=0x563e261d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7380 0x563e261d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 -- 172.21.15.31:0/4088824253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 msgr2=0x563e261cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 -- 172.21.15.31:0/4088824253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210017070 con 0x563e261c7c90 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21e59c700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7380 0x563e261d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:23.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 0x563e261cf7b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd214012750 tx=0x7fd214012a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b9dba7e5238d25 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.512+0000 7fd205ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd205ffb700 1 -- 172.21.15.31:0/4088824253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214017070 con 0x563e261c7c90 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd205ffb700 1 -- 172.21.15.31:0/4088824253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd214004830 con 0x563e261c7c90 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e261bb2b0 con 0x563e261c7c90 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd205ffb700 1 -- 172.21.15.31:0/4088824253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd214004a30 con 0x563e261c7c90 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd230d6eb80 10 client.15999 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563e260e0170 con 0x563e261c7c90 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.513+0000 7fd205ffb700 1 -- 172.21.15.31:0/4088824253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e261bb2b0 con 0x563e261c7c90 2023-12-14T18:29:23.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd205ffb700 1 -- 172.21.15.31:0/4088824253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd21401c740 con 0x563e261c7c90 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd230d6eb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd230d6eb80 10 client.15999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd230d6eb80 20 client.15999 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e261e25e0 0x563e261e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd230d6eb80 4 client.15999 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e26112ce0 con 0x563e261e25e0 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.514+0000 7fd21e59c700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e261e25e0 0x563e261e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.515+0000 7fd21e59c700 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e261e25e0 0x563e261e49f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5567c997cc2a6c24 server_cookie=484d5a2e7b832db4 in_seq=0 out_seq=0 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.515+0000 7fd205ffb700 10 client.15999 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd205ffb700 1 -- 172.21.15.31:0/4088824253 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563e26112ce0 con 0x563e261e25e0 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd205ffb700 10 client.15999 handle_command_reply: tid=0 2023-12-14T18:29:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 client.15999 shutdown 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 msgr2=0x563e261cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 0x563e261cf7b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd214012750 tx=0x7fd214012a60 comp rx=0 tx=0).stop 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 shutdown_connections 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e261e25e0 0x563e261e49f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7380 0x563e261d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/4088824253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7c90 0x563e261cf7b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd205ffb700 0 client.15999 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 >> 172.21.15.31:0/4088824253 conn(0x563e261bdca0 msgr2=0x563e261c33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 shutdown_connections 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 1 -- 172.21.15.31:0/4088824253 wait complete. 2023-12-14T18:29:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.516+0000 7fd230d6eb80 20 client.15999 trim_cache size 0 max 16384 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 Processor -- start 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 -- start start 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e261c8390 con 0x563e261c7380 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e261c85b0 con 0x563e261d2b50 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.531+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e261d0370 con 0x563e261c7c90 2023-12-14T18:29:23.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54878/0 (socket says 172.21.15.31:54878) 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 -- 172.21.15.31:0/2189718617 learned_addr learned my addr 172.21.15.31:0/2189718617 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 -- 172.21.15.31:0/2189718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 msgr2=0x563e261d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 -- 172.21.15.31:0/2189718617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 msgr2=0x563e261caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 -- 172.21.15.31:0/2189718617 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e260dfec0 con 0x563e261c7380 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd21e59c700 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d1fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e4e0 tx=0x7fd21001ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efccfe92378a1450 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd20d7fa700 1 -- 172.21.15.31:0/2189718617 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21000ec00 con 0x563e261c7380 2023-12-14T18:29:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd20d7fa700 1 -- 172.21.15.31:0/2189718617 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd210033120 con 0x563e261c7380 2023-12-14T18:29:23.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.532+0000 7fd20d7fa700 1 -- 172.21.15.31:0/2189718617 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210005490 con 0x563e261c7380 2023-12-14T18:29:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 -- 172.21.15.31:0/2189718617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 msgr2=0x563e261d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d1fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd21000e4e0 tx=0x7fd21001ceb0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 -- 172.21.15.31:0/2189718617 shutdown_connections 2023-12-14T18:29:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261d2b50 0x563e261caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d1fc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/2189718617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261c7c90 0x563e261d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 -- 172.21.15.31:0/2189718617 >> 172.21.15.31:0/2189718617 conn(0x563e261bdca0 msgr2=0x563e261dde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 -- 172.21.15.31:0/2189718617 shutdown_connections 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.533+0000 7fd230d6eb80 1 -- 172.21.15.31:0/2189718617 wait complete. 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 -- start start 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 Processor -- start 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261c7c90 0x563e261d7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261d2b50 0x563e26235460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e261d7cc0 con 0x563e261c7380 2023-12-14T18:29:23.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e261d7e30 con 0x563e261c7c90 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e262359d0 con 0x563e261d2b50 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd230d6eb80 10 client.0 _ll_register_callbacks cb 0x7fd214009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd21e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd21e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54894/0 (socket says 172.21.15.31:54894) 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd21e59c700 1 -- 172.21.15.31:0/1820919246 learned_addr learned my addr 172.21.15.31:0/1820919246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261c7c90 0x563e261d7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.534+0000 7fd21ed9d700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261d2b50 0x563e26235460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd21e59c700 1 -- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261d2b50 msgr2=0x563e26235460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd21e59c700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261d2b50 0x563e26235460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd21e59c700 1 -- 172.21.15.31:0/1820919246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261c7c90 msgr2=0x563e261d7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:23.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd21e59c700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261c7c90 0x563e261d7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd21e59c700 1 -- 172.21.15.31:0/1820919246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21002a040 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd21e59c700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d6fd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd210035040 tx=0x7fd21000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270298881ace5aa0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd205ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd205ffb700 1 -- 172.21.15.31:0/1820919246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd210040070 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd205ffb700 1 -- 172.21.15.31:0/1820919246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd210033390 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd210021070 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.535+0000 7fd205ffb700 1 -- 172.21.15.31:0/1820919246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd210033550 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 10 client.16005 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563e260c69a0 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd205ffb700 1 -- 172.21.15.31:0/1820919246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd210021070 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd205ffb700 1 -- 172.21.15.31:0/1820919246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd210043020 con 0x563e261c7380 2023-12-14T18:29:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 10 client.16005 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 10 client.16005 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 20 client.16005 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e26236de0 0x563e262391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 4 client.16005 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563e26112a00 con 0x563e26236de0 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.536+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e26236de0 0x563e262391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.537+0000 7fd205ffb700 10 client.16005 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.537+0000 7fd21dd9b700 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e26236de0 0x563e262391f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14f2995c329b0161 server_cookie=61774cf7b8e09243 in_seq=0 out_seq=0 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.537+0000 7fd205ffb700 1 -- 172.21.15.31:0/1820919246 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563e26112a00 con 0x563e26236de0 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.537+0000 7fd205ffb700 10 client.16005 handle_command_reply: tid=0 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.537+0000 7fd230d6eb80 1 client.16005 shutdown 2023-12-14T18:29:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.537+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 msgr2=0x563e261d6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d6fd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd210035040 tx=0x7fd21000ecf0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 shutdown_connections 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x563e26236de0 0x563e262391f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563e261d2b50 0x563e26235460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563e261c7380 0x563e261d6fd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd205ffb700 0 client.16005 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 --2- 172.21.15.31:0/1820919246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563e261c7c90 0x563e261d7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 >> 172.21.15.31:0/1820919246 conn(0x563e261bdca0 msgr2=0x563e261caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 shutdown_connections 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 1 -- 172.21.15.31:0/1820919246 wait complete. 2023-12-14T18:29:23.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd230d6eb80 20 client.16005 trim_cache size 0 max 16384 2023-12-14T18:29:23.546 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:23.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd2080a13e0 msgr2=0x7fd2080a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd224be0700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd2080a13e0 0x7fd2080a3890 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd210009c60 tx=0x7fd210009260 comp rx=0 tx=0).stop 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 msgr2=0x7fd2201a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.538+0000 7fd224be0700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd2201a46f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd218009fd0 tx=0x7fd2180094c0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 shutdown_connections 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd22010a7b0 0x7fd2201a46f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd2080a13e0 0x7fd2080a3890 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220101470 0x7fd2200ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 --2- 172.21.15.31:0/3113800877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd220101d80 0x7fd2200ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 >> 172.21.15.31:0/3113800877 conn(0x7fd2200763a0 msgr2=0x7fd2200fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:23.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 shutdown_connections 2023-12-14T18:29:23.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:23.539+0000 7fd224be0700 1 -- 172.21.15.31:0/3113800877 wait complete. 2023-12-14T18:29:23.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:29:23.777 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:23 smithi031 ceph-mon[42085]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-12-14T18:29:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:23 smithi204 ceph-mon[43866]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-12-14T18:29:23.865 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:23.865 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:23.971 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:23.971 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:23 smithi176 ceph-mon[43506]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-12-14T18:29:23.995 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:23.995 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:23.995 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 44 -lt 100 ']' 2023-12-14T18:29:23.996 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 44 + 1 2023-12-14T18:29:24.019 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=45 2023-12-14T18:29:24.019 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:24.038 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:24.038 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:24.039 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:24.039 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:24.039 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:25.023 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:25.024 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:25.154 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:25.154 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:25.178 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:25.178 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:25.179 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 45 -lt 100 ']' 2023-12-14T18:29:25.179 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 45 + 1 2023-12-14T18:29:25.211 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=46 2023-12-14T18:29:25.211 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:25.270 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:25.270 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:25.271 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:25.271 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:25.271 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:25.801 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:25 smithi031 ceph-mon[42085]: pgmap v51: 129 pgs: 129 active+clean; 2.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 18 op/s 2023-12-14T18:29:25.802 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:25 smithi031 ceph-mon[42085]: mgrmap e24: x(active, since 92s), standbys: z, y 2023-12-14T18:29:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:25 smithi204 ceph-mon[43866]: pgmap v51: 129 pgs: 129 active+clean; 2.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 18 op/s 2023-12-14T18:29:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:25 smithi204 ceph-mon[43866]: mgrmap e24: x(active, since 92s), standbys: z, y 2023-12-14T18:29:25.923 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:25.923 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:25 smithi176 ceph-mon[43506]: pgmap v51: 129 pgs: 129 active+clean; 2.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 18 op/s 2023-12-14T18:29:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:25 smithi176 ceph-mon[43506]: mgrmap e24: x(active, since 92s), standbys: z, y 2023-12-14T18:29:26.094 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:26.094 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:26.135 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:26.136 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:26.136 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 46 -lt 100 ']' 2023-12-14T18:29:26.136 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 46 + 1 2023-12-14T18:29:26.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=47 2023-12-14T18:29:26.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:26.183 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:26.183 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:26.184 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:26.184 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:26.184 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:26.798 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:26.798 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:26.969 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:26.969 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:26.985 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:26.985 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:26.985 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 47 -lt 100 ']' 2023-12-14T18:29:26.985 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 47 + 1 2023-12-14T18:29:27.014 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=48 2023-12-14T18:29:27.014 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:27.059 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:27.059 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:27.061 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:27.061 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:27.061 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 -- 172.21.15.31:0/1317096782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 msgr2=0x7facfc10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 --2- 172.21.15.31:0/1317096782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc10d0a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7facf400b0d0 tx=0x7facf400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 -- 172.21.15.31:0/1317096782 shutdown_connections 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 --2- 172.21.15.31:0/1317096782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc10d0a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 --2- 172.21.15.31:0/1317096782 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc0680c0 0x7facfc068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 --2- 172.21.15.31:0/1317096782 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc068a60 0x7facfc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 -- 172.21.15.31:0/1317096782 >> 172.21.15.31:0/1317096782 conn(0x7facfc075fd0 msgr2=0x7facfc0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 -- 172.21.15.31:0/1317096782 shutdown_connections 2023-12-14T18:29:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.387+0000 7fad016a6700 1 -- 172.21.15.31:0/1317096782 wait complete. 2023-12-14T18:29:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.388+0000 7fad016a6700 1 Processor -- start 2023-12-14T18:29:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 -- start start 2023-12-14T18:29:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc0680c0 0x7facfc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc068a60 0x7facfc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc19c5f0 con 0x7facfc1054d0 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc19c760 con 0x7facfc068a60 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7fad016a6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc19c920 con 0x7facfc0680c0 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56570/0 (socket says 172.21.15.31:56570) 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7facfb7fe700 1 -- 172.21.15.31:0/2224608778 learned_addr learned my addr 172.21.15.31:0/2224608778 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7facfa7fc700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc068a60 0x7facfc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.389+0000 7facfaffd700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc0680c0 0x7facfc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfb7fe700 1 -- 172.21.15.31:0/2224608778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc0680c0 msgr2=0x7facfc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfb7fe700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc0680c0 0x7facfc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfb7fe700 1 -- 172.21.15.31:0/2224608778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc068a60 msgr2=0x7facfc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfb7fe700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc068a60 0x7facfc198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfb7fe700 1 -- 172.21.15.31:0/2224608778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4009d00 con 0x7facfc1054d0 2023-12-14T18:29:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfa7fc700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc068a60 0x7facfc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7facfb7fe700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc19bff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7facf4007820 tx=0x7facf4007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7face3fff700 1 -- 172.21.15.31:0/2224608778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400e040 con 0x7facfc1054d0 2023-12-14T18:29:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7face3fff700 1 -- 172.21.15.31:0/2224608778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf400bcb0 con 0x7facfc1054d0 2023-12-14T18:29:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc1102f0 con 0x7facfc1054d0 2023-12-14T18:29:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7face3fff700 1 -- 172.21.15.31:0/2224608778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40058a0 con 0x7facfc1054d0 2023-12-14T18:29:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.390+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc110160 con 0x7facfc1054d0 2023-12-14T18:29:27.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.393+0000 7face3fff700 1 -- 172.21.15.31:0/2224608778 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7facf4004a50 con 0x7facfc1054d0 2023-12-14T18:29:27.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.394+0000 7face3fff700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7face40a12c0 0x7face40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.394+0000 7facfaffd700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7face40a12c0 0x7face40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.394+0000 7face3fff700 1 -- 172.21.15.31:0/2224608778 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7facf40cde40 con 0x7facfc1054d0 2023-12-14T18:29:27.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.394+0000 7facfaffd700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7face40a12c0 0x7face40a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facec009c60 tx=0x7facec009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.443+0000 7fad0d834b80 1 Processor -- start 2023-12-14T18:29:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.443+0000 7fad0d834b80 1 -- start start 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda6104750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda610fb50 0x55dda6111f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7fad0d834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5fee5e0 con 0x55dda6104380 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7fad0d834b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda605d870 con 0x55dda610fb50 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7fad0d834b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda6056d00 con 0x55dda6104c90 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda6104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda6104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56586/0 (socket says 172.21.15.31:56586) 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 -- 172.21.15.31:0/3803290736 learned_addr learned my addr 172.21.15.31:0/3803290736 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfa7fc700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfb7fe700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda610fb50 0x55dda6111f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 -- 172.21.15.31:0/3803290736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 msgr2=0x55dda610f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 -- 172.21.15.31:0/3803290736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda610fb50 msgr2=0x55dda6111f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.444+0000 7facfaffd700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda610fb50 0x55dda6111f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7facfaffd700 1 -- 172.21.15.31:0/3803290736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda5ed83b0 con 0x55dda6104380 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7facfa7fc700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7facfb7fe700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda610fb50 0x55dda6111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7facfaffd700 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda6104750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7facec00e4e0 tx=0x7facec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9791d1bd71df5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7face17fa700 1 -- 172.21.15.31:0/3803290736 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00ee00 con 0x55dda6104380 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7face17fa700 1 -- 172.21.15.31:0/3803290736 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facec021070 con 0x55dda6104380 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.445+0000 7face17fa700 1 -- 172.21.15.31:0/3803290736 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec01d7b0 con 0x55dda6104380 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 -- 172.21.15.31:0/3803290736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 msgr2=0x55dda6104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda6104750 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7facec00e4e0 tx=0x7facec014000 comp rx=0 tx=0).stop 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 -- 172.21.15.31:0/3803290736 shutdown_connections 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda610fb50 0x55dda6111f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda6104750 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3803290736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 -- 172.21.15.31:0/3803290736 >> 172.21.15.31:0/3803290736 conn(0x55dda60faca0 msgr2=0x55dda61007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 -- 172.21.15.31:0/3803290736 shutdown_connections 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 -- 172.21.15.31:0/3803290736 wait complete. 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.446+0000 7fad0d834b80 1 -- start start 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 Processor -- start 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 0x55dda610f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda610cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5ff2f90 con 0x55dda610fb50 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5ff2c30 con 0x55dda6104380 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5eda6a0 con 0x55dda6104c90 2023-12-14T18:29:27.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7fad0d834b80 10 client.0 _ll_register_callbacks cb 0x55dda6016910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda610cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7facfaffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 0x55dda610f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda610cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56596/0 (socket says 172.21.15.31:56596) 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7facfaffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 0x55dda610f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:38704/0 (socket says 172.21.15.31:38704) 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7facfb7fe700 1 -- 172.21.15.31:0/855107703 learned_addr learned my addr 172.21.15.31:0/855107703 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.447+0000 7facfa7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfb7fe700 1 -- 172.21.15.31:0/855107703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 msgr2=0x55dda610c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfb7fe700 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfb7fe700 1 -- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 msgr2=0x55dda610f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfb7fe700 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 0x55dda610f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfb7fe700 1 -- 172.21.15.31:0/855107703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec017070 con 0x55dda610fb50 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfaffd700 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 0x55dda610f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facfb7fe700 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda610cd70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7facf409a910 tx=0x7facf409a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d334164c347b00bf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:27.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facdeff5700 1 -- 172.21.15.31:0/855107703 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf409eea0 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facdeff5700 1 -- 172.21.15.31:0/855107703 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7facf400be20 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dda60f82b0 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7facdeff5700 1 -- 172.21.15.31:0/855107703 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf409ac10 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7fad0d834b80 10 client.16017 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.448+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dda601d170 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7facdeff5700 1 -- 172.21.15.31:0/855107703 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf40ca080 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7facdeff5700 1 -- 172.21.15.31:0/855107703 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facf409ae70 con 0x55dda610fb50 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7fad0d834b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7fad0d834b80 10 client.16017 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7fad0d834b80 20 client.16017 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:27.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7fad0d834b80 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda611f200 0x55dda6121610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7fad0d834b80 4 client.16017 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:27.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dda604fce0 con 0x55dda611f200 2023-12-14T18:29:27.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.449+0000 7facfaffd700 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda611f200 0x55dda6121610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.450+0000 7facdeff5700 10 client.16017 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:27.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.450+0000 7facfaffd700 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda611f200 0x55dda6121610 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc5a77933d5ae020 server_cookie=6332dbc3b32a3f02 in_seq=0 out_seq=0 2023-12-14T18:29:27.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7facdeff5700 1 -- 172.21.15.31:0/855107703 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dda604fce0 con 0x55dda611f200 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7facdeff5700 10 client.16017 handle_command_reply: tid=0 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 client.16017 shutdown 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 msgr2=0x55dda610cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda610cd70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7facf409a910 tx=0x7facf409a750 comp rx=0 tx=0).stop 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 shutdown_connections 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda610cd70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104380 0x55dda610f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda611f200 0x55dda6121610 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 --2- 172.21.15.31:0/855107703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104c90 0x55dda610c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 >> 172.21.15.31:0/855107703 conn(0x55dda60faca0 msgr2=0x55dda61003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:27.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7facdeff5700 0 client.16017 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:27.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 shutdown_connections 2023-12-14T18:29:27.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 1 -- 172.21.15.31:0/855107703 wait complete. 2023-12-14T18:29:27.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.451+0000 7fad0d834b80 20 client.16017 trim_cache size 0 max 16384 2023-12-14T18:29:27.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 Processor -- start 2023-12-14T18:29:27.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 -- start start 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda610efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda610f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda610fb50 0x55dda6107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda6105390 con 0x55dda6104380 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda61055b0 con 0x55dda6104c90 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7fad0d834b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda6101bc0 con 0x55dda610fb50 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.466+0000 7facfaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda610efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda610efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56606/0 (socket says 172.21.15.31:56606) 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 -- 172.21.15.31:0/1843175071 learned_addr learned my addr 172.21.15.31:0/1843175071 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfa7fc700 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda610f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfb7fe700 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda610fb50 0x55dda6107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 -- 172.21.15.31:0/1843175071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda610fb50 msgr2=0x55dda6107ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda610fb50 0x55dda6107ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 -- 172.21.15.31:0/1843175071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 msgr2=0x55dda610f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda610f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 -- 172.21.15.31:0/1843175071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda601cec0 con 0x55dda6104380 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfa7fc700 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda610f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7facfaffd700 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda610efc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7facec00efd0 tx=0x7facec009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2460b48a01b7d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7face1ffb700 1 -- 172.21.15.31:0/1843175071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec00ecc0 con 0x55dda6104380 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7face1ffb700 1 -- 172.21.15.31:0/1843175071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facec033170 con 0x55dda6104380 2023-12-14T18:29:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.467+0000 7face1ffb700 1 -- 172.21.15.31:0/1843175071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec005620 con 0x55dda6104380 2023-12-14T18:29:27.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 -- 172.21.15.31:0/1843175071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 msgr2=0x55dda610efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda610efc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7facec00efd0 tx=0x7facec009e50 comp rx=0 tx=0).stop 2023-12-14T18:29:27.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 -- 172.21.15.31:0/1843175071 shutdown_connections 2023-12-14T18:29:27.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda610fb50 0x55dda6107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda6104380 0x55dda610efc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 --2- 172.21.15.31:0/1843175071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda610f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 -- 172.21.15.31:0/1843175071 >> 172.21.15.31:0/1843175071 conn(0x55dda60faca0 msgr2=0x55dda6121210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 -- 172.21.15.31:0/1843175071 shutdown_connections 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.468+0000 7fad0d834b80 1 -- 172.21.15.31:0/1843175071 wait complete. 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 -- start start 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 Processor -- start 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 0x55dda6113c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda61141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda61146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda6116cd0 con 0x55dda610fb50 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda6172080 con 0x55dda6104c90 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda6172210 con 0x55dda6104380 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.469+0000 7fad0d834b80 10 client.0 _ll_register_callbacks cb 0x7facf4009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 0x55dda6113c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda61146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 0x55dda6113c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:41956/0 (socket says 172.21.15.31:41956) 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda61146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56608/0 (socket says 172.21.15.31:56608) 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 -- 172.21.15.31:0/3879072157 learned_addr learned my addr 172.21.15.31:0/3879072157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfa7fc700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda61141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 -- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 msgr2=0x55dda61141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda61141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 -- 172.21.15.31:0/3879072157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 msgr2=0x55dda61146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda61146f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 -- 172.21.15.31:0/3879072157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf0011440 con 0x55dda6104380 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfa7fc700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda61141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfb7fe700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda61146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facfaffd700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 0x55dda6113c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7facec00ed70 tx=0x7facec00eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ed1c843625db39 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.470+0000 7facdeff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7facdeff5700 1 -- 172.21.15.31:0/3879072157 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec03f070 con 0x55dda6104380 2023-12-14T18:29:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facec021070 con 0x55dda6104380 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7facdeff5700 1 -- 172.21.15.31:0/3879072157 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7facec033460 con 0x55dda6104380 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7facdeff5700 1 -- 172.21.15.31:0/3879072157 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7facec033ae0 con 0x55dda6104380 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 10 client.25876 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7facec02a040 con 0x55dda6104380 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7facdeff5700 1 -- 172.21.15.31:0/3879072157 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facec021070 con 0x55dda6104380 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7facdeff5700 1 -- 172.21.15.31:0/3879072157 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facec042030 con 0x55dda6104380 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 10 client.25876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 20 client.25876 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda6173460 0x55dda6175840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 4 client.25876 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.471+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dda604fa00 con 0x55dda6173460 2023-12-14T18:29:27.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.472+0000 7facfa7fc700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda6173460 0x55dda6175840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.472+0000 7facfa7fc700 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda6173460 0x55dda6175840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=297c09c1f6d9c5ee server_cookie=6743d0528d362184 in_seq=0 out_seq=0 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.472+0000 7facdeff5700 10 client.25876 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7facdeff5700 1 -- 172.21.15.31:0/3879072157 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dda604fa00 con 0x55dda6173460 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7facdeff5700 10 client.25876 handle_command_reply: tid=0 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 client.25876 shutdown 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 msgr2=0x55dda6113c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 0x55dda6113c70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7facec00ed70 tx=0x7facec00eda0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 shutdown_connections 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dda610fb50 0x55dda61146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dda6173460 0x55dda6175840 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dda6104380 0x55dda6113c70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 --2- 172.21.15.31:0/3879072157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dda6104c90 0x55dda61141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7facdeff5700 0 client.25876 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 >> 172.21.15.31:0/3879072157 conn(0x55dda60faca0 msgr2=0x55dda6107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 shutdown_connections 2023-12-14T18:29:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 1 -- 172.21.15.31:0/3879072157 wait complete. 2023-12-14T18:29:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.473+0000 7fad0d834b80 20 client.25876 trim_cache size 0 max 16384 2023-12-14T18:29:27.479 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7face40a12c0 msgr2=0x7face40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7face40a12c0 0x7face40a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facec009c60 tx=0x7facec009260 comp rx=0 tx=0).stop 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 msgr2=0x7facfc19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc19bff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7facf4007820 tx=0x7facf4007850 comp rx=0 tx=0).stop 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 shutdown_connections 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facfc1054d0 0x7facfc19bff0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7face40a12c0 0x7face40a3770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facfc0680c0 0x7facfc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 --2- 172.21.15.31:0/2224608778 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7facfc068a60 0x7facfc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 >> 172.21.15.31:0/2224608778 conn(0x7facfc075fd0 msgr2=0x7facfc0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 shutdown_connections 2023-12-14T18:29:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:27.474+0000 7fad016a6700 1 -- 172.21.15.31:0/2224608778 wait complete. 2023-12-14T18:29:27.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:29:27.656 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:27.656 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:27.810 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:27.810 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:27.820 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:27.820 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:27.820 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 48 -lt 100 ']' 2023-12-14T18:29:27.820 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 48 + 1 2023-12-14T18:29:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:27 smithi204 ceph-mon[43866]: pgmap v52: 129 pgs: 129 active+clean; 2.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-12-14T18:29:27.860 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=49 2023-12-14T18:29:27.860 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:27.872 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:27.872 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:27.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:27.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:27.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:27 smithi031 ceph-mon[42085]: pgmap v52: 129 pgs: 129 active+clean; 2.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-12-14T18:29:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:27 smithi176 ceph-mon[43506]: pgmap v52: 129 pgs: 129 active+clean; 2.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 13 op/s 2023-12-14T18:29:28.414 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:28.415 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:28.490 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:28.490 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:28.510 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:28.510 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:28.510 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 49 -lt 100 ']' 2023-12-14T18:29:28.510 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 49 + 1 2023-12-14T18:29:28.560 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=50 2023-12-14T18:29:28.560 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:28.624 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:28.624 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:28.625 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:28.626 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:28.626 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:29.273 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:29.273 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:29.485 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:29.485 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:29.493 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:29.493 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:29.494 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 50 -lt 100 ']' 2023-12-14T18:29:29.494 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 50 + 1 2023-12-14T18:29:29.536 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=51 2023-12-14T18:29:29.536 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:29.602 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:29.602 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:29.603 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:29.603 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:29.603 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:29 smithi204 ceph-mon[43866]: pgmap v53: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-12-14T18:29:29.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:29 smithi031 ceph-mon[42085]: pgmap v53: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-12-14T18:29:29.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:29 smithi176 ceph-mon[43506]: pgmap v53: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-12-14T18:29:30.830 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:30.831 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:31.085 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:31.085 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:31.102 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:31.103 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:31.103 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 51 -lt 100 ']' 2023-12-14T18:29:31.103 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 51 + 1 2023-12-14T18:29:31.134 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=52 2023-12-14T18:29:31.134 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:31.218 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:31.218 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:31.220 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:31.220 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:31.220 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 -- 172.21.15.31:0/2916685867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc0680c0 msgr2=0x7f2fdc068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2916685867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc0680c0 0x7f2fdc068490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0099c0 tx=0x7f2fcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 -- 172.21.15.31:0/2916685867 shutdown_connections 2023-12-14T18:29:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2916685867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2fdc1054d0 0x7f2fdc10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2916685867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc0680c0 0x7f2fdc068490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2916685867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fdc068a60 0x7f2fdc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 -- 172.21.15.31:0/2916685867 >> 172.21.15.31:0/2916685867 conn(0x7f2fdc075fd0 msgr2=0x7f2fdc0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 -- 172.21.15.31:0/2916685867 shutdown_connections 2023-12-14T18:29:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.361+0000 7f2fe29de700 1 -- 172.21.15.31:0/2916685867 wait complete. 2023-12-14T18:29:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 Processor -- start 2023-12-14T18:29:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 -- start start 2023-12-14T18:29:31.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fdc0680c0 0x7f2fdc1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 0x7f2fdc196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2fdc1054d0 0x7f2fdc19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc19b3d0 con 0x7f2fdc068a60 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc19b540 con 0x7f2fdc1054d0 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.362+0000 7f2fe29de700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc19b700 con 0x7f2fdc0680c0 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 0x7f2fdc196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 0x7f2fdc196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56622/0 (socket says 172.21.15.31:56622) 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 -- 172.21.15.31:0/2692544094 learned_addr learned my addr 172.21.15.31:0/2692544094 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fdc0680c0 0x7f2fdc1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2fdc1054d0 0x7f2fdc19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 -- 172.21.15.31:0/2692544094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fdc0680c0 msgr2=0x7f2fdc1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fdc0680c0 0x7f2fdc1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 -- 172.21.15.31:0/2692544094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2fdc1054d0 msgr2=0x7f2fdc19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2fdc1054d0 0x7f2fdc19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 -- 172.21.15.31:0/2692544094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0095f0 con 0x7f2fdc068a60 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 0x7f2fdc196930 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e940 tx=0x7f2fd000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.363+0000 7f2fd97fa700 1 -- 172.21.15.31:0/2692544094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0009730 con 0x7f2fdc068a60 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.364+0000 7f2fd97fa700 1 -- 172.21.15.31:0/2692544094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fd0010430 con 0x7f2fdc068a60 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.364+0000 7f2fd97fa700 1 -- 172.21.15.31:0/2692544094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000cea0 con 0x7f2fdc068a60 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.364+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc1102e0 con 0x7f2fdc068a60 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.364+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc110150 con 0x7f2fdc068a60 2023-12-14T18:29:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.366+0000 7f2fd97fa700 1 -- 172.21.15.31:0/2692544094 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2fd0010810 con 0x7f2fdc068a60 2023-12-14T18:29:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.367+0000 7f2fd97fa700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2fc40a12c0 0x7f2fc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.367+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2fc40a12c0 0x7f2fc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.367+0000 7f2fd97fa700 1 -- 172.21.15.31:0/2692544094 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2fd0014070 con 0x7f2fdc068a60 2023-12-14T18:29:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.368+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2fc40a12c0 0x7f2fc40a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009990 tx=0x7f2fcc005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:31.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 Processor -- start 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 -- start start 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a39884e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 0x55a398860340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3987385e0 con 0x55a39885df60 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3987a7870 con 0x55a39884ec90 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.433+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3987a0d00 con 0x55a39884e380 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 0x55a398860340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 0x55a398860340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56632/0 (socket says 172.21.15.31:56632) 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 -- 172.21.15.31:0/2892993773 learned_addr learned my addr 172.21.15.31:0/2892993773 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:31.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a39884e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 -- 172.21.15.31:0/2892993773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 msgr2=0x55a39884e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a39884e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 -- 172.21.15.31:0/2892993773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 msgr2=0x55a39885da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 -- 172.21.15.31:0/2892993773 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3986223b0 con 0x55a39885df60 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 0x55a398860340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4011680 tx=0x7f2fd4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50790dc335482b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.434+0000 7f2fc27fc700 1 -- 172.21.15.31:0/2892993773 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400e040 con 0x55a39885df60 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.435+0000 7f2fc27fc700 1 -- 172.21.15.31:0/2892993773 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fd400f040 con 0x55a39885df60 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.435+0000 7f2fc27fc700 1 -- 172.21.15.31:0/2892993773 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004860 con 0x55a39885df60 2023-12-14T18:29:31.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.435+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/2892993773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 msgr2=0x55a398860340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.435+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 0x55a398860340 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4011680 tx=0x7f2fd4009920 comp rx=0 tx=0).stop 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/2892993773 shutdown_connections 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39885df60 0x55a398860340 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a39884e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/2892993773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/2892993773 >> 172.21.15.31:0/2892993773 conn(0x55a398844ca0 msgr2=0x55a39884a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/2892993773 shutdown_connections 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/2892993773 wait complete. 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.436+0000 7f2feeb6cb80 1 -- start start 2023-12-14T18:29:31.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 Processor -- start 2023-12-14T18:29:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39885df60 0x55a39885b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39873cf90 con 0x55a39884e380 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39873cc30 con 0x55a39884ec90 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3986246a0 con 0x55a39885df60 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2feeb6cb80 10 client.0 _ll_register_callbacks cb 0x55a398760910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.437+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56634/0 (socket says 172.21.15.31:56634) 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 -- 172.21.15.31:0/1956459052 learned_addr learned my addr 172.21.15.31:0/1956459052 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39885df60 0x55a39885b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 -- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39885df60 msgr2=0x55a39885b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39885df60 0x55a39885b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 -- 172.21.15.31:0/1956459052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 msgr2=0x55a39885ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 -- 172.21.15.31:0/1956459052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd4011330 con 0x55a39884e380 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 0x55a39885d8e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009fd0 tx=0x7f2fcc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36d0bd52a973666 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.438+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/1956459052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc02c040 con 0x55a39884e380 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/1956459052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2fcc004460 con 0x55a39884e380 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3988422b0 con 0x55a39884e380 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/1956459052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fcc004660 con 0x55a39884e380 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 10 client.16035 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a398767170 con 0x55a39884e380 2023-12-14T18:29:31.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/1956459052 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3988422b0 con 0x55a39884e380 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/1956459052 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2fcc026430 con 0x55a39884e380 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 10 client.16035 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 20 client.16035 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39886d8b0 0x55a39886fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 4 client.16035 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.439+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a398799ce0 con 0x55a39886d8b0 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.440+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39886d8b0 0x55a39886fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.440+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39886d8b0 0x55a39886fc90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ccead8696c7a59 server_cookie=7f7db090abb107ef in_seq=0 out_seq=0 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.440+0000 7f2fbb7fe700 10 client.16035 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/1956459052 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a398799ce0 con 0x55a39886d8b0 2023-12-14T18:29:31.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2fbb7fe700 10 client.16035 handle_command_reply: tid=0 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 client.16035 shutdown 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 msgr2=0x55a39885d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 0x55a39885d8e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009fd0 tx=0x7f2fcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 shutdown_connections 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39886d8b0 0x55a39886fc90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39885df60 0x55a39885b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2fbb7fe700 0 client.16035 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/1956459052 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a39885ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.441+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 >> 172.21.15.31:0/1956459052 conn(0x55a398844ca0 msgr2=0x55a39884a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.442+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 shutdown_connections 2023-12-14T18:29:31.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.442+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/1956459052 wait complete. 2023-12-14T18:29:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.442+0000 7f2feeb6cb80 20 client.16035 trim_cache size 0 max 16384 2023-12-14T18:29:31.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 Processor -- start 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 -- start start 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a39885d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884ec90 0x55a398855ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39884f390 con 0x55a398844ca0 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39884f5b0 con 0x55a39884e380 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.456+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a39885b710 con 0x55a39884ec90 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a39885d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a39885d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56648/0 (socket says 172.21.15.31:56648) 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3944151418 learned_addr learned my addr 172.21.15.31:0/3944151418 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:31.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884ec90 0x55a398855ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3944151418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884ec90 msgr2=0x55a398855ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884ec90 0x55a398855ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3944151418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884e380 msgr2=0x55a39885d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3944151418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a398766ec0 con 0x55a398844ca0 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884ec90 0x55a398855ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a39885d3a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0099c0 tx=0x7f2fcc009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb537755dc5e5c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fc2ffd700 1 -- 172.21.15.31:0/3944151418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc02c050 con 0x55a398844ca0 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fc2ffd700 1 -- 172.21.15.31:0/3944151418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fcc01e040 con 0x55a398844ca0 2023-12-14T18:29:31.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.457+0000 7f2fc2ffd700 1 -- 172.21.15.31:0/3944151418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00f6c0 con 0x55a398844ca0 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3944151418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 msgr2=0x55a39885d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a39885d3a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0099c0 tx=0x7f2fcc009290 comp rx=0 tx=0).stop 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3944151418 shutdown_connections 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884ec90 0x55a398855ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a39885d3a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3944151418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884e380 0x55a39885d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3944151418 >> 172.21.15.31:0/3944151418 conn(0x55a398860610 msgr2=0x55a398869c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3944151418 shutdown_connections 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.458+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3944151418 wait complete. 2023-12-14T18:29:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.459+0000 7f2feeb6cb80 1 -- start start 2023-12-14T18:29:31.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.459+0000 7f2feeb6cb80 1 Processor -- start 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a398862490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a3988629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a3988c0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a398863270 con 0x55a398844ca0 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3988c0c80 con 0x55a39884ec90 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3988c0dc0 con 0x55a39884e380 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2feeb6cb80 10 client.0 _ll_register_callbacks cb 0x7f2fcc003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a398862490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a398862490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56660/0 (socket says 172.21.15.31:56660) 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3591083656 learned_addr learned my addr 172.21.15.31:0/3591083656 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:31.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a3988629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fe0f7b700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a3988c0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 msgr2=0x55a3988629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a3988629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3591083656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 msgr2=0x55a3988c0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a3988c0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 -- 172.21.15.31:0/3591083656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a398767170 con 0x55a398844ca0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fdbfff700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a398862490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00a010 tx=0x7f2fcc00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a200d0eb3f27584b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.460+0000 7f2fbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/3591083656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc02c040 con 0x55a398844ca0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/3591083656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2fcc0054d0 con 0x55a398844ca0 2023-12-14T18:29:31.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fd400f040 con 0x55a398844ca0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/3591083656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fcc005690 con 0x55a398844ca0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 10 client.16041 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a39874d9a0 con 0x55a398844ca0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/3591083656 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2fd400f040 con 0x55a398844ca0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/3591083656 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2fcc049020 con 0x55a398844ca0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 10 client.16041 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 20 client.16041 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39885df60 0x55a3988c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 4 client.16041 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.461+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a398799a00 con 0x55a39885df60 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.462+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39885df60 0x55a3988c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.462+0000 7f2fdb7fe700 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39885df60 0x55a3988c3fd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a90657d99acc6bf0 server_cookie=adf37d4ce1835ee1 in_seq=0 out_seq=0 2023-12-14T18:29:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.462+0000 7f2fbb7fe700 10 client.16041 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2fbb7fe700 1 -- 172.21.15.31:0/3591083656 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a398799a00 con 0x55a39885df60 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2fbb7fe700 10 client.16041 handle_command_reply: tid=0 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 client.16041 shutdown 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 msgr2=0x55a398862490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a398862490 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00a010 tx=0x7f2fcc00bfa0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 shutdown_connections 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a39885df60 0x55a3988c3fd0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a39884ec90 0x55a3988c0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a398844ca0 0x55a398862490 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2fbb7fe700 0 client.16041 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:31.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 --2- 172.21.15.31:0/3591083656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a39884e380 0x55a3988629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 >> 172.21.15.31:0/3591083656 conn(0x55a398860610 msgr2=0x55a398855ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 shutdown_connections 2023-12-14T18:29:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 1 -- 172.21.15.31:0/3591083656 wait complete. 2023-12-14T18:29:31.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.463+0000 7f2feeb6cb80 20 client.16041 trim_cache size 0 max 16384 2023-12-14T18:29:31.472 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2fc40a12c0 msgr2=0x7f2fc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2fc40a12c0 0x7f2fc40a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009990 tx=0x7f2fcc005fb0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 msgr2=0x7f2fdc196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 0x7f2fdc196930 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000e940 tx=0x7f2fd000ec50 comp rx=0 tx=0).stop 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 shutdown_connections 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2fdc1054d0 0x7f2fdc19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2fc40a12c0 0x7f2fc40a3770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fdc0680c0 0x7f2fdc1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 --2- 172.21.15.31:0/2692544094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fdc068a60 0x7f2fdc196930 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 >> 172.21.15.31:0/2692544094 conn(0x7f2fdc075fd0 msgr2=0x7f2fdc0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 shutdown_connections 2023-12-14T18:29:31.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:31.464+0000 7f2fe29de700 1 -- 172.21.15.31:0/2692544094 wait complete. 2023-12-14T18:29:31.805 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:31 smithi031 ceph-mon[42085]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-12-14T18:29:31.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:31 smithi204 ceph-mon[43866]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-12-14T18:29:31.880 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:31.880 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:31 smithi176 ceph-mon[43506]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 16 op/s 2023-12-14T18:29:32.035 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:32.035 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:32.068 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:32.068 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:32.068 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 52 -lt 100 ']' 2023-12-14T18:29:32.068 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 52 + 1 2023-12-14T18:29:32.093 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=53 2023-12-14T18:29:32.093 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:32.117 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:32.117 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:32.118 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:32.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:32.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:32.430 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:32.430 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:32.520 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:29:32.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:29:32.578 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:32.578 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:32.593 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:32.593 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:32.593 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 53 -lt 100 ']' 2023-12-14T18:29:32.593 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 53 + 1 2023-12-14T18:29:32.618 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=54 2023-12-14T18:29:32.618 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:32.638 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:32.638 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:32.639 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:32.639 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:32.639 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:32.964 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:32.964 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:33.114 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:33.114 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:33.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:33.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:33.119 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 54 -lt 100 ']' 2023-12-14T18:29:33.119 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 54 + 1 2023-12-14T18:29:33.168 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=55 2023-12-14T18:29:33.168 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:33.198 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:33.199 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:33.200 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:33.200 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:33.200 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:33.680 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:33.680 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:33.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:33 smithi204 ceph-mon[43866]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-12-14T18:29:33.884 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:33.884 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:33.901 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:33.901 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:33.902 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 55 -lt 100 ']' 2023-12-14T18:29:33.902 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 55 + 1 2023-12-14T18:29:33.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:33 smithi031 ceph-mon[42085]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-12-14T18:29:33.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:33 smithi176 ceph-mon[43506]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-12-14T18:29:34.001 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=56 2023-12-14T18:29:34.001 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:34.065 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:34.065 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:34.067 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:34.067 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:34.067 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:34.655 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:34.655 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:34.834 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:34.834 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:34.885 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:34.886 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:34.886 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 56 -lt 100 ']' 2023-12-14T18:29:34.886 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 56 + 1 2023-12-14T18:29:34.917 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=57 2023-12-14T18:29:34.917 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:34.945 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:34.945 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:34.947 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:34.947 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:34.947 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:35.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fc1f4396700 1 -- 172.21.15.31:0/286708784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 msgr2=0x7fc1ec1016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fc1f4396700 1 --2- 172.21.15.31:0/286708784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1016c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009990 tx=0x7fc1e8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fca463ae700 1 -- 172.21.15.31:0/3441549474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca40104400 msgr2=0x7fca4010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fca463ae700 1 --2- 172.21.15.31:0/3441549474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca40104400 0x7fca4010bfd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca38009fc0 tx=0x7fca3800b330 comp rx=0 tx=0).stop 2023-12-14T18:29:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fc1f4396700 1 -- 172.21.15.31:0/286708784 shutdown_connections 2023-12-14T18:29:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fc1f4396700 1 --2- 172.21.15.31:0/286708784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec10a810 0x7fc1ec10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fc1f4396700 1 --2- 172.21.15.31:0/286708784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1016c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.302+0000 7fc1f4396700 1 --2- 172.21.15.31:0/286708784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec101c00 0x7fc1ec10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fc1f4396700 1 -- 172.21.15.31:0/286708784 >> 172.21.15.31:0/286708784 conn(0x7fc1ec0763a0 msgr2=0x7fc1ec0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 -- 172.21.15.31:0/3441549474 shutdown_connections 2023-12-14T18:29:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 --2- 172.21.15.31:0/3441549474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca40104400 0x7fca4010bfd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fc1f4396700 1 -- 172.21.15.31:0/286708784 shutdown_connections 2023-12-14T18:29:35.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fc1f4396700 1 -- 172.21.15.31:0/286708784 wait complete. 2023-12-14T18:29:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 --2- 172.21.15.31:0/3441549474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca400ff3c0 0x7fca400ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 --2- 172.21.15.31:0/3441549474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca400ffd60 0x7fca40103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 -- 172.21.15.31:0/3441549474 >> 172.21.15.31:0/3441549474 conn(0x7fca400763a0 msgr2=0x7fca400767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 -- 172.21.15.31:0/3441549474 shutdown_connections 2023-12-14T18:29:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.303+0000 7fca463ae700 1 -- 172.21.15.31:0/3441549474 wait complete. 2023-12-14T18:29:35.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 Processor -- start 2023-12-14T18:29:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fca463ae700 1 Processor -- start 2023-12-14T18:29:35.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 -- start start 2023-12-14T18:29:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fca463ae700 1 -- start start 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec101c00 0x7fc1ec196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec10a810 0x7fc1ec19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec19b3c0 con 0x7fc1ec101c00 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec19b530 con 0x7fc1ec10a810 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f4396700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec19b6f0 con 0x7fc1ec1012f0 2023-12-14T18:29:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f1931700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec101c00 0x7fc1ec196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f1931700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec101c00 0x7fc1ec196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37934/0 (socket says 172.21.15.31:37934) 2023-12-14T18:29:35.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.304+0000 7fc1f1931700 1 -- 172.21.15.31:0/3386225786 learned_addr learned my addr 172.21.15.31:0/3386225786 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca463ae700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 0x7fca401964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca463ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca400ffd60 0x7fca40196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca463ae700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca40104400 0x7fca4019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca463ae700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4019b4a0 con 0x7fca400ffd60 2023-12-14T18:29:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca463ae700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4019b610 con 0x7fca40104400 2023-12-14T18:29:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca463ae700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4019b7d0 con 0x7fca400ff3c0 2023-12-14T18:29:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2933700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec10a810 0x7fc1ec19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2132700 1 -- 172.21.15.31:0/3386225786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec10a810 msgr2=0x7fc1ec19ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2132700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec10a810 0x7fc1ec19ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2132700 1 -- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec101c00 msgr2=0x7fc1ec196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2132700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec101c00 0x7fc1ec196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2132700 1 -- 172.21.15.31:0/3386225786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e80095f0 con 0x7fc1ec1012f0 2023-12-14T18:29:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2933700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec10a810 0x7fc1ec19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:29:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 0x7fca401964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca400ffd60 0x7fca40196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 0x7fca401964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46848/0 (socket says 172.21.15.31:46848) 2023-12-14T18:29:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 -- 172.21.15.31:0/4048223929 learned_addr learned my addr 172.21.15.31:0/4048223929 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca4494b700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca40104400 0x7fca4019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f2132700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1963e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8000c00 tx=0x7fc1e8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 -- 172.21.15.31:0/4048223929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca40104400 msgr2=0x7fca4019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca40104400 0x7fca4019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 -- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca400ffd60 msgr2=0x7fca40196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca400ffd60 0x7fca40196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca3ffff700 1 -- 172.21.15.31:0/4048223929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38009c40 con 0x7fca400ff3c0 2023-12-14T18:29:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fca4494b700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca40104400 0x7fca4019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1e37fe700 1 -- 172.21.15.31:0/3386225786 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e801d070 con 0x7fc1ec1012f0 2023-12-14T18:29:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1e37fe700 1 -- 172.21.15.31:0/3386225786 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e8021410 con 0x7fc1ec1012f0 2023-12-14T18:29:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.305+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ec1102e0 con 0x7fc1ec1012f0 2023-12-14T18:29:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fc1e37fe700 1 -- 172.21.15.31:0/3386225786 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8017650 con 0x7fc1ec1012f0 2023-12-14T18:29:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fca3ffff700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 0x7fca401964c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca3000e9a0 tx=0x7fca3000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec110150 con 0x7fc1ec1012f0 2023-12-14T18:29:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fca3d7fa700 1 -- 172.21.15.31:0/4048223929 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3000eee0 con 0x7fca400ff3c0 2023-12-14T18:29:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fca3d7fa700 1 -- 172.21.15.31:0/4048223929 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca3000cbd0 con 0x7fca400ff3c0 2023-12-14T18:29:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca40110390 con 0x7fca400ff3c0 2023-12-14T18:29:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fca3d7fa700 1 -- 172.21.15.31:0/4048223929 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30009790 con 0x7fca400ff3c0 2023-12-14T18:29:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.306+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca40110200 con 0x7fca400ff3c0 2023-12-14T18:29:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.309+0000 7fc1e37fe700 1 -- 172.21.15.31:0/3386225786 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc1e80217f0 con 0x7fc1ec1012f0 2023-12-14T18:29:35.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.309+0000 7fca3d7fa700 1 -- 172.21.15.31:0/4048223929 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fca3000cd40 con 0x7fca400ff3c0 2023-12-14T18:29:35.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.309+0000 7fc1e37fe700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc1d80a1340 0x7fc1d80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.309+0000 7fc1f1931700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc1d80a1340 0x7fc1d80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.310+0000 7fc1e37fe700 1 -- 172.21.15.31:0/3386225786 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc1e80ce440 con 0x7fc1ec1012f0 2023-12-14T18:29:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.310+0000 7fca3d7fa700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fca280a1340 0x7fca280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.310+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fca280a1340 0x7fca280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.310+0000 7fca3d7fa700 1 -- 172.21.15.31:0/4048223929 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fca30014070 con 0x7fca400ff3c0 2023-12-14T18:29:35.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.310+0000 7fc1f1931700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc1d80a1340 0x7fc1d80a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec1979f0 tx=0x7fc1dc009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.311+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fca280a1340 0x7fca280a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fca40197ad0 tx=0x7fca34009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.383+0000 7fc200524b80 1 Processor -- start 2023-12-14T18:29:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.383+0000 7fca5253cb80 1 Processor -- start 2023-12-14T18:29:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.383+0000 7fca5253cb80 1 -- start start 2023-12-14T18:29:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.383+0000 7fc200524b80 1 -- start start 2023-12-14T18:29:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db95fde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db95fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca5253cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db94e8580 con 0x560db960d500 2023-12-14T18:29:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca5253cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db9557810 con 0x560db95fe370 2023-12-14T18:29:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca5253cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db9550ca0 con 0x560db945b430 2023-12-14T18:29:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc200524b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 0x55ff611ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc200524b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc200524b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611f5bd0 0x55ff611f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc200524b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff610d4660 con 0x55ff611f5bd0 2023-12-14T18:29:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc200524b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff611438f0 con 0x55ff611ea400 2023-12-14T18:29:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc200524b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6113cd80 con 0x55ff611ead10 2023-12-14T18:29:35.389 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db95fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 0x55ff611ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc1f1931700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc1f1931700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46880/0 (socket says 172.21.15.31:46880) 2023-12-14T18:29:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 0x55ff611ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:52784/0 (socket says 172.21.15.31:52784) 2023-12-14T18:29:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc1f1931700 1 -- 172.21.15.31:0/612003349 learned_addr learned my addr 172.21.15.31:0/612003349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db95fe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:52776/0 (socket says 172.21.15.31:52776) 2023-12-14T18:29:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 -- 172.21.15.31:0/2570252730 learned_addr learned my addr 172.21.15.31:0/2570252730 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca4494b700 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.391 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3ffff700 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db95fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fc1f2933700 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611f5bd0 0x55ff611f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 -- 172.21.15.31:0/2570252730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 msgr2=0x560db95fde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db95fde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 -- 172.21.15.31:0/2570252730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 msgr2=0x560db960f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.384+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fca3f7fe700 1 -- 172.21.15.31:0/2570252730 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db93d2350 con 0x560db95fe370 2023-12-14T18:29:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1f2132700 1 -- 172.21.15.31:0/612003349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 msgr2=0x55ff611f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1f2132700 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1f2132700 1 -- 172.21.15.31:0/612003349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611f5bd0 msgr2=0x55ff611f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1f2132700 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611f5bd0 0x55ff611f7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1f2132700 1 -- 172.21.15.31:0/612003349 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff60fbe430 con 0x55ff611ea400 2023-12-14T18:29:35.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1f2132700 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 0x55ff611ea7d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80cb7a0 tx=0x7fc1e809b0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88095cc01d2d6695 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db95fe7e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca34011cf0 tx=0x7fca34014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34b409ef5d563c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1e0dac700 1 -- 172.21.15.31:0/612003349 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e809ab10 con 0x55ff611ea400 2023-12-14T18:29:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1e0dac700 1 -- 172.21.15.31:0/612003349 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e8021e90 con 0x55ff611ea400 2023-12-14T18:29:35.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fc1e0dac700 1 -- 172.21.15.31:0/612003349 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e800fad0 con 0x55ff611ea400 2023-12-14T18:29:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fca267fc700 1 -- 172.21.15.31:0/2570252730 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34004830 con 0x560db95fe370 2023-12-14T18:29:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fca267fc700 1 -- 172.21.15.31:0/2570252730 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca34004e50 con 0x560db95fe370 2023-12-14T18:29:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.385+0000 7fca267fc700 1 -- 172.21.15.31:0/2570252730 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340038a0 con 0x560db95fe370 2023-12-14T18:29:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 -- 172.21.15.31:0/2570252730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 msgr2=0x560db95fe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db95fe7e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca34011cf0 tx=0x7fca34014000 comp rx=0 tx=0).stop 2023-12-14T18:29:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 -- 172.21.15.31:0/612003349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 msgr2=0x55ff611ea7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 0x55ff611ea7d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80cb7a0 tx=0x7fc1e809b0a0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 -- 172.21.15.31:0/2570252730 shutdown_connections 2023-12-14T18:29:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db95fde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2570252730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db95fe7e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 -- 172.21.15.31:0/2570252730 >> 172.21.15.31:0/2570252730 conn(0x560db95f4890 msgr2=0x560db95fa200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 -- 172.21.15.31:0/612003349 shutdown_connections 2023-12-14T18:29:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611f5bd0 0x55ff611f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ea400 0x55ff611ea7d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 --2- 172.21.15.31:0/612003349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 -- 172.21.15.31:0/612003349 >> 172.21.15.31:0/612003349 conn(0x55ff611e0d20 msgr2=0x55ff611e6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 -- 172.21.15.31:0/2570252730 shutdown_connections 2023-12-14T18:29:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fca5253cb80 1 -- 172.21.15.31:0/2570252730 wait complete. 2023-12-14T18:29:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.386+0000 7fc200524b80 1 -- 172.21.15.31:0/612003349 shutdown_connections 2023-12-14T18:29:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.387+0000 7fc200524b80 1 -- 172.21.15.31:0/612003349 wait complete. 2023-12-14T18:29:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.387+0000 7fca5253cb80 1 -- start start 2023-12-14T18:29:35.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.387+0000 7fc200524b80 1 -- start start 2023-12-14T18:29:35.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.387+0000 7fc200524b80 1 Processor -- start 2023-12-14T18:29:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.387+0000 7fca5253cb80 1 Processor -- start 2023-12-14T18:29:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db960ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db95fe370 0x560db960a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611fac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff611fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff611f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff610d9010 con 0x55ff611ea400 2023-12-14T18:29:35.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff610d8cb0 con 0x55ff611ead10 2023-12-14T18:29:35.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff60fc0720 con 0x55ff611f5bd0 2023-12-14T18:29:35.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc200524b80 10 client.0 _ll_register_callbacks cb 0x55ff610fc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db960d500 0x560db960a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db94ecf30 con 0x560db945b430 2023-12-14T18:29:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db94ecbd0 con 0x560db960d500 2023-12-14T18:29:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db93d4640 con 0x560db95fe370 2023-12-14T18:29:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca5253cb80 10 client.0 _ll_register_callbacks cb 0x560db95108b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611fac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37980/0 (socket says 172.21.15.31:37980) 2023-12-14T18:29:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db960ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db960ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37966/0 (socket says 172.21.15.31:37966) 2023-12-14T18:29:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 -- 172.21.15.31:0/1424003065 learned_addr learned my addr 172.21.15.31:0/1424003065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f2132700 1 -- 172.21.15.31:0/417414403 learned_addr learned my addr 172.21.15.31:0/417414403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f1931700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff611fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca4494b700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db960d500 0x560db960a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db95fe370 0x560db960a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f2933700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff611f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 -- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db95fe370 msgr2=0x560db960a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db95fe370 0x560db960a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 -- 172.21.15.31:0/1424003065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db960d500 msgr2=0x560db960a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db960d500 0x560db960a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f2132700 1 -- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 msgr2=0x55ff611f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fca3ffff700 1 -- 172.21.15.31:0/1424003065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca34017070 con 0x560db945b430 2023-12-14T18:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.388+0000 7fc1f2132700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff611f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1f2132700 1 -- 172.21.15.31:0/417414403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 msgr2=0x55ff611fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1f2132700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff611fb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db95fe370 0x560db960a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:35.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1f2132700 1 -- 172.21.15.31:0/417414403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec110150 con 0x55ff611ea400 2023-12-14T18:29:35.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1f1931700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff611fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca3ffff700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db960ce70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fca300cac80 tx=0x7fca300a2450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5b7ff938f32f9b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:35.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1f2132700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611fac50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc1e809aeb0 tx=0x7fc1e809ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c81f270bcc575a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca1f7fe700 1 -- 172.21.15.31:0/1424003065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300b1070 con 0x560db945b430 2023-12-14T18:29:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560db95f1ea0 con 0x560db945b430 2023-12-14T18:29:35.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/417414403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e809b1b0 con 0x55ff611ea400 2023-12-14T18:29:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca1f7fe700 1 -- 172.21.15.31:0/1424003065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fca30099700 con 0x560db945b430 2023-12-14T18:29:35.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/417414403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc1e809f1e0 con 0x55ff611ea400 2023-12-14T18:29:35.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff611de330 con 0x55ff611ea400 2023-12-14T18:29:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca1f7fe700 1 -- 172.21.15.31:0/1424003065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca300a1fc0 con 0x560db945b430 2023-12-14T18:29:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca5253cb80 10 client.16062 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560db9517110 con 0x560db945b430 2023-12-14T18:29:35.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/417414403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e80a3570 con 0x55ff611ea400 2023-12-14T18:29:35.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc200524b80 10 client.16065 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:35.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.389+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff611031f0 con 0x55ff611ea400 2023-12-14T18:29:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca1f7fe700 1 -- 172.21.15.31:0/1424003065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca300c8080 con 0x560db945b430 2023-12-14T18:29:35.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/417414403 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ff611de330 con 0x55ff611ea400 2023-12-14T18:29:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca1f7fe700 1 -- 172.21.15.31:0/1424003065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca30098ff0 con 0x560db945b430 2023-12-14T18:29:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca5253cb80 10 client.16062 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca5253cb80 10 client.16062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca5253cb80 20 client.16062 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca5253cb80 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db961ce80 0x560db961f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca5253cb80 4 client.16062 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560db9549c80 con 0x560db961ce80 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db961ce80 0x560db961f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fca1f7fe700 10 client.16062 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db961ce80 0x560db961f260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2304a8cd81ba9b1f server_cookie=43eee0a7e003f332 in_seq=0 out_seq=0 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/417414403 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1e809a6a0 con 0x55ff611ea400 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc200524b80 10 client.16065 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc200524b80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc200524b80 20 client.16065 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc200524b80 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff61204c20 0x55ff61207000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc200524b80 4 client.16065 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff61135d60 con 0x55ff61204c20 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.390+0000 7fc1f1931700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff61204c20 0x55ff61207000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fc1d5ffb700 10 client.16065 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fc1f1931700 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff61204c20 0x55ff61207000 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cde2b3f885b64b6 server_cookie=126538ac9d42fbe2 in_seq=0 out_seq=0 2023-12-14T18:29:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fca1f7fe700 1 -- 172.21.15.31:0/1424003065 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560db9549c80 con 0x560db961ce80 2023-12-14T18:29:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fca1f7fe700 10 client.16062 handle_command_reply: tid=0 2023-12-14T18:29:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.391+0000 7fca5253cb80 1 client.16062 shutdown 2023-12-14T18:29:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 msgr2=0x560db960ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db960ce70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fca300cac80 tx=0x7fca300a2450 comp rx=0 tx=0).stop 2023-12-14T18:29:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/417414403 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff61135d60 con 0x55ff61204c20 2023-12-14T18:29:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc1d5ffb700 10 client.16065 handle_command_reply: tid=0 2023-12-14T18:29:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 shutdown_connections 2023-12-14T18:29:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db961ce80 0x560db961f260 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db960d500 0x560db960a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db960ce70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 --2- 172.21.15.31:0/1424003065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db95fe370 0x560db960a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 >> 172.21.15.31:0/1424003065 conn(0x560db95f4890 msgr2=0x560db95f9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 client.16065 shutdown 2023-12-14T18:29:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca1f7fe700 0 client.16062 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 shutdown_connections 2023-12-14T18:29:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 1 -- 172.21.15.31:0/1424003065 wait complete. 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 msgr2=0x55ff611fac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611fac50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc1e809aeb0 tx=0x7fc1e809ab10 comp rx=0 tx=0).stop 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 shutdown_connections 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff61204c20 0x55ff61207000 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff611f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611fac50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 --2- 172.21.15.31:0/417414403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff611fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 >> 172.21.15.31:0/417414403 conn(0x55ff611e0d20 msgr2=0x55ff611e6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc1d5ffb700 0 client.16065 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fca5253cb80 20 client.16062 trim_cache size 0 max 16384 2023-12-14T18:29:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 shutdown_connections 2023-12-14T18:29:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 1 -- 172.21.15.31:0/417414403 wait complete. 2023-12-14T18:29:35.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.392+0000 7fc200524b80 20 client.16065 trim_cache size 0 max 16384 2023-12-14T18:29:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.408+0000 7fca5253cb80 1 Processor -- start 2023-12-14T18:29:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.408+0000 7fca5253cb80 1 -- start start 2023-12-14T18:29:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db96140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db961ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca5253cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db95fe770 con 0x560db960d500 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca5253cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db95f9b10 con 0x560db95fe370 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca5253cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db960f8e0 con 0x560db945b430 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37986/0 (socket says 172.21.15.31:37986) 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 -- 172.21.15.31:0/2105467111 learned_addr learned my addr 172.21.15.31:0/2105467111 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db961ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca3ffff700 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db96140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 -- 172.21.15.31:0/2105467111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 msgr2=0x560db96140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db96140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 -- 172.21.15.31:0/2105467111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 msgr2=0x560db961ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db961ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.409+0000 7fca4494b700 1 -- 172.21.15.31:0/2105467111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db95f60d0 con 0x560db960d500 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.410+0000 7fca4494b700 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960c930 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fca38007b00 tx=0x7fca38007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b7d1687f4c6455 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.410+0000 7fca26ffd700 1 -- 172.21.15.31:0/2105467111 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38021070 con 0x560db960d500 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.410+0000 7fca26ffd700 1 -- 172.21.15.31:0/2105467111 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca38007c50 con 0x560db960d500 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.410+0000 7fca26ffd700 1 -- 172.21.15.31:0/2105467111 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3800e040 con 0x560db960d500 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 -- 172.21.15.31:0/2105467111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 msgr2=0x560db960c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960c930 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fca38007b00 tx=0x7fca38007b30 comp rx=0 tx=0).stop 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 -- 172.21.15.31:0/2105467111 shutdown_connections 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db960d500 0x560db960c930 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db945b430 0x560db96140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2105467111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db961ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 -- 172.21.15.31:0/2105467111 >> 172.21.15.31:0/2105467111 conn(0x560db95f4890 msgr2=0x560db95f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 -- 172.21.15.31:0/2105467111 shutdown_connections 2023-12-14T18:29:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 -- 172.21.15.31:0/2105467111 wait complete. 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 -- start start 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.411+0000 7fca5253cb80 1 Processor -- start 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db9611ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db96123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db960d500 0x560db966fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db9670220 con 0x560db945b430 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db9670360 con 0x560db95fe370 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db96704a0 con 0x560db960d500 2023-12-14T18:29:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca5253cb80 10 client.0 _ll_register_callbacks cb 0x7fca3009a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:35.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 Processor -- start 2023-12-14T18:29:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db9611ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db9611ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38000/0 (socket says 172.21.15.31:38000) 2023-12-14T18:29:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 -- 172.21.15.31:0/2624541922 learned_addr learned my addr 172.21.15.31:0/2624541922 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 -- start start 2023-12-14T18:29:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db96123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca4494b700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db960d500 0x560db966fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611f5bd0 0x55ff611f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff611f6520 con 0x55ff611ea400 2023-12-14T18:29:35.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff611eb600 con 0x55ff611f5bd0 2023-12-14T18:29:35.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fc200524b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff611f30d0 con 0x55ff611ead10 2023-12-14T18:29:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 -- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db960d500 msgr2=0x560db966fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db960d500 0x560db966fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 -- 172.21.15.31:0/2624541922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 msgr2=0x560db96123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db96123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.412+0000 7fca3ffff700 1 -- 172.21.15.31:0/2624541922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca40068060 con 0x560db945b430 2023-12-14T18:29:35.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f1931700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f1931700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46918/0 (socket says 172.21.15.31:46918) 2023-12-14T18:29:35.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f1931700 1 -- 172.21.15.31:0/4057000204 learned_addr learned my addr 172.21.15.31:0/4057000204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2933700 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611f5bd0 0x55ff611f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca3ffff700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db9611ea0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fca30099cb0 tx=0x7fca300cd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df00cd05df3bc50 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca1f7fe700 1 -- 172.21.15.31:0/2624541922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300b1070 con 0x560db945b430 2023-12-14T18:29:35.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 -- 172.21.15.31:0/4057000204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 msgr2=0x55ff611f4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 -- 172.21.15.31:0/4057000204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611f5bd0 msgr2=0x55ff611f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611f5bd0 0x55ff611f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 -- 172.21.15.31:0/4057000204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff611f57d0 con 0x55ff611ea400 2023-12-14T18:29:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f1931700 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:35.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2933700 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611f5bd0 0x55ff611f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca1f7fe700 1 -- 172.21.15.31:0/2624541922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fca30095ff0 con 0x560db945b430 2023-12-14T18:29:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca34004e50 con 0x560db945b430 2023-12-14T18:29:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca1f7fe700 1 -- 172.21.15.31:0/2624541922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca300a2290 con 0x560db945b430 2023-12-14T18:29:35.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1f2132700 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611f4410 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8099500 tx=0x7fc1e80cb880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcd1f8f0873ab62 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca5253cb80 10 client.16071 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560db937eb00 con 0x560db945b430 2023-12-14T18:29:35.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.413+0000 7fc1e15ad700 1 -- 172.21.15.31:0/4057000204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e809f1e0 con 0x55ff611ea400 2023-12-14T18:29:35.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fc1e15ad700 1 -- 172.21.15.31:0/4057000204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e80a3570 con 0x55ff611ea400 2023-12-14T18:29:35.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fc1e15ad700 1 -- 172.21.15.31:0/4057000204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e809b1b0 con 0x55ff611ea400 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca1f7fe700 1 -- 172.21.15.31:0/2624541922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca34004e50 con 0x560db945b430 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca1f7fe700 1 -- 172.21.15.31:0/2624541922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca300ca190 con 0x560db945b430 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca5253cb80 10 client.16071 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca5253cb80 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca5253cb80 20 client.16071 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db96718e0 0x560db9673cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca5253cb80 4 client.16071 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560db95499a0 con 0x560db96718e0 2023-12-14T18:29:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.414+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db96718e0 0x560db9673cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 -- 172.21.15.31:0/4057000204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 msgr2=0x55ff611f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611f4410 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8099500 tx=0x7fc1e80cb880 comp rx=0 tx=0).stop 2023-12-14T18:29:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fca1f7fe700 10 client.16071 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fca3f7fe700 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db96718e0 0x560db9673cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80340b75b11657c server_cookie=30ed756d27bb6dfe in_seq=0 out_seq=0 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 -- 172.21.15.31:0/4057000204 shutdown_connections 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611f5bd0 0x55ff611f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff611f4410 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 --2- 172.21.15.31:0/4057000204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611ead10 0x55ff611f4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 -- 172.21.15.31:0/4057000204 >> 172.21.15.31:0/4057000204 conn(0x55ff611e0d20 msgr2=0x55ff611efea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 -- 172.21.15.31:0/4057000204 shutdown_connections 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 -- 172.21.15.31:0/4057000204 wait complete. 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 -- start start 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 Processor -- start 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.415+0000 7fc200524b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff6124cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff6124d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff61251b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff612520b0 con 0x55ff611ea400 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff612521f0 con 0x55ff611ead10 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff612523b0 con 0x55ff611f5bd0 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 10 client.0 _ll_register_callbacks cb 0x7fc1ec074c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff6124cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff6124cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38016/0 (socket says 172.21.15.31:38016) 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 -- 172.21.15.31:0/4245740500 learned_addr learned my addr 172.21.15.31:0/4245740500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2933700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff61251b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f1931700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff6124d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 -- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 msgr2=0x55ff61251b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff61251b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 -- 172.21.15.31:0/4245740500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 msgr2=0x55ff6124d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:35.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff6124d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 -- 172.21.15.31:0/4245740500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e4009190 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f1931700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff6124d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2933700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff61251b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1f2132700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff6124cc30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80cb800 tx=0x7fc1e8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef3c00c8dbe474a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/4245740500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80a32e0 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/4245740500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc1e809b680 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.416+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1e8021e90 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/4245740500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e809ab10 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 10 client.16077 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff611031f0 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/4245740500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e80be020 con 0x55ff611ea400 2023-12-14T18:29:35.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/4245740500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1e8096fa0 con 0x55ff611ea400 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 20 client.16077 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff612596e0 0x55ff6125bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 4 client.16077 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ff61135a80 con 0x55ff612596e0 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.417+0000 7fc1f1931700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff612596e0 0x55ff6125bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.418+0000 7fc1d5ffb700 10 client.16077 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.418+0000 7fc1f1931700 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff612596e0 0x55ff6125bac0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404624c5f918fa1d server_cookie=cb7296ae723e3b1e in_seq=0 out_seq=0 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca1f7fe700 1 -- 172.21.15.31:0/2624541922 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560db95499a0 con 0x560db96718e0 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca1f7fe700 10 client.16071 handle_command_reply: tid=0 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 client.16071 shutdown 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 msgr2=0x560db9611ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db9611ea0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fca30099cb0 tx=0x7fca300cd2c0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 shutdown_connections 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560db96718e0 0x560db9673cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560db960d500 0x560db966fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fc1d5ffb700 1 -- 172.21.15.31:0/4245740500 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ff61135a80 con 0x55ff612596e0 2023-12-14T18:29:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fc1d5ffb700 10 client.16077 handle_command_reply: tid=0 2023-12-14T18:29:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560db945b430 0x560db9611ea0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 --2- 172.21.15.31:0/2624541922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560db95fe370 0x560db96123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 >> 172.21.15.31:0/2624541922 conn(0x560db95f4890 msgr2=0x560db96191e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fca1f7fe700 0 client.16071 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fc200524b80 1 client.16077 shutdown 2023-12-14T18:29:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 shutdown_connections 2023-12-14T18:29:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.435+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 msgr2=0x55ff6124cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff6124cc30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80cb800 tx=0x7fc1e8009290 comp rx=0 tx=0).stop 2023-12-14T18:29:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fca5253cb80 1 -- 172.21.15.31:0/2624541922 wait complete. 2023-12-14T18:29:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 shutdown_connections 2023-12-14T18:29:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ff612596e0 0x55ff6125bac0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ff611f5bd0 0x55ff61251b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ff611ea400 0x55ff6124cc30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc1d5ffb700 0 client.16077 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:35.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 --2- 172.21.15.31:0/4245740500 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ff611ead10 0x55ff6124d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 >> 172.21.15.31:0/4245740500 conn(0x55ff611e0d20 msgr2=0x55ff611f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 shutdown_connections 2023-12-14T18:29:35.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 1 -- 172.21.15.31:0/4245740500 wait complete. 2023-12-14T18:29:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fca5253cb80 20 client.16071 trim_cache size 0 max 16384 2023-12-14T18:29:35.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc200524b80 20 client.16077 trim_cache size 0 max 16384 2023-12-14T18:29:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:29:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:29:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:29:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:29:35.451 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:29:35.451 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:29:35.451 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "154fadd6-5949-4f13-a3d9-01aa6a457965", 2023-12-14T18:29:35.451 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:29:35.451 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:29:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fca280a1340 msgr2=0x7fca280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fca463ae700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fca280a1340 0x7fca280a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fca40197ad0 tx=0x7fca34009310 comp rx=0 tx=0).stop 2023-12-14T18:29:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc1d80a1340 msgr2=0x7fc1d80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc1f4396700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc1d80a1340 0x7fc1d80a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec1979f0 tx=0x7fc1dc009260 comp rx=0 tx=0).stop 2023-12-14T18:29:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 msgr2=0x7fca401964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 0x7fca401964c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca3000e9a0 tx=0x7fca3000ed60 comp rx=0 tx=0).stop 2023-12-14T18:29:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.436+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 msgr2=0x7fc1ec1963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:35.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1963e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8000c00 tx=0x7fc1e8005740 comp rx=0 tx=0).stop 2023-12-14T18:29:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 shutdown_connections 2023-12-14T18:29:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca40104400 0x7fca4019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fca280a1340 0x7fca280a37f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca400ff3c0 0x7fca401964c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 --2- 172.21.15.31:0/4048223929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca400ffd60 0x7fca40196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 >> 172.21.15.31:0/4048223929 conn(0x7fca400763a0 msgr2=0x7fca400fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 shutdown_connections 2023-12-14T18:29:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc1ec10a810 0x7fc1ec19ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc1d80a1340 0x7fc1d80a37f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1ec1012f0 0x7fc1ec1963e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 --2- 172.21.15.31:0/3386225786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1ec101c00 0x7fc1ec196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:35.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 >> 172.21.15.31:0/3386225786 conn(0x7fc1ec0763a0 msgr2=0x7fc1ec1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 shutdown_connections 2023-12-14T18:29:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 shutdown_connections 2023-12-14T18:29:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fca463ae700 1 -- 172.21.15.31:0/4048223929 wait complete. 2023-12-14T18:29:35.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:35.437+0000 7fc1f4396700 1 -- 172.21.15.31:0/3386225786 wait complete. 2023-12-14T18:29:35.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:29:35.746 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:35.747 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:35.811 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:35.811 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:35.825 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:35.825 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:35.826 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 57 -lt 100 ']' 2023-12-14T18:29:35.826 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 57 + 1 2023-12-14T18:29:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:35 smithi204 ceph-mon[43866]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-12-14T18:29:35.875 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=58 2023-12-14T18:29:35.875 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:35.888 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:35.888 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:35.890 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:35.890 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:35.890 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:35.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:35 smithi031 ceph-mon[42085]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-12-14T18:29:35.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:35 smithi176 ceph-mon[43506]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 16 op/s 2023-12-14T18:29:36.580 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:36.580 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:36.659 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:36.659 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:36.675 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:36.675 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:36.675 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 58 -lt 100 ']' 2023-12-14T18:29:36.675 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 58 + 1 2023-12-14T18:29:36.708 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=59 2023-12-14T18:29:36.708 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:36.768 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:36.768 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:36.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:36.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:36.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:36 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:29:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:36 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:36 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:29:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:36 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:36 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:29:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:36 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:36 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:29:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:36 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:36 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:29:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:36 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:36 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:29:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:36 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:37.338 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:37.338 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:37.559 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:37.559 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:37.634 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:37.635 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:37.635 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 59 -lt 100 ']' 2023-12-14T18:29:37.635 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 59 + 1 2023-12-14T18:29:37.683 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=60 2023-12-14T18:29:37.683 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:37.725 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:37.725 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:37.727 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:37.727 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:37.727 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:37 smithi204 ceph-mon[43866]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-12-14T18:29:37.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:37 smithi031 ceph-mon[42085]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-12-14T18:29:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:37 smithi176 ceph-mon[43506]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 7 op/s 2023-12-14T18:29:38.304 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:38.305 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:38.451 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:38.451 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:38.467 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:38.467 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:38.467 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 60 -lt 100 ']' 2023-12-14T18:29:38.467 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 60 + 1 2023-12-14T18:29:38.491 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=61 2023-12-14T18:29:38.492 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:38.532 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:38.532 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:38.533 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:38.533 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:38.534 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 -- 172.21.15.31:0/839759081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff98068a60 msgr2=0x7eff98104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 --2- 172.21.15.31:0/839759081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff98068a60 0x7eff98104df0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7eff800099c0 tx=0x7eff80009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 -- 172.21.15.31:0/839759081 shutdown_connections 2023-12-14T18:29:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 --2- 172.21.15.31:0/839759081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff981054d0 0x7eff9810d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:38.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 --2- 172.21.15.31:0/839759081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff980680c0 0x7eff98068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 --2- 172.21.15.31:0/839759081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff98068a60 0x7eff98104df0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 -- 172.21.15.31:0/839759081 >> 172.21.15.31:0/839759081 conn(0x7eff98075fd0 msgr2=0x7eff980763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 -- 172.21.15.31:0/839759081 shutdown_connections 2023-12-14T18:29:38.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.943+0000 7eff9ede8700 1 -- 172.21.15.31:0/839759081 wait complete. 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.944+0000 7eff9ede8700 1 Processor -- start 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.944+0000 7eff9ede8700 1 -- start start 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9ede8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff980680c0 0x7eff981963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9ede8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff98068a60 0x7eff981968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9ede8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 0x7eff9819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9ede8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9819b390 con 0x7eff981054d0 2023-12-14T18:29:38.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9ede8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9819b500 con 0x7eff980680c0 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9ede8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9819b6c0 con 0x7eff98068a60 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 0x7eff9819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9cb84700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff980680c0 0x7eff981963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 0x7eff9819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38028/0 (socket says 172.21.15.31:38028) 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 -- 172.21.15.31:0/3899212762 learned_addr learned my addr 172.21.15.31:0/3899212762 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff97fff700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff98068a60 0x7eff981968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 -- 172.21.15.31:0/3899212762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff98068a60 msgr2=0x7eff981968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff98068a60 0x7eff981968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 -- 172.21.15.31:0/3899212762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff980680c0 msgr2=0x7eff981963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:38.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff980680c0 0x7eff981963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff9d385700 1 -- 172.21.15.31:0/3899212762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800095f0 con 0x7eff981054d0 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.945+0000 7eff97fff700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff98068a60 0x7eff981968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.946+0000 7eff9d385700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 0x7eff9819ad50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7eff9000ef30 tx=0x7eff9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.946+0000 7eff95ffb700 1 -- 172.21.15.31:0/3899212762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90018070 con 0x7eff981054d0 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.946+0000 7eff95ffb700 1 -- 172.21.15.31:0/3899212762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff9000f040 con 0x7eff981054d0 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.946+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff981102a0 con 0x7eff981054d0 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.946+0000 7eff95ffb700 1 -- 172.21.15.31:0/3899212762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000cdb0 con 0x7eff981054d0 2023-12-14T18:29:38.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.946+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff98110110 con 0x7eff981054d0 2023-12-14T18:29:38.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.949+0000 7eff95ffb700 1 -- 172.21.15.31:0/3899212762 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7eff90003a00 con 0x7eff981054d0 2023-12-14T18:29:38.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.949+0000 7eff95ffb700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff840a1340 0x7eff840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:38.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.950+0000 7eff9cb84700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff840a1340 0x7eff840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:38.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.950+0000 7eff95ffb700 1 -- 172.21.15.31:0/3899212762 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7eff900ccc40 con 0x7eff981054d0 2023-12-14T18:29:38.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:38.950+0000 7eff9cb84700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff840a1340 0x7eff840a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff88007be0 tx=0x7eff880073d0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 Processor -- start 2023-12-14T18:29:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 -- start start 2023-12-14T18:29:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187288750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187293610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187295f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591871725e0 con 0x559187288c90 2023-12-14T18:29:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591871e1870 con 0x559187293b50 2023-12-14T18:29:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7effaaf76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591871dad00 con 0x559187288380 2023-12-14T18:29:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7eff97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187293610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7eff97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187293610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38030/0 (socket says 172.21.15.31:38030) 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7eff97fff700 1 -- 172.21.15.31:0/276110019 learned_addr learned my addr 172.21.15.31:0/276110019 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7eff9cb84700 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187288750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.007+0000 7eff9d385700 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187295f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff97fff700 1 -- 172.21.15.31:0/276110019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 msgr2=0x559187288750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff97fff700 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187288750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff97fff700 1 -- 172.21.15.31:0/276110019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 msgr2=0x559187295f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff97fff700 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187295f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff97fff700 1 -- 172.21.15.31:0/276110019 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918705c3b0 con 0x559187288c90 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff97fff700 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187293610 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7eff80004800 tx=0x7eff80005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb320e92d9f0d069 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff8effd700 1 -- 172.21.15.31:0/276110019 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8001d040 con 0x559187288c90 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff8effd700 1 -- 172.21.15.31:0/276110019 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff80004830 con 0x559187288c90 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.008+0000 7eff8effd700 1 -- 172.21.15.31:0/276110019 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff80005220 con 0x559187288c90 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 -- 172.21.15.31:0/276110019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 msgr2=0x559187293610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187293610 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7eff80004800 tx=0x7eff80005c00 comp rx=0 tx=0).stop 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 -- 172.21.15.31:0/276110019 shutdown_connections 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187295f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187288750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 --2- 172.21.15.31:0/276110019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187293610 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 -- 172.21.15.31:0/276110019 >> 172.21.15.31:0/276110019 conn(0x55918727eca0 msgr2=0x5591872847e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 -- 172.21.15.31:0/276110019 shutdown_connections 2023-12-14T18:29:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.009+0000 7effaaf76b80 1 -- 172.21.15.31:0/276110019 wait complete. 2023-12-14T18:29:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.010+0000 7effaaf76b80 1 -- start start 2023-12-14T18:29:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.010+0000 7effaaf76b80 1 Processor -- start 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187288380 0x559187293540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x5591872907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187293b50 0x559187290e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559187176f90 con 0x559187288c90 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559187176c30 con 0x559187288380 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918705e6a0 con 0x559187293b50 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7effaaf76b80 10 client.0 _ll_register_callbacks cb 0x55918719a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x5591872907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x5591872907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38046/0 (socket says 172.21.15.31:38046) 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 -- 172.21.15.31:0/2768744737 learned_addr learned my addr 172.21.15.31:0/2768744737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff9cb84700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187288380 0x559187293540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff9d385700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187293b50 0x559187290e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 -- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187293b50 msgr2=0x559187290e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187293b50 0x559187290e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 -- 172.21.15.31:0/2768744737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187288380 msgr2=0x559187293540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187288380 0x559187293540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.011+0000 7eff97fff700 1 -- 172.21.15.31:0/2768744737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9810d8e0 con 0x559187288c90 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7eff97fff700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x5591872907f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7eff80009d10 tx=0x7eff8000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4396ab96a35ce8ee server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7eff77fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7eff77fff700 1 -- 172.21.15.31:0/2768744737 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8001d040 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7eff77fff700 1 -- 172.21.15.31:0/2768744737 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7eff8000f9f0 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55918727c2b0 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7eff77fff700 1 -- 172.21.15.31:0/2768744737 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff8000fbb0 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7effaaf76b80 10 client.16089 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.012+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5591871a1170 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7eff77fff700 1 -- 172.21.15.31:0/2768744737 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55918727c2b0 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7eff77fff700 1 -- 172.21.15.31:0/2768744737 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff800233f0 con 0x559187288c90 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7effaaf76b80 10 client.16089 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7effaaf76b80 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7effaaf76b80 20 client.16089 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872a3010 0x5591872a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7effaaf76b80 4 client.16089 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591871d3ce0 con 0x5591872a3010 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.013+0000 7eff9cb84700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872a3010 0x5591872a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.014+0000 7eff9cb84700 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872a3010 0x5591872a5420 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d92c0f6652bb3ce server_cookie=dac8bfd9f83773ec in_seq=0 out_seq=0 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.014+0000 7eff77fff700 10 client.16089 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7eff77fff700 1 -- 172.21.15.31:0/2768744737 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5591871d3ce0 con 0x5591872a3010 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7eff77fff700 10 client.16089 handle_command_reply: tid=0 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 client.16089 shutdown 2023-12-14T18:29:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 msgr2=0x5591872907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x5591872907f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7eff80009d10 tx=0x7eff8000bfd0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 shutdown_connections 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872a3010 0x5591872a5420 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187293b50 0x559187290e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7eff77fff700 0 client.16089 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187288380 0x559187293540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2768744737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x5591872907f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 >> 172.21.15.31:0/2768744737 conn(0x55918727eca0 msgr2=0x5591872843d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 shutdown_connections 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 1 -- 172.21.15.31:0/2768744737 wait complete. 2023-12-14T18:29:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.015+0000 7effaaf76b80 20 client.16089 trim_cache size 0 max 16384 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 Processor -- start 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 -- start start 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 0x559187282f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288c90 0x559187293000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187293540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918728c680 con 0x559187288380 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559187283470 con 0x559187293b50 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7effaaf76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559187283630 con 0x559187288c90 2023-12-14T18:29:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 0x559187282f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 0x559187282f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38056/0 (socket says 172.21.15.31:38056) 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 -- 172.21.15.31:0/68386531 learned_addr learned my addr 172.21.15.31:0/68386531 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff97fff700 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288c90 0x559187293000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9d385700 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187293540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 -- 172.21.15.31:0/68386531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288c90 msgr2=0x559187293000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288c90 0x559187293000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 -- 172.21.15.31:0/68386531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 msgr2=0x559187293540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187293540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 -- 172.21.15.31:0/68386531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591871a0ec0 con 0x559187288380 2023-12-14T18:29:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff97fff700 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288c90 0x559187293000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff9cb84700 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 0x559187282f30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x5591872a5cf0 tx=0x7eff88016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304fa040b60608ff server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.032+0000 7eff8f7fe700 1 -- 172.21.15.31:0/68386531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88026040 con 0x559187288380 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7eff8f7fe700 1 -- 172.21.15.31:0/68386531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff880284c0 con 0x559187288380 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7eff8f7fe700 1 -- 172.21.15.31:0/68386531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff88003c00 con 0x559187288380 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 -- 172.21.15.31:0/68386531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 msgr2=0x559187282f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 0x559187282f30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x5591872a5cf0 tx=0x7eff88016000 comp rx=0 tx=0).stop 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 -- 172.21.15.31:0/68386531 shutdown_connections 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x559187293540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288380 0x559187282f30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 --2- 172.21.15.31:0/68386531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288c90 0x559187293000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 -- 172.21.15.31:0/68386531 >> 172.21.15.31:0/68386531 conn(0x55918727eca0 msgr2=0x5591872a9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 -- 172.21.15.31:0/68386531 shutdown_connections 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 -- 172.21.15.31:0/68386531 wait complete. 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 -- start start 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.037+0000 7effaaf76b80 1 Processor -- start 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187298c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187299190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x5591872996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918729ac20 con 0x559187288c90 2023-12-14T18:29:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591872f6ea0 con 0x559187293b50 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591872f7060 con 0x559187288380 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7effaaf76b80 10 client.0 _ll_register_callbacks cb 0x7eff90004cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187299190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187299190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38062/0 (socket says 172.21.15.31:38062) 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 -- 172.21.15.31:0/2173315779 learned_addr learned my addr 172.21.15.31:0/2173315779 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff9cb84700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187298c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff9d385700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x5591872996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 -- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 msgr2=0x559187298c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187298c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 -- 172.21.15.31:0/2173315779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 msgr2=0x5591872996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.038+0000 7eff97fff700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x5591872996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff97fff700 1 -- 172.21.15.31:0/2173315779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591871a1170 con 0x559187288c90 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff9d385700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x5591872996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff97fff700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187299190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7eff800098e0 tx=0x7eff800236a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249b0cb1ec5128a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff77fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff77fff700 1 -- 172.21.15.31:0/2173315779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8001d040 con 0x559187288c90 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff77fff700 1 -- 172.21.15.31:0/2173315779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7eff80000e40 con 0x559187288c90 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff80004830 con 0x559187288c90 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff77fff700 1 -- 172.21.15.31:0/2173315779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff80023a70 con 0x559187288c90 2023-12-14T18:29:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7effaaf76b80 10 client.16095 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7eff88004130 con 0x559187288c90 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.039+0000 7eff77fff700 1 -- 172.21.15.31:0/2173315779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff80004830 con 0x559187288c90 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7eff77fff700 1 -- 172.21.15.31:0/2173315779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff800099f0 con 0x559187288c90 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7effaaf76b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7effaaf76b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7effaaf76b80 20 client.16095 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872f8320 0x5591872fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7effaaf76b80 4 client.16095 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5591871d3a00 con 0x5591872f8320 2023-12-14T18:29:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7eff9cb84700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872f8320 0x5591872fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7eff9cb84700 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872f8320 0x5591872fa730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c9fb84aa19d380 server_cookie=82cdfa5e36aacae3 in_seq=0 out_seq=0 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.040+0000 7eff77fff700 10 client.16095 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7eff77fff700 1 -- 172.21.15.31:0/2173315779 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5591871d3a00 con 0x5591872f8320 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7eff77fff700 10 client.16095 handle_command_reply: tid=0 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 client.16095 shutdown 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 msgr2=0x559187299190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187299190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7eff800098e0 tx=0x7eff800236a0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 shutdown_connections 2023-12-14T18:29:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5591872f8320 0x5591872fa730 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559187293b50 0x5591872996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559187288380 0x559187298c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7eff77fff700 0 client.16095 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 --2- 172.21.15.31:0/2173315779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559187288c90 0x559187299190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 >> 172.21.15.31:0/2173315779 conn(0x55918727eca0 msgr2=0x55918728bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 shutdown_connections 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 1 -- 172.21.15.31:0/2173315779 wait complete. 2023-12-14T18:29:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.041+0000 7effaaf76b80 20 client.16095 trim_cache size 0 max 16384 2023-12-14T18:29:39.055 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:29:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:29:39.055 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:29:39.055 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff840a1340 msgr2=0x7eff840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff840a1340 0x7eff840a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff88007be0 tx=0x7eff880073d0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 msgr2=0x7eff9819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 0x7eff9819ad50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7eff9000ef30 tx=0x7eff9000c3a0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 shutdown_connections 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff981054d0 0x7eff9819ad50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff980680c0 0x7eff981963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff840a1340 0x7eff840a37f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 --2- 172.21.15.31:0/3899212762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff98068a60 0x7eff981968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 >> 172.21.15.31:0/3899212762 conn(0x7eff98075fd0 msgr2=0x7eff980fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 shutdown_connections 2023-12-14T18:29:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:39.042+0000 7eff9ede8700 1 -- 172.21.15.31:0/3899212762 wait complete. 2023-12-14T18:29:39.146 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:39.146 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:39.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:29:39.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:29:39.360 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:39.361 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:39.369 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:39.369 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:39.369 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 61 -lt 100 ']' 2023-12-14T18:29:39.369 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 61 + 1 2023-12-14T18:29:39.416 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=62 2023-12-14T18:29:39.416 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:39.428 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:39.428 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:39.430 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:39.430 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:39.430 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:39.812 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:39 smithi031 ceph-mon[42085]: pgmap v58: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-12-14T18:29:39.813 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:39 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:29:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:39 smithi204 ceph-mon[43866]: pgmap v58: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-12-14T18:29:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:39 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:29:39.947 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:39.947 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:39 smithi176 ceph-mon[43506]: pgmap v58: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 12 op/s 2023-12-14T18:29:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:39 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:29:40.110 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:40.110 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:40.117 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:40.117 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:40.118 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 62 -lt 100 ']' 2023-12-14T18:29:40.118 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 62 + 1 2023-12-14T18:29:40.150 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=63 2023-12-14T18:29:40.150 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:40.241 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:40.241 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:40.243 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:40.243 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:40.243 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:40.854 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:40.854 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:41.059 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:41.059 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:41.085 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:41.085 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:41.085 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 63 -lt 100 ']' 2023-12-14T18:29:41.086 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 63 + 1 2023-12-14T18:29:41.127 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=64 2023-12-14T18:29:41.127 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:41.186 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:41.186 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:41.187 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:41.187 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:41.187 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:41 smithi204 ceph-mon[43866]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:29:41.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:41 smithi031 ceph-mon[42085]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:29:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:41 smithi176 ceph-mon[43506]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:29:42.287 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:42.288 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:42.491 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:42.492 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:42.510 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:42.511 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:42.511 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 64 -lt 100 ']' 2023-12-14T18:29:42.511 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 64 + 1 2023-12-14T18:29:42.549 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=65 2023-12-14T18:29:42.549 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:42.594 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:42.594 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:42.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:42.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:42.596 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:42.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.900+0000 7f7db423c700 1 -- 172.21.15.31:0/3980308281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac10cb60 msgr2=0x7f7dac10ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:42.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.900+0000 7f7db423c700 1 --2- 172.21.15.31:0/3980308281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac10cb60 0x7f7dac10ef40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7da400b0a0 tx=0x7f7da400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 -- 172.21.15.31:0/3980308281 shutdown_connections 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 --2- 172.21.15.31:0/3980308281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac10cb60 0x7f7dac10ef40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 --2- 172.21.15.31:0/3980308281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dac0fefb0 0x7f7dac0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 --2- 172.21.15.31:0/3980308281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac0ff8c0 0x7f7dac10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 -- 172.21.15.31:0/3980308281 >> 172.21.15.31:0/3980308281 conn(0x7f7dac076370 msgr2=0x7f7dac076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 -- 172.21.15.31:0/3980308281 shutdown_connections 2023-12-14T18:29:42.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.901+0000 7f7db423c700 1 -- 172.21.15.31:0/3980308281 wait complete. 2023-12-14T18:29:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 Processor -- start 2023-12-14T18:29:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 -- start start 2023-12-14T18:29:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 0x7f7dac1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dac0ff8c0 0x7f7dac198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:42.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac10cb60 0x7f7dac19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac19c5b0 con 0x7f7dac0fefb0 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac19c720 con 0x7f7dac0ff8c0 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db423c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dac19c8e0 con 0x7f7dac10cb60 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db1fd8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 0x7f7dac1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db1fd8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 0x7f7dac1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38080/0 (socket says 172.21.15.31:38080) 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db1fd8700 1 -- 172.21.15.31:0/3793495558 learned_addr learned my addr 172.21.15.31:0/3793495558 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.902+0000 7f7db27d9700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac10cb60 0x7f7dac19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db17d7700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dac0ff8c0 0x7f7dac198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:42.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db1fd8700 1 -- 172.21.15.31:0/3793495558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac10cb60 msgr2=0x7f7dac19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db1fd8700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac10cb60 0x7f7dac19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db1fd8700 1 -- 172.21.15.31:0/3793495558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dac0ff8c0 msgr2=0x7f7dac198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db1fd8700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dac0ff8c0 0x7f7dac198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db1fd8700 1 -- 172.21.15.31:0/3793495558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da4009d00 con 0x7f7dac0fefb0 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db27d9700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac10cb60 0x7f7dac19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db1fd8700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 0x7f7dac1986c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e9a0 tx=0x7f7da800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8009790 con 0x7f7dac0fefb0 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7da8010430 con 0x7f7dac0fefb0 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dac112490 con 0x7f7dac0fefb0 2023-12-14T18:29:42.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.903+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8018680 con 0x7f7dac0fefb0 2023-12-14T18:29:42.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.904+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dac112300 con 0x7f7dac0fefb0 2023-12-14T18:29:42.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.905+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dac112620 con 0x7f7dac0fefb0 2023-12-14T18:29:42.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.906+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7da80105a0 con 0x7f7dac0fefb0 2023-12-14T18:29:42.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.907+0000 7f7da2ffd700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7d980a12c0 0x7f7d980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:42.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.907+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7da8014070 con 0x7f7dac0fefb0 2023-12-14T18:29:42.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.907+0000 7f7db17d7700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7d980a12c0 0x7f7d980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:42.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.908+0000 7f7db17d7700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7d980a12c0 0x7f7d980a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7dac199cd0 tx=0x7f7d9c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:42.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:42.916+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7da80964e0 con 0x7f7dac0fefb0 2023-12-14T18:29:43.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.178+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dac0679f0 con 0x7f7dac0fefb0 2023-12-14T18:29:43.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.179+0000 7f7da2ffd700 1 -- 172.21.15.31:0/3793495558 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f7da8026090 con 0x7f7dac0fefb0 2023-12-14T18:29:43.181 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:29:43.181 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7d980a12c0 msgr2=0x7f7d980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7d980a12c0 0x7f7d980a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7dac199cd0 tx=0x7f7d9c009260 comp rx=0 tx=0).stop 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 msgr2=0x7f7dac1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 0x7f7dac1986c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7da800e9a0 tx=0x7f7da800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 shutdown_connections 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7d980a12c0 0x7f7d980a3770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dac10cb60 0x7f7dac19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dac0fefb0 0x7f7dac1986c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 --2- 172.21.15.31:0/3793495558 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7dac0ff8c0 0x7f7dac198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 >> 172.21.15.31:0/3793495558 conn(0x7f7dac076370 msgr2=0x7f7dac0fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 shutdown_connections 2023-12-14T18:29:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:43.181+0000 7f7db423c700 1 -- 172.21.15.31:0/3793495558 wait complete. 2023-12-14T18:29:43.185 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:29:43.294 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:43.294 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:43.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:29:43.468 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:43.468 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:43.483 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:43.483 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:43.483 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 65 -lt 100 ']' 2023-12-14T18:29:43.484 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 65 + 1 2023-12-14T18:29:43.524 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=66 2023-12-14T18:29:43.524 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:43.537 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:43.537 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:43.539 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:43.539 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:43.539 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:43 smithi204 ceph-mon[43866]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-12-14T18:29:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:43 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3793495558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:29:43.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:43 smithi031 ceph-mon[42085]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-12-14T18:29:43.847 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:43 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3793495558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:29:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:43 smithi176 ceph-mon[43506]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-12-14T18:29:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:43 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3793495558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:29:43.995 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:43.995 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:44.171 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:44.171 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:44.207 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:44.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:44.208 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 66 -lt 100 ']' 2023-12-14T18:29:44.208 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 66 + 1 2023-12-14T18:29:44.257 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=67 2023-12-14T18:29:44.257 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:44.288 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:44.288 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:44.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:44.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:44.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:44.745 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:44.745 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:44.966 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:44.966 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:44.975 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:44.975 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:44.975 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 67 -lt 100 ']' 2023-12-14T18:29:44.975 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 67 + 1 2023-12-14T18:29:45.015 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=68 2023-12-14T18:29:45.015 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:45.074 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:45.074 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:45.075 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:45.075 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:45.075 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:45.670 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:45.670 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:45 smithi204 ceph-mon[43866]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 17 op/s 2023-12-14T18:29:45.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:45 smithi031 ceph-mon[42085]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 17 op/s 2023-12-14T18:29:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:45 smithi176 ceph-mon[43506]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 17 op/s 2023-12-14T18:29:46.108 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:46.108 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:46.125 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:46.125 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:46.125 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 68 -lt 100 ']' 2023-12-14T18:29:46.125 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 68 + 1 2023-12-14T18:29:46.152 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=69 2023-12-14T18:29:46.152 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:46.216 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:46.216 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:46.218 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:46.218 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:46.218 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 -- 172.21.15.31:0/1545780406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 msgr2=0x7f1898101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 --2- 172.21.15.31:0/1545780406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f1898101710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f189000b0a0 tx=0x7f189000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 -- 172.21.15.31:0/1545780406 shutdown_connections 2023-12-14T18:29:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 --2- 172.21.15.31:0/1545780406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f189810a860 0x7f189810cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 --2- 172.21.15.31:0/1545780406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f1898101710 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 --2- 172.21.15.31:0/1545780406 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1898101c50 0x7f189810a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 -- 172.21.15.31:0/1545780406 >> 172.21.15.31:0/1545780406 conn(0x7f1898075fc0 msgr2=0x7f18980763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 -- 172.21.15.31:0/1545780406 shutdown_connections 2023-12-14T18:29:46.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.592+0000 7f189e6d9700 1 -- 172.21.15.31:0/1545780406 wait complete. 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.593+0000 7f189e6d9700 1 Processor -- start 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.593+0000 7f189e6d9700 1 -- start start 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189e6d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f18981965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189e6d9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1898101c50 0x7f1898196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189e6d9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f189810a860 0x7f189819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189e6d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189819b590 con 0x7f1898101340 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189e6d9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189819b700 con 0x7f189810a860 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189e6d9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189819b8c0 con 0x7f1898101c50 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f18981965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f18981965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42262/0 (socket says 172.21.15.31:42262) 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 -- 172.21.15.31:0/4166918569 learned_addr learned my addr 172.21.15.31:0/4166918569 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f189cc76700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f189810a860 0x7f189819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f18977fe700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1898101c50 0x7f1898196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 -- 172.21.15.31:0/4166918569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1898101c50 msgr2=0x7f1898196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1898101c50 0x7f1898196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 -- 172.21.15.31:0/4166918569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f189810a860 msgr2=0x7f189819af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f189810a860 0x7f189819af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 -- 172.21.15.31:0/4166918569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890009d00 con 0x7f1898101340 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f18977fe700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1898101c50 0x7f1898196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.594+0000 7f1897fff700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f18981965b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f189000b070 tx=0x7f189000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.595+0000 7f18957fa700 1 -- 172.21.15.31:0/4166918569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000e040 con 0x7f1898101340 2023-12-14T18:29:46.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.595+0000 7f18957fa700 1 -- 172.21.15.31:0/4166918569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1890009480 con 0x7f1898101340 2023-12-14T18:29:46.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.595+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1898110320 con 0x7f1898101340 2023-12-14T18:29:46.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.595+0000 7f18957fa700 1 -- 172.21.15.31:0/4166918569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189001b6c0 con 0x7f1898101340 2023-12-14T18:29:46.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.595+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1898110190 con 0x7f1898101340 2023-12-14T18:29:46.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.598+0000 7f18957fa700 1 -- 172.21.15.31:0/4166918569 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f189001d470 con 0x7f1898101340 2023-12-14T18:29:46.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.598+0000 7f18957fa700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18800a1340 0x7f18800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.598+0000 7f18957fa700 1 -- 172.21.15.31:0/4166918569 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f18900ce120 con 0x7f1898101340 2023-12-14T18:29:46.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.598+0000 7f18977fe700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18800a1340 0x7f18800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.599+0000 7f18977fe700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18800a1340 0x7f18800a37f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1898197bc0 tx=0x7f188c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:46.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 Processor -- start 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 -- start start 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d29750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29c90 0x5559a4d34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d36f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4c135e0 con 0x5559a4d34b50 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4c82870 con 0x5559a4d29380 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f18aa867b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4c7bd00 con 0x5559a4d29c90 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f189cc76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d36f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f189cc76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d36f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42276/0 (socket says 172.21.15.31:42276) 2023-12-14T18:29:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.702+0000 7f189cc76700 1 -- 172.21.15.31:0/3202403964 learned_addr learned my addr 172.21.15.31:0/3202403964 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f18977fe700 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29c90 0x5559a4d34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f1897fff700 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d29750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f189cc76700 1 -- 172.21.15.31:0/3202403964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29c90 msgr2=0x5559a4d34610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f189cc76700 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29c90 0x5559a4d34610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f189cc76700 1 -- 172.21.15.31:0/3202403964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 msgr2=0x5559a4d29750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f189cc76700 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d29750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f189cc76700 1 -- 172.21.15.31:0/3202403964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559a4afd3b0 con 0x5559a4d34b50 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f189cc76700 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d36f30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1888009c10 tx=0x7f1888009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bd044e4fd92614 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.703+0000 7f187e7fc700 1 -- 172.21.15.31:0/3202403964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888011cb0 con 0x5559a4d34b50 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f187e7fc700 1 -- 172.21.15.31:0/3202403964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1888011e10 con 0x5559a4d34b50 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f187e7fc700 1 -- 172.21.15.31:0/3202403964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880193e0 con 0x5559a4d34b50 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 -- 172.21.15.31:0/3202403964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 msgr2=0x5559a4d36f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d36f30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1888009c10 tx=0x7f1888009f20 comp rx=0 tx=0).stop 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 -- 172.21.15.31:0/3202403964 shutdown_connections 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d36f30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d29750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3202403964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29c90 0x5559a4d34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.704+0000 7f18aa867b80 1 -- 172.21.15.31:0/3202403964 >> 172.21.15.31:0/3202403964 conn(0x5559a4d1fca0 msgr2=0x5559a4d257e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.705+0000 7f18aa867b80 1 -- 172.21.15.31:0/3202403964 shutdown_connections 2023-12-14T18:29:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.705+0000 7f18aa867b80 1 -- 172.21.15.31:0/3202403964 wait complete. 2023-12-14T18:29:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.705+0000 7f18aa867b80 1 -- start start 2023-12-14T18:29:46.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.705+0000 7f18aa867b80 1 Processor -- start 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29380 0x5559a4d3e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d3e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d38740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4c17f90 con 0x5559a4d34b50 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4c17c30 con 0x5559a4d29c90 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4aff6a0 con 0x5559a4d29380 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18aa867b80 10 client.0 _ll_register_callbacks cb 0x5559a4c3b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f189cc76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d38740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f189cc76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d38740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42290/0 (socket says 172.21.15.31:42290) 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f189cc76700 1 -- 172.21.15.31:0/1514433485 learned_addr learned my addr 172.21.15.31:0/1514433485 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:46.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f1897fff700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29380 0x5559a4d3e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.706+0000 7f18977fe700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d3e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f189cc76700 1 -- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29380 msgr2=0x5559a4d3e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f189cc76700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29380 0x5559a4d3e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f189cc76700 1 -- 172.21.15.31:0/1514433485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 msgr2=0x5559a4d3e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f189cc76700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d3e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f189cc76700 1 -- 172.21.15.31:0/1514433485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18880098c0 con 0x5559a4d34b50 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f1897fff700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29380 0x5559a4d3e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f189cc76700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d38740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1888003b30 tx=0x7f1888003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7edbbc191cfb869 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18777fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18777fe700 1 -- 172.21.15.31:0/1514433485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888026070 con 0x5559a4d34b50 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559a4d1d2b0 con 0x5559a4d34b50 2023-12-14T18:29:46.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18777fe700 1 -- 172.21.15.31:0/1514433485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1888004260 con 0x5559a4d34b50 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18777fe700 1 -- 172.21.15.31:0/1514433485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1888003e50 con 0x5559a4d34b50 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18aa867b80 10 client.16113 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.707+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5559a4c42170 con 0x5559a4d34b50 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18777fe700 1 -- 172.21.15.31:0/1514433485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559a4d1d2b0 con 0x5559a4d34b50 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18777fe700 1 -- 172.21.15.31:0/1514433485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1888029020 con 0x5559a4d34b50 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18aa867b80 10 client.16113 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18aa867b80 10 client.16113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18aa867b80 20 client.16113 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18aa867b80 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d487a0 0x5559a4d4abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18aa867b80 4 client.16113 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559a4c74ce0 con 0x5559a4d487a0 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.708+0000 7f1897fff700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d487a0 0x5559a4d4abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.709+0000 7f1897fff700 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d487a0 0x5559a4d4abb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3cc5032661b79a8 server_cookie=56338dd0cc7a506f in_seq=0 out_seq=0 2023-12-14T18:29:46.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.709+0000 7f18777fe700 10 client.16113 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18777fe700 1 -- 172.21.15.31:0/1514433485 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5559a4c74ce0 con 0x5559a4d487a0 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18777fe700 10 client.16113 handle_command_reply: tid=0 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 client.16113 shutdown 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 msgr2=0x5559a4d38740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d38740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1888003b30 tx=0x7f1888003c30 comp rx=0 tx=0).stop 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 shutdown_connections 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d487a0 0x5559a4d4abb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d34b50 0x5559a4d38740 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d29380 0x5559a4d3e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 --2- 172.21.15.31:0/1514433485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d3e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 >> 172.21.15.31:0/1514433485 conn(0x5559a4d1fca0 msgr2=0x5559a4d25370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18777fe700 0 client.16113 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:46.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 shutdown_connections 2023-12-14T18:29:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 1 -- 172.21.15.31:0/1514433485 wait complete. 2023-12-14T18:29:46.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.710+0000 7f18aa867b80 20 client.16113 trim_cache size 0 max 16384 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.725+0000 7f18aa867b80 1 Processor -- start 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.725+0000 7f18aa867b80 1 -- start start 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d37660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 0x5559a4d30e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18aa867b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4d30a00 con 0x5559a4d29c90 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18aa867b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4d373d0 con 0x5559a4d29380 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18aa867b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4d34100 con 0x5559a4d34b50 2023-12-14T18:29:46.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 0x5559a4d30e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 0x5559a4d30e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42298/0 (socket says 172.21.15.31:42298) 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 -- 172.21.15.31:0/3957254205 learned_addr learned my addr 172.21.15.31:0/3957254205 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f1897fff700 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d37660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f189cc76700 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 -- 172.21.15.31:0/3957254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 msgr2=0x5559a4d313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d313c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 -- 172.21.15.31:0/3957254205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 msgr2=0x5559a4d37660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d37660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 -- 172.21.15.31:0/3957254205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559a4c41ec0 con 0x5559a4d29c90 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f1897fff700 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d37660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.726+0000 7f18977fe700 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 0x5559a4d30e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f188c0041c0 tx=0x7f188c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fe18f6443b18f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.727+0000 7f187effd700 1 -- 172.21.15.31:0/3957254205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c003880 con 0x5559a4d29c90 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.727+0000 7f187effd700 1 -- 172.21.15.31:0/3957254205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f188c012450 con 0x5559a4d29c90 2023-12-14T18:29:46.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.727+0000 7f187effd700 1 -- 172.21.15.31:0/3957254205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c003ea0 con 0x5559a4d29c90 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.727+0000 7f18aa867b80 1 -- 172.21.15.31:0/3957254205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 msgr2=0x5559a4d30e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.727+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 0x5559a4d30e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f188c0041c0 tx=0x7f188c014000 comp rx=0 tx=0).stop 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 -- 172.21.15.31:0/3957254205 shutdown_connections 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29380 0x5559a4d37660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 --2- 172.21.15.31:0/3957254205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29c90 0x5559a4d30e80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 -- 172.21.15.31:0/3957254205 >> 172.21.15.31:0/3957254205 conn(0x5559a4d1fca0 msgr2=0x5559a4d34350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 -- 172.21.15.31:0/3957254205 shutdown_connections 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 -- 172.21.15.31:0/3957254205 wait complete. 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 -- start start 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.728+0000 7f18aa867b80 1 Processor -- start 2023-12-14T18:29:46.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 0x5559a4d95220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d3de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4d3e440 con 0x5559a4d29380 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4d3e5b0 con 0x5559a4d29c90 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559a4d3e770 con 0x5559a4d34b50 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18aa867b80 10 client.0 _ll_register_callbacks cb 0x7f18900074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 0x5559a4d95220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 0x5559a4d95220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42314/0 (socket says 172.21.15.31:42314) 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 -- 172.21.15.31:0/2812901012 learned_addr learned my addr 172.21.15.31:0/2812901012 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f189cc76700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d3de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18977fe700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 -- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 msgr2=0x5559a4d3de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d3de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 -- 172.21.15.31:0/2812901012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 msgr2=0x5559a4d95760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d95760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 -- 172.21.15.31:0/2812901012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1898110190 con 0x5559a4d29380 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18977fe700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d95760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f1897fff700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 0x5559a4d95220 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1890097960 tx=0x7f189000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b9606137e81aac server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.729+0000 7f18777fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18777fe700 1 -- 172.21.15.31:0/2812901012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189009ef20 con 0x5559a4d29380 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1888011e10 con 0x5559a4d29380 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18777fe700 1 -- 172.21.15.31:0/2812901012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f18900b3070 con 0x5559a4d29380 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18777fe700 1 -- 172.21.15.31:0/2812901012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18900a32b0 con 0x5559a4d29380 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 10 client.16119 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:46.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5559a4c289a0 con 0x5559a4d29380 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18777fe700 1 -- 172.21.15.31:0/2812901012 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1890064070 con 0x5559a4d29380 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18777fe700 1 -- 172.21.15.31:0/2812901012 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f189009a3e0 con 0x5559a4d29380 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 10 client.16119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 20 client.16119 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d9ce20 0x5559a4d9f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 4 client.16119 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.730+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559a4c74a00 con 0x5559a4d9ce20 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18977fe700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d9ce20 0x5559a4d9f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18977fe700 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d9ce20 0x5559a4d9f200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=430d0c5a0f18eb64 server_cookie=b7f4585035d582a7 in_seq=0 out_seq=0 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18777fe700 10 client.16119 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18777fe700 1 -- 172.21.15.31:0/2812901012 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559a4c74a00 con 0x5559a4d9ce20 2023-12-14T18:29:46.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18777fe700 10 client.16119 handle_command_reply: tid=0 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18aa867b80 1 client.16119 shutdown 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 msgr2=0x5559a4d95220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.731+0000 7f18aa867b80 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 0x5559a4d95220 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1890097960 tx=0x7f189000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 shutdown_connections 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5559a4d9ce20 0x5559a4d9f200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5559a4d34b50 0x5559a4d3de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5559a4d29380 0x5559a4d95220 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 --2- 172.21.15.31:0/2812901012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5559a4d29c90 0x5559a4d95760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18777fe700 0 client.16119 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 >> 172.21.15.31:0/2812901012 conn(0x5559a4d1fca0 msgr2=0x5559a4d30e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 shutdown_connections 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 1 -- 172.21.15.31:0/2812901012 wait complete. 2023-12-14T18:29:46.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f18aa867b80 20 client.16119 trim_cache size 0 max 16384 2023-12-14T18:29:46.740 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18800a1340 msgr2=0x7f18800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f189e6d9700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18800a1340 0x7f18800a37f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1898197bc0 tx=0x7f188c009260 comp rx=0 tx=0).stop 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 msgr2=0x7f18981965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.732+0000 7f189e6d9700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f18981965b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f189000b070 tx=0x7f189000bce0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 shutdown_connections 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f189810a860 0x7f189819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1898101340 0x7f18981965b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18800a1340 0x7f18800a37f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 --2- 172.21.15.31:0/4166918569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1898101c50 0x7f1898196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 >> 172.21.15.31:0/4166918569 conn(0x7f1898075fc0 msgr2=0x7f18981022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:46.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 shutdown_connections 2023-12-14T18:29:46.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:46.733+0000 7f189e6d9700 1 -- 172.21.15.31:0/4166918569 wait complete. 2023-12-14T18:29:46.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:29:47.003 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:47.003 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:47.126 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:47.126 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:47.150 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:47.150 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:47.150 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 69 -lt 100 ']' 2023-12-14T18:29:47.150 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 69 + 1 2023-12-14T18:29:47.198 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=70 2023-12-14T18:29:47.199 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:47.219 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:47.219 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:47.220 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:47.221 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:47.221 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:47.586 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:47.586 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:47.673 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:47.673 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:47.684 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:47.684 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:47.684 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 70 -lt 100 ']' 2023-12-14T18:29:47.684 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 70 + 1 2023-12-14T18:29:47.715 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=71 2023-12-14T18:29:47.715 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:47.758 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:47.758 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:47.759 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:47.760 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:47.760 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:47 smithi204 ceph-mon[43866]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-12-14T18:29:47.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:47 smithi031 ceph-mon[42085]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-12-14T18:29:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:47 smithi176 ceph-mon[43506]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-12-14T18:29:48.453 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:48.453 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:48.657 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:48.657 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:48.683 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:48.683 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:48.683 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 71 -lt 100 ']' 2023-12-14T18:29:48.683 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 71 + 1 2023-12-14T18:29:48.835 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=72 2023-12-14T18:29:48.835 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:48.871 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:48.871 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:48.872 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:48.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:48.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:49.519 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:49.519 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:49.707 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:49.707 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:49.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:49 smithi031 ceph-mon[42085]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2023-12-14T18:29:49.733 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:49.733 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:49.733 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 72 -lt 100 ']' 2023-12-14T18:29:49.733 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 72 + 1 2023-12-14T18:29:49.773 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=73 2023-12-14T18:29:49.773 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:49.825 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:49.825 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:49.827 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:49.827 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:49.827 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:49 smithi204 ceph-mon[43866]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2023-12-14T18:29:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:49 smithi176 ceph-mon[43506]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2023-12-14T18:29:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.016+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1804784664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70100c50 msgr2=0x7f2a70101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.016+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1804784664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70100c50 0x7f2a70101020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a600099c0 tx=0x7f2a60009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1804784664 shutdown_connections 2023-12-14T18:29:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1804784664 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a70105560 0x7f2a7010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1804784664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70100c50 0x7f2a70101020 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1804784664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a701015f0 0x7f2a70104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1804784664 >> 172.21.15.31:0/1804784664 conn(0x7f2a70077550 msgr2=0x7f2a70077950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 -- 172.21.15.31:0/3511232581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b80680c0 msgr2=0x7f85b8068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 --2- 172.21.15.31:0/3511232581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b80680c0 0x7f85b8068490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f85a80098f0 tx=0x7f85a8009c00 comp rx=0 tx=0).stop 2023-12-14T18:29:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1804784664 shutdown_connections 2023-12-14T18:29:50.021 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1804784664 wait complete. 2023-12-14T18:29:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 -- 172.21.15.31:0/3511232581 shutdown_connections 2023-12-14T18:29:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 --2- 172.21.15.31:0/3511232581 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85b81054d0 0x7f85b810d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 --2- 172.21.15.31:0/3511232581 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b80680c0 0x7f85b8068490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 --2- 172.21.15.31:0/3511232581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b8068a60 0x7f85b8104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 -- 172.21.15.31:0/3511232581 >> 172.21.15.31:0/3511232581 conn(0x7f85b8075fd0 msgr2=0x7f85b80763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 -- 172.21.15.31:0/3511232581 shutdown_connections 2023-12-14T18:29:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.017+0000 7f85be250700 1 -- 172.21.15.31:0/3511232581 wait complete. 2023-12-14T18:29:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 Processor -- start 2023-12-14T18:29:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 -- start start 2023-12-14T18:29:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f2a75c8f700 1 Processor -- start 2023-12-14T18:29:50.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f2a75c8f700 1 -- start start 2023-12-14T18:29:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 0x7f85b81941c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85b8068a60 0x7f85b8194700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b81054d0 0x7f85b8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81991a0 con 0x7f85b80680c0 2023-12-14T18:29:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8199310 con 0x7f85b81054d0 2023-12-14T18:29:50.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.018+0000 7f85be250700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81994d0 con 0x7f85b8068a60 2023-12-14T18:29:50.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 0x7f85b81941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a75c8f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a70100c50 0x7f2a70196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 0x7f85b81941c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42336/0 (socket says 172.21.15.31:42336) 2023-12-14T18:29:50.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 -- 172.21.15.31:0/3486140186 learned_addr learned my addr 172.21.15.31:0/3486140186 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a75c8f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 0x7f2a70196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a75c8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70105560 0x7f2a7019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a75c8f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019b320 con 0x7f2a70105560 2023-12-14T18:29:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a75c8f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019b490 con 0x7f2a701015f0 2023-12-14T18:29:50.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a75c8f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019b650 con 0x7f2a70100c50 2023-12-14T18:29:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85aeffd700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85b8068a60 0x7f85b8194700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b7fff700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b81054d0 0x7f85b8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a70100c50 0x7f2a70196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 0x7f2a70196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 0x7f2a70196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41400/0 (socket says 172.21.15.31:41400) 2023-12-14T18:29:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a6effd700 1 -- 172.21.15.31:0/1649750458 learned_addr learned my addr 172.21.15.31:0/1649750458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70105560 0x7f2a7019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 -- 172.21.15.31:0/3486140186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85b8068a60 msgr2=0x7f85b8194700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85b8068a60 0x7f85b8194700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 -- 172.21.15.31:0/3486140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b81054d0 msgr2=0x7f85b8198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b81054d0 0x7f85b8198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.019+0000 7f85b77fe700 1 -- 172.21.15.31:0/3486140186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8009520 con 0x7f85b80680c0 2023-12-14T18:29:50.029 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6effd700 1 -- 172.21.15.31:0/1649750458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a70100c50 msgr2=0x7f2a70196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 0x7f85b81941c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009230 tx=0x7f85a800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6effd700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a70100c50 0x7f2a70196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6effd700 1 -- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70105560 msgr2=0x7f2a7019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6effd700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70105560 0x7f2a7019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6effd700 1 -- 172.21.15.31:0/1649750458 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a64009520 con 0x7f2a701015f0 2023-12-14T18:29:50.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f85b57fa700 1 -- 172.21.15.31:0/3486140186 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a801c070 con 0x7f85b80680c0 2023-12-14T18:29:50.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6effd700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 0x7f2a70196880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009fd0 tx=0x7f2a6400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f85b57fa700 1 -- 172.21.15.31:0/3486140186 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a80058e0 con 0x7f85b80680c0 2023-12-14T18:29:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b810e110 con 0x7f85b80680c0 2023-12-14T18:29:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f85b57fa700 1 -- 172.21.15.31:0/3486140186 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a80038c0 con 0x7f85b80680c0 2023-12-14T18:29:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b810df80 con 0x7f85b80680c0 2023-12-14T18:29:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6cff9700 1 -- 172.21.15.31:0/1649750458 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400cbd0 con 0x7f2a701015f0 2023-12-14T18:29:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a68009c40 con 0x7f2a701015f0 2023-12-14T18:29:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6cff9700 1 -- 172.21.15.31:0/1649750458 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a6400ee90 con 0x7f2a701015f0 2023-12-14T18:29:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a6cff9700 1 -- 172.21.15.31:0/1649750458 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64003ce0 con 0x7f2a701015f0 2023-12-14T18:29:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.020+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600095f0 con 0x7f2a701015f0 2023-12-14T18:29:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.024+0000 7f2a6cff9700 1 -- 172.21.15.31:0/1649750458 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2a64003f20 con 0x7f2a701015f0 2023-12-14T18:29:50.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.024+0000 7f85b57fa700 1 -- 172.21.15.31:0/3486140186 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f85a8020030 con 0x7f85b80680c0 2023-12-14T18:29:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.024+0000 7f2a6cff9700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2a580a12c0 0x7f2a580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.024+0000 7f85b57fa700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85a40a1340 0x7f85a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.025+0000 7f85aeffd700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85a40a1340 0x7f85a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.025+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2a580a12c0 0x7f2a580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.025+0000 7f2a6cff9700 1 -- 172.21.15.31:0/1649750458 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2a64014070 con 0x7f2a701015f0 2023-12-14T18:29:50.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.025+0000 7f85b57fa700 1 -- 172.21.15.31:0/3486140186 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f85a80ce080 con 0x7f85b80680c0 2023-12-14T18:29:50.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.025+0000 7f85aeffd700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85a40a1340 0x7f85a40a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85b81957d0 tx=0x7f85a0009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.025+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2a580a12c0 0x7f2a580a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009fd0 tx=0x7f2a60005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.103 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 Processor -- start 2023-12-14T18:29:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 Processor -- start 2023-12-14T18:29:50.104 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 -- start start 2023-12-14T18:29:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 -- start start 2023-12-14T18:29:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84381e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f843827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8426c580 con 0x557f8438d0f0 2023-12-14T18:29:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f842db810 con 0x557f84382370 2023-12-14T18:29:50.105 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f842d4ca0 con 0x557f841df430 2023-12-14T18:29:50.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffcd8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffce5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffbc25e0 con 0x55b7ffce3b50 2023-12-14T18:29:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffc31870 con 0x55b7ffcd8c90 2023-12-14T18:29:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffc2ad00 con 0x55b7ffcd8380 2023-12-14T18:29:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.101+0000 7f2a6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84381e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42354/0 (socket says 172.21.15.31:42354) 2023-12-14T18:29:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84381e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:58414/0 (socket says 172.21.15.31:58414) 2023-12-14T18:29:50.107 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3748238693 learned_addr learned my addr 172.21.15.31:0/3748238693 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.108 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6effd700 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f843827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffcd8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffcd8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:58430/0 (socket says 172.21.15.31:58430) 2023-12-14T18:29:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 -- 172.21.15.31:0/3013627343 learned_addr learned my addr 172.21.15.31:0/3013627343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85aeffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b7fff700 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffce5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3748238693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 msgr2=0x557f843827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f843827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3748238693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 msgr2=0x557f8438f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/3748238693 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f84156350 con 0x557f841df430 2023-12-14T18:29:50.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 -- 172.21.15.31:0/3013627343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 msgr2=0x55b7ffce3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.110 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84381e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a600058e0 tx=0x7f2a6001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec19d44af71d82ff server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 -- 172.21.15.31:0/3013627343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 msgr2=0x55b7ffce5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffce5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.102+0000 7f85b77fe700 1 -- 172.21.15.31:0/3013627343 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ffaac3b0 con 0x55b7ffcd8380 2023-12-14T18:29:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffcd8750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f85a809ade0 tx=0x7f85a800faa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be43ef2343051b1e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f2a55ffb700 1 -- 172.21.15.31:0/3748238693 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6000b990 con 0x557f841df430 2023-12-14T18:29:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f2a55ffb700 1 -- 172.21.15.31:0/3748238693 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a6001e040 con 0x557f841df430 2023-12-14T18:29:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f2a55ffb700 1 -- 172.21.15.31:0/3748238693 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a600306e0 con 0x557f841df430 2023-12-14T18:29:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f85adffb700 1 -- 172.21.15.31:0/3013627343 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a80a3210 con 0x55b7ffcd8380 2023-12-14T18:29:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f85adffb700 1 -- 172.21.15.31:0/3013627343 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a800bab0 con 0x55b7ffcd8380 2023-12-14T18:29:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.103+0000 7f85adffb700 1 -- 172.21.15.31:0/3013627343 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8021e90 con 0x55b7ffcd8380 2023-12-14T18:29:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3748238693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 msgr2=0x557f84381e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84381e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a600058e0 tx=0x7f2a6001f000 comp rx=0 tx=0).stop 2023-12-14T18:29:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3748238693 shutdown_connections 2023-12-14T18:29:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84381e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3748238693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f843827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3748238693 >> 172.21.15.31:0/3748238693 conn(0x557f84378890 msgr2=0x557f8437e200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3748238693 shutdown_connections 2023-12-14T18:29:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3748238693 wait complete. 2023-12-14T18:29:50.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3013627343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 msgr2=0x55b7ffcd8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffcd8750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f85a809ade0 tx=0x7f85a800faa0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3013627343 shutdown_connections 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffce5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffcd8750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3013627343 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3013627343 >> 172.21.15.31:0/3013627343 conn(0x55b7ffcceca0 msgr2=0x55b7ffcd47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3013627343 shutdown_connections 2023-12-14T18:29:50.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3013627343 wait complete. 2023-12-14T18:29:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.104+0000 7f2a81e1db80 1 -- start start 2023-12-14T18:29:50.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f85ca3deb80 1 -- start start 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 Processor -- start 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f8438cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f84389e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84270f30 con 0x557f8438d0f0 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84270bd0 con 0x557f84382370 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84158640 con 0x557f841df430 2023-12-14T18:29:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a81e1db80 10 client.0 _ll_register_callbacks cb 0x557f842948b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:50.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f85ca3deb80 1 Processor -- start 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f8438cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42366/0 (socket says 172.21.15.31:42366) 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.105+0000 7f2a6ffff700 1 -- 172.21.15.31:0/2621844259 learned_addr learned my addr 172.21.15.31:0/2621844259 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6effd700 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f84389e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2621844259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 msgr2=0x557f84389e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f84389e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2621844259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 msgr2=0x557f8438a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6f7fe700 1 -- 172.21.15.31:0/2621844259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7010d9b0 con 0x557f841df430 2023-12-14T18:29:50.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8380 0x55b7ffce3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffce3b50 0x55b7ffce49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffbc6f90 con 0x55b7ffcd8380 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffbc6c30 con 0x55b7ffcd8c90 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffaae6a0 con 0x55b7ffce3b50 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85ca3deb80 10 client.0 _ll_register_callbacks cb 0x55b7ffbea910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85aeffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8380 0x55b7ffce3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f8438cb80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a600036c0 tx=0x7f2a600048c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae8fb713803ca662 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a4effd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:29:50.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85aeffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41430/0 (socket says 172.21.15.31:41430) 2023-12-14T18:29:50.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffce3b50 0x55b7ffce49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f85aeffd700 1 -- 172.21.15.31:0/1120093150 learned_addr learned my addr 172.21.15.31:0/1120093150 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a4effd700 1 -- 172.21.15.31:0/2621844259 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60034070 con 0x557f841df430 2023-12-14T18:29:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.106+0000 7f2a4effd700 1 -- 172.21.15.31:0/2621844259 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2a60004290 con 0x557f841df430 2023-12-14T18:29:50.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85aeffd700 1 -- 172.21.15.31:0/1120093150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffce3b50 msgr2=0x55b7ffce49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85aeffd700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffce3b50 0x55b7ffce49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85aeffd700 1 -- 172.21.15.31:0/1120093150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8380 msgr2=0x55b7ffce3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85aeffd700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8380 0x55b7ffce3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f84375ea0 con 0x557f841df430 2023-12-14T18:29:50.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85aeffd700 1 -- 172.21.15.31:0/1120093150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b810df80 con 0x55b7ffcd8c90 2023-12-14T18:29:50.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85b77fe700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8380 0x55b7ffce3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:50.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85b7fff700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffce3b50 0x55b7ffce49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a4effd700 1 -- 172.21.15.31:0/2621844259 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a600309a0 con 0x557f841df430 2023-12-14T18:29:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a81e1db80 10 client.25993 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557f8429b110 con 0x557f841df430 2023-12-14T18:29:50.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85aeffd700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce4460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fd0 tx=0x7f85a00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c0604cd0e54f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f8596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:29:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a4effd700 1 -- 172.21.15.31:0/2621844259 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f84375ea0 con 0x557f841df430 2023-12-14T18:29:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f8596ffd700 1 -- 172.21.15.31:0/1120093150 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0004a30 con 0x55b7ffcd8c90 2023-12-14T18:29:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f8596ffd700 1 -- 172.21.15.31:0/1120093150 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f85a0004b90 con 0x55b7ffcd8c90 2023-12-14T18:29:50.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7ffccc2b0 con 0x55b7ffcd8c90 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a4effd700 1 -- 172.21.15.31:0/2621844259 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a6000f460 con 0x557f841df430 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a81e1db80 10 client.25993 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.107+0000 7f2a81e1db80 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a81e1db80 20 client.25993 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f8439c660 0x557f8439ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a81e1db80 4 client.25993 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f842cdc80 con 0x557f8439c660 2023-12-14T18:29:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f8596ffd700 1 -- 172.21.15.31:0/1120093150 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85a0004d50 con 0x55b7ffcd8c90 2023-12-14T18:29:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 10 client.25988 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:50.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b7ffbf1170 con 0x55b7ffcd8c90 2023-12-14T18:29:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a6effd700 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f8439c660 0x557f8439ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f8596ffd700 1 -- 172.21.15.31:0/1120093150 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7ffccc2b0 con 0x55b7ffcd8c90 2023-12-14T18:29:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a4effd700 10 client.25993 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f2a6effd700 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f8439c660 0x557f8439ea40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab8032282cd452f server_cookie=335b6d8ca17d4817 in_seq=0 out_seq=0 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f8596ffd700 1 -- 172.21.15.31:0/1120093150 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85a001e990 con 0x55b7ffcd8c90 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 10 client.25988 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 10 client.25988 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 20 client.25988 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffcf3190 0x55b7ffcf55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 4 client.25988 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.108+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7ffc23ce0 con 0x55b7ffcf3190 2023-12-14T18:29:50.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f85b77fe700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffcf3190 0x55b7ffcf55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a4effd700 1 -- 172.21.15.31:0/2621844259 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f842cdc80 con 0x557f8439c660 2023-12-14T18:29:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a4effd700 10 client.25993 handle_command_reply: tid=0 2023-12-14T18:29:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 client.25993 shutdown 2023-12-14T18:29:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f85b77fe700 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffcf3190 0x55b7ffcf55a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f4f45bf9d9dd875 server_cookie=733ebeafebb8ed55 in_seq=0 out_seq=0 2023-12-14T18:29:50.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f8596ffd700 10 client.25988 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 msgr2=0x557f8438cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f8438cb80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a600036c0 tx=0x7f2a600048c0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 shutdown_connections 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f8439c660 0x557f8439ea40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f8438d0f0 0x557f8438a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f8438cb80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/2621844259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f84382370 0x557f84389e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a4effd700 0 client.25993 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 >> 172.21.15.31:0/2621844259 conn(0x557f84378890 msgr2=0x557f8437dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 shutdown_connections 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.109+0000 7f2a81e1db80 1 -- 172.21.15.31:0/2621844259 wait complete. 2023-12-14T18:29:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f2a81e1db80 20 client.25993 trim_cache size 0 max 16384 2023-12-14T18:29:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f8596ffd700 1 -- 172.21.15.31:0/1120093150 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b7ffc23ce0 con 0x55b7ffcf3190 2023-12-14T18:29:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f8596ffd700 10 client.25988 handle_command_reply: tid=0 2023-12-14T18:29:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 client.25988 shutdown 2023-12-14T18:29:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 msgr2=0x55b7ffce4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce4460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009fd0 tx=0x7f85a00047c0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 shutdown_connections 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffcf3190 0x55b7ffcf55a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffce3b50 0x55b7ffce49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f8596ffd700 0 client.25988 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8380 0x55b7ffce3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/1120093150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce4460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 >> 172.21.15.31:0/1120093150 conn(0x55b7ffcceca0 msgr2=0x55b7ffcd43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 shutdown_connections 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.110+0000 7f85ca3deb80 1 -- 172.21.15.31:0/1120093150 wait complete. 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.111+0000 7f85ca3deb80 20 client.25988 trim_cache size 0 max 16384 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.125+0000 7f85ca3deb80 1 Processor -- start 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.125+0000 7f85ca3deb80 1 -- start start 2023-12-14T18:29:50.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8380 0x55b7ffce8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffcdd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffcd5590 con 0x55b7ffce3b50 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffcd95b0 con 0x55b7ffcd8380 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffcdeae0 con 0x55b7ffcd8c90 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffcdd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffcdd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42394/0 (socket says 172.21.15.31:42394) 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85b7fff700 1 -- 172.21.15.31:0/4113988492 learned_addr learned my addr 172.21.15.31:0/4113988492 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85aeffd700 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85b77fe700 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8380 0x55b7ffce8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85aeffd700 1 -- 172.21.15.31:0/4113988492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8380 msgr2=0x55b7ffce8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85aeffd700 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8380 0x55b7ffce8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85aeffd700 1 -- 172.21.15.31:0/4113988492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 msgr2=0x55b7ffcdd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85aeffd700 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffcdd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85aeffd700 1 -- 172.21.15.31:0/4113988492 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ffce3570 con 0x55b7ffcd8c90 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.126+0000 7f85b7fff700 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffcdd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f85aeffd700 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce2530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85a0013010 tx=0x7f85a00046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7408f4a621699d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f85ae7fc700 1 -- 172.21.15.31:0/4113988492 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a000fdb0 con 0x55b7ffcd8c90 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f85ae7fc700 1 -- 172.21.15.31:0/4113988492 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a001a3a0 con 0x55b7ffcd8c90 2023-12-14T18:29:50.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f85ae7fc700 1 -- 172.21.15.31:0/4113988492 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0004d20 con 0x55b7ffcd8c90 2023-12-14T18:29:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 Processor -- start 2023-12-14T18:29:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 -- start start 2023-12-14T18:29:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f8438c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f8438cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8438f970 con 0x557f84382370 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84398ae0 con 0x557f8438d0f0 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.127+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f8437cfb0 con 0x557f841df430 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f8438c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f8438cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f8438cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41452/0 (socket says 172.21.15.31:41452) 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f8438c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42402/0 (socket says 172.21.15.31:42402) 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3769337612 learned_addr learned my addr 172.21.15.31:0/3769337612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:58472/0 (socket says 172.21.15.31:58472) 2023-12-14T18:29:50.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 -- 172.21.15.31:0/4113988492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8c90 msgr2=0x55b7ffce2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce2530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85a0013010 tx=0x7f85a00046e0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3769337612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 msgr2=0x557f84393c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3769337612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 msgr2=0x557f8438c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f8438c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 -- 172.21.15.31:0/3769337612 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f84382a10 con 0x557f8438d0f0 2023-12-14T18:29:50.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 -- 172.21.15.31:0/4113988492 shutdown_connections 2023-12-14T18:29:50.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffce3b50 0x55b7ffcdd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffcd8380 0x55b7ffce8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/4113988492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce2530 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6effd700 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f8438c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:50.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 -- 172.21.15.31:0/4113988492 >> 172.21.15.31:0/4113988492 conn(0x55b7ffcceca0 msgr2=0x55b7ffcd2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 -- 172.21.15.31:0/4113988492 shutdown_connections 2023-12-14T18:29:50.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f85ca3deb80 1 -- 172.21.15.31:0/4113988492 wait complete. 2023-12-14T18:29:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f8438cb80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ea40 tx=0x7f2a68009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841a5ad9ac1de26b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.128+0000 7f2a567fc700 1 -- 172.21.15.31:0/3769337612 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800e040 con 0x557f8438d0f0 2023-12-14T18:29:50.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 -- start start 2023-12-14T18:29:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f2a567fc700 1 -- 172.21.15.31:0/3769337612 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a6800f040 con 0x557f8438d0f0 2023-12-14T18:29:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f2a567fc700 1 -- 172.21.15.31:0/3769337612 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68017c10 con 0x557f8438d0f0 2023-12-14T18:29:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 Processor -- start 2023-12-14T18:29:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffce66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 0x55b7ffd46050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffce7420 con 0x55b7ffcd8c90 2023-12-14T18:29:50.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffce7590 con 0x55b7ffce3b50 2023-12-14T18:29:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ffd46610 con 0x55b7ffcd8380 2023-12-14T18:29:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85ca3deb80 10 client.0 _ll_register_callbacks cb 0x7f85a00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3769337612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 msgr2=0x557f8438cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f8438cb80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ea40 tx=0x7f2a68009920 comp rx=0 tx=0).stop 2023-12-14T18:29:50.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.129+0000 7f85aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3769337612 shutdown_connections 2023-12-14T18:29:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f8438cb80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 0x55b7ffd46050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42410/0 (socket says 172.21.15.31:42410) 2023-12-14T18:29:50.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 0x55b7ffd46050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41460/0 (socket says 172.21.15.31:41460) 2023-12-14T18:29:50.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 -- 172.21.15.31:0/3514974348 learned_addr learned my addr 172.21.15.31:0/3514974348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/3769337612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f8438c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3769337612 >> 172.21.15.31:0/3769337612 conn(0x557f84378890 msgr2=0x557f843983d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffce66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3769337612 shutdown_connections 2023-12-14T18:29:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 -- 172.21.15.31:0/3769337612 wait complete. 2023-12-14T18:29:50.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 -- 172.21.15.31:0/3514974348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 msgr2=0x55b7ffce66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffce66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 -- 172.21.15.31:0/3514974348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8c90 msgr2=0x55b7ffce6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 -- 172.21.15.31:0/3514974348 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b000d1e0 con 0x55b7ffce3b50 2023-12-14T18:29:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f2a81e1db80 1 -- start start 2023-12-14T18:29:50.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85b7fff700 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 0x55b7ffd46050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85b001ba30 tx=0x7f85b001ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c8f9c95bed665d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f8596ffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:29:50.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f8596ffd700 1 -- 172.21.15.31:0/3514974348 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b0020070 con 0x55b7ffce3b50 2023-12-14T18:29:50.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f8596ffd700 1 -- 172.21.15.31:0/3514974348 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f85b001a6d0 con 0x55b7ffce3b50 2023-12-14T18:29:50.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.130+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85a800bab0 con 0x55b7ffce3b50 2023-12-14T18:29:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 Processor -- start 2023-12-14T18:29:50.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f8596ffd700 1 -- 172.21.15.31:0/3514974348 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85b001a890 con 0x55b7ffce3b50 2023-12-14T18:29:50.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 10 client.25997 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:50.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b7ffbf1170 con 0x55b7ffce3b50 2023-12-14T18:29:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f84393e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f84391e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84392510 con 0x557f84382370 2023-12-14T18:29:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84392680 con 0x557f8438d0f0 2023-12-14T18:29:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f84392840 con 0x557f841df430 2023-12-14T18:29:50.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f8596ffd700 1 -- 172.21.15.31:0/3514974348 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85a800bab0 con 0x55b7ffce3b50 2023-12-14T18:29:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a81e1db80 10 client.0 _ll_register_callbacks cb 0x7f2a6800e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:50.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f8596ffd700 1 -- 172.21.15.31:0/3514974348 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f85b0025ac0 con 0x55b7ffce3b50 2023-12-14T18:29:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f84393e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a6effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f84393e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42424/0 (socket says 172.21.15.31:42424) 2023-12-14T18:29:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a6effd700 1 -- 172.21.15.31:0/474731156 learned_addr learned my addr 172.21.15.31:0/474731156 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:50.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 10 client.25997 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:50.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 10 client.25997 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:29:50.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 20 client.25997 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:50.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffd47e20 0x55b7ffd4a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 4 client.25997 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:50.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7ffc23a00 con 0x55b7ffd47e20 2023-12-14T18:29:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.131+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f84391e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffd47e20 0x55b7ffd4a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6effd700 1 -- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 msgr2=0x557f84393900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6effd700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6effd700 1 -- 172.21.15.31:0/474731156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 msgr2=0x557f84391e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6effd700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f84391e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6effd700 1 -- 172.21.15.31:0/474731156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600095f0 con 0x557f84382370 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6ffff700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f84391e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a6effd700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f84393e40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a64099d70 tx=0x7f2a640098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95a0084824aaeb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a4effd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:50.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f8596ffd700 10 client.25997 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:50.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f85b77fe700 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffd47e20 0x55b7ffd4a230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=915894d6d7c76bc8 server_cookie=70fd81a5e2a12821 in_seq=0 out_seq=0 2023-12-14T18:29:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a4effd700 1 -- 172.21.15.31:0/474731156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a640041d0 con 0x557f84382370 2023-12-14T18:29:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a6001e040 con 0x557f84382370 2023-12-14T18:29:50.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f2a4effd700 1 -- 172.21.15.31:0/474731156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2a64004330 con 0x557f84382370 2023-12-14T18:29:50.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f8596ffd700 1 -- 172.21.15.31:0/3514974348 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7ffc23a00 con 0x55b7ffd47e20 2023-12-14T18:29:50.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f8596ffd700 10 client.25997 handle_command_reply: tid=0 2023-12-14T18:29:50.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.132+0000 7f85ca3deb80 1 client.25997 shutdown 2023-12-14T18:29:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a4effd700 1 -- 172.21.15.31:0/474731156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a640ca9c0 con 0x557f84382370 2023-12-14T18:29:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 10 client.16155 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:50.163 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f2a6800e6f0 con 0x557f84382370 2023-12-14T18:29:50.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 msgr2=0x55b7ffd46050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 0x55b7ffd46050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f85b001ba30 tx=0x7f85b001ba60 comp rx=0 tx=0).stop 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 shutdown_connections 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b7ffce3b50 0x55b7ffd46050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55b7ffd47e20 0x55b7ffd4a230 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b7ffcd8380 0x55b7ffce66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 --2- 172.21.15.31:0/3514974348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b7ffcd8c90 0x55b7ffce6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f8596ffd700 0 client.25997 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:29:50.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 >> 172.21.15.31:0/3514974348 conn(0x55b7ffcceca0 msgr2=0x55b7ffcddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a4effd700 1 -- 172.21.15.31:0/474731156 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a64063070 con 0x557f84382370 2023-12-14T18:29:50.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 shutdown_connections 2023-12-14T18:29:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a4effd700 1 -- 172.21.15.31:0/474731156 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a640995b0 con 0x557f84382370 2023-12-14T18:29:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 10 client.16155 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:50.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 20 client.16155 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f843f1630 0x557f843f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 4 client.16155 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.133+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f842cd9a0 con 0x557f843f1630 2023-12-14T18:29:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.134+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f843f1630 0x557f843f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:50.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.134+0000 7f85ca3deb80 1 -- 172.21.15.31:0/3514974348 wait complete. 2023-12-14T18:29:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.134+0000 7f2a6f7fe700 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f843f1630 0x557f843f3a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6d6833f1f519c server_cookie=3d0fb32aaf398d5d in_seq=0 out_seq=0 2023-12-14T18:29:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.134+0000 7f2a4effd700 10 client.16155 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:50.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.134+0000 7f85ca3deb80 20 client.25997 trim_cache size 0 max 16384 2023-12-14T18:29:50.167 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85a40a1340 msgr2=0x7f85a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85a40a1340 0x7f85a40a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f85b81957d0 tx=0x7f85a0009310 comp rx=0 tx=0).stop 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 msgr2=0x7f85b81941c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 0x7f85b81941c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009230 tx=0x7f85a800b810 comp rx=0 tx=0).stop 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 shutdown_connections 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f85b81054d0 0x7f85b8198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85b80680c0 0x7f85b81941c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f85a40a1340 0x7f85a40a37f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 --2- 172.21.15.31:0/3486140186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85b8068a60 0x7f85b8194700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 >> 172.21.15.31:0/3486140186 conn(0x7f85b8075fd0 msgr2=0x7f85b80fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 shutdown_connections 2023-12-14T18:29:50.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:50.135+0000 7f85be250700 1 -- 172.21.15.31:0/3486140186 wait complete. 2023-12-14T18:29:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a4effd700 1 -- 172.21.15.31:0/474731156 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f842cd9a0 con 0x557f843f1630 2023-12-14T18:29:50.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a4effd700 10 client.16155 handle_command_reply: tid=0 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 client.16155 shutdown 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 msgr2=0x557f84393e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f84393e40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2a64099d70 tx=0x7f2a640098e0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 shutdown_connections 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557f843f1630 0x557f843f3a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557f8438d0f0 0x557f84391e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f841df430 0x557f84393900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a4effd700 0 client.16155 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 --2- 172.21.15.31:0/474731156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557f84382370 0x557f84393e40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 >> 172.21.15.31:0/474731156 conn(0x557f84378890 msgr2=0x557f84383f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 shutdown_connections 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 1 -- 172.21.15.31:0/474731156 wait complete. 2023-12-14T18:29:50.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.154+0000 7f2a81e1db80 20 client.16155 trim_cache size 0 max 16384 2023-12-14T18:29:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:29:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:29:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:29:50.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2a580a12c0 msgr2=0x7f2a580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2a580a12c0 0x7f2a580a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009fd0 tx=0x7f2a60005fb0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 msgr2=0x7f2a70196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 0x7f2a70196880 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009fd0 tx=0x7f2a6400ed10 comp rx=0 tx=0).stop 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 shutdown_connections 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2a70105560 0x7f2a7019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2a580a12c0 0x7f2a580a3770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a70100c50 0x7f2a70196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 --2- 172.21.15.31:0/1649750458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2a701015f0 0x7f2a70196880 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 >> 172.21.15.31:0/1649750458 conn(0x7f2a70077550 msgr2=0x7f2a700fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 shutdown_connections 2023-12-14T18:29:50.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:29:50.155+0000 7f2a75c8f700 1 -- 172.21.15.31:0/1649750458 wait complete. 2023-12-14T18:29:50.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:29:50.477 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:50.477 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:50.575 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:50.575 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:50.579 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:50.579 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:50.579 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 73 -lt 100 ']' 2023-12-14T18:29:50.579 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 73 + 1 2023-12-14T18:29:50.615 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=74 2023-12-14T18:29:50.615 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:50.659 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:50.659 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:50.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:50.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:50.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:51.077 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:51.077 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:51.151 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:51.151 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:51.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:51.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:51.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 74 -lt 100 ']' 2023-12-14T18:29:51.157 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 74 + 1 2023-12-14T18:29:51.198 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=75 2023-12-14T18:29:51.198 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:51.269 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:51.270 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:51.271 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:51.271 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:51.271 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:51.744 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:51.744 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:51.915 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:51.915 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:51.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:51.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:51.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 75 -lt 100 ']' 2023-12-14T18:29:51.923 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 75 + 1 2023-12-14T18:29:51.956 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=76 2023-12-14T18:29:51.956 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:51.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:51 smithi031 ceph-mon[42085]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-12-14T18:29:51.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:51 smithi176 ceph-mon[43506]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-12-14T18:29:52.015 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:52.015 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:52.016 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:52.016 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:52.016 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:51 smithi204 ceph-mon[43866]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2023-12-14T18:29:52.769 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:52.769 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:53.048 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:53.048 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:53.058 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:53.058 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:53.058 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 76 -lt 100 ']' 2023-12-14T18:29:53.058 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 76 + 1 2023-12-14T18:29:53.139 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=77 2023-12-14T18:29:53.139 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:53.189 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:53.189 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:53.191 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:53.191 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:53.191 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 -- 172.21.15.31:0/3233340248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8288100c50 msgr2=0x7f8288101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 --2- 172.21.15.31:0/3233340248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8288100c50 0x7f8288101020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f82780099c0 tx=0x7f8278009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 -- 172.21.15.31:0/3233340248 shutdown_connections 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 --2- 172.21.15.31:0/3233340248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8288105550 0x7f828810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 --2- 172.21.15.31:0/3233340248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8288100c50 0x7f8288101020 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 --2- 172.21.15.31:0/3233340248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82881015f0 0x7f8288104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 -- 172.21.15.31:0/3233340248 >> 172.21.15.31:0/3233340248 conn(0x7f8288077570 msgr2=0x7f8288077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 -- 172.21.15.31:0/3233340248 shutdown_connections 2023-12-14T18:29:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.907+0000 7f828da4d700 1 -- 172.21.15.31:0/3233340248 wait complete. 2023-12-14T18:29:53.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 Processor -- start 2023-12-14T18:29:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 -- start start 2023-12-14T18:29:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288100c50 0x7f8288196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 0x7f82881969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8288105550 0x7f828819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819b410 con 0x7f82881015f0 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819b580 con 0x7f8288105550 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.908+0000 7f828da4d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828819b740 con 0x7f8288100c50 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 0x7f82881969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 0x7f82881969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42440/0 (socket says 172.21.15.31:42440) 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 -- 172.21.15.31:0/1265711140 learned_addr learned my addr 172.21.15.31:0/1265711140 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f8286ffd700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288100c50 0x7f8288196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82877fe700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8288105550 0x7f828819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 -- 172.21.15.31:0/1265711140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288100c50 msgr2=0x7f8288196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:53.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288100c50 0x7f8288196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 -- 172.21.15.31:0/1265711140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8288105550 msgr2=0x7f828819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8288105550 0x7f828819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 -- 172.21.15.31:0/1265711140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82780095f0 con 0x7f82881015f0 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f82867fc700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 0x7f82881969c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f827c00e910 tx=0x7f827c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f828ca4b700 1 -- 172.21.15.31:0/1265711140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c004a30 con 0x7f82881015f0 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.909+0000 7f828ca4b700 1 -- 172.21.15.31:0/1265711140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c0054e0 con 0x7f82881015f0 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.910+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8288110310 con 0x7f82881015f0 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.910+0000 7f828ca4b700 1 -- 172.21.15.31:0/1265711140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00cc40 con 0x7f82881015f0 2023-12-14T18:29:53.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.910+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8288110180 con 0x7f82881015f0 2023-12-14T18:29:53.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.912+0000 7f828ca4b700 1 -- 172.21.15.31:0/1265711140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f827c005000 con 0x7f82881015f0 2023-12-14T18:29:53.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.913+0000 7f828ca4b700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f82700a12c0 0x7f82700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.913+0000 7f8286ffd700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f82700a12c0 0x7f82700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.913+0000 7f828ca4b700 1 -- 172.21.15.31:0/1265711140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f827c014070 con 0x7f82881015f0 2023-12-14T18:29:53.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.914+0000 7f8286ffd700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f82700a12c0 0x7f82700a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8278009990 tx=0x7f8278005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:53.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:53 smithi031 ceph-mon[42085]: pgmap v65: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-12-14T18:29:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:53 smithi176 ceph-mon[43506]: pgmap v65: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-12-14T18:29:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 Processor -- start 2023-12-14T18:29:53.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 -- start start 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b49380 0x55c096b49750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b58a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b58f60 0x55c096b5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096a335e0 con 0x55c096b49c90 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096aa2870 con 0x55c096b58f60 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096a9bd00 con 0x55c096b49380 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f82867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b58a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f82867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b58a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42456/0 (socket says 172.21.15.31:42456) 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f82867fc700 1 -- 172.21.15.31:0/2219184847 learned_addr learned my addr 172.21.15.31:0/2219184847 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:53.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f82877fe700 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b58f60 0x55c096b5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.993+0000 7f8286ffd700 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b49380 0x55c096b49750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f82867fc700 1 -- 172.21.15.31:0/2219184847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b49380 msgr2=0x55c096b49750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f82867fc700 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b49380 0x55c096b49750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f82867fc700 1 -- 172.21.15.31:0/2219184847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b58f60 msgr2=0x55c096b5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f82867fc700 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b58f60 0x55c096b5b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f82867fc700 1 -- 172.21.15.31:0/2219184847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c09691d3b0 con 0x55c096b49c90 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f82867fc700 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b58a20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f827c0a1fe0 tx=0x7f827c099aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b922066bf0fbab server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f826dffb700 1 -- 172.21.15.31:0/2219184847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c09e0f0 con 0x55c096b49c90 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f826dffb700 1 -- 172.21.15.31:0/2219184847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c0cae90 con 0x55c096b49c90 2023-12-14T18:29:53.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.994+0000 7f826dffb700 1 -- 172.21.15.31:0/2219184847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c004b90 con 0x55c096b49c90 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 -- 172.21.15.31:0/2219184847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 msgr2=0x55c096b58a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b58a20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f827c0a1fe0 tx=0x7f827c099aa0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 -- 172.21.15.31:0/2219184847 shutdown_connections 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b58f60 0x55c096b5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b49380 0x55c096b49750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/2219184847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b58a20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 -- 172.21.15.31:0/2219184847 >> 172.21.15.31:0/2219184847 conn(0x55c096b3fca0 msgr2=0x55c096b457e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 -- 172.21.15.31:0/2219184847 shutdown_connections 2023-12-14T18:29:54.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.995+0000 7f8299bdbb80 1 -- 172.21.15.31:0/2219184847 wait complete. 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.996+0000 7f8299bdbb80 1 -- start start 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.996+0000 7f8299bdbb80 1 Processor -- start 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b55c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b56370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096a37f90 con 0x55c096b49380 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096a37c30 con 0x55c096b49c90 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c09691f6a0 con 0x55c096b58f60 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8299bdbb80 10 client.0 _ll_register_callbacks cb 0x55c096a5b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8286ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8286ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42468/0 (socket says 172.21.15.31:42468) 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f8286ffd700 1 -- 172.21.15.31:0/1657250276 learned_addr learned my addr 172.21.15.31:0/1657250276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f82867fc700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b55c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.997+0000 7f82877fe700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b56370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82867fc700 1 -- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 msgr2=0x55c096b56370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82867fc700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b56370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82867fc700 1 -- 172.21.15.31:0/1657250276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 msgr2=0x55c096b589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82867fc700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82867fc700 1 -- 172.21.15.31:0/1657250276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8288110180 con 0x55c096b49c90 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8286ffd700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82877fe700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b56370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f82867fc700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b55c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f827c009630 tx=0x7f827c004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9296b0517bcb0df7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:54.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8266ffd700 1 -- 172.21.15.31:0/1657250276 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0a2510 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8266ffd700 1 -- 172.21.15.31:0/1657250276 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f827c098460 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c096b3d2b0 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8266ffd700 1 -- 172.21.15.31:0/1657250276 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f827c004b90 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8299bdbb80 10 client.26012 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.998+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c096a62170 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8266ffd700 1 -- 172.21.15.31:0/1657250276 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c0ca020 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8266ffd700 1 -- 172.21.15.31:0/1657250276 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f827c0a2080 con 0x55c096b49c90 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8299bdbb80 10 client.26012 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8299bdbb80 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8299bdbb80 20 client.26012 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:54.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096b685a0 0x55c096b6a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8299bdbb80 4 client.26012 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c096a94ce0 con 0x55c096b685a0 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:53.999+0000 7f8286ffd700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096b685a0 0x55c096b6a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.000+0000 7f8286ffd700 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096b685a0 0x55c096b6a980 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f7603ec7c57713 server_cookie=5147d591410e8a88 in_seq=0 out_seq=0 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.000+0000 7f8266ffd700 10 client.26012 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.000+0000 7f8266ffd700 1 -- 172.21.15.31:0/1657250276 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c096a94ce0 con 0x55c096b685a0 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.000+0000 7f8266ffd700 10 client.26012 handle_command_reply: tid=0 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 client.26012 shutdown 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 msgr2=0x55c096b55c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b55c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f827c009630 tx=0x7f827c004f60 comp rx=0 tx=0).stop 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 shutdown_connections 2023-12-14T18:29:54.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096b685a0 0x55c096b6a980 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b56370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/1657250276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b55c80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8266ffd700 0 client.26012 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 >> 172.21.15.31:0/1657250276 conn(0x55c096b3fca0 msgr2=0x55c096b453d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 shutdown_connections 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 1 -- 172.21.15.31:0/1657250276 wait complete. 2023-12-14T18:29:54.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.001+0000 7f8299bdbb80 20 client.26012 trim_cache size 0 max 16384 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.015+0000 7f8299bdbb80 1 Processor -- start 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 -- start start 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b6f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b58490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096b6f820 con 0x55c096b49380 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096b55c40 con 0x55c096b49c90 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096b56370 con 0x55c096b58f60 2023-12-14T18:29:54.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b6f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b6f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42476/0 (socket says 172.21.15.31:42476) 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 -- 172.21.15.31:0/474928496 learned_addr learned my addr 172.21.15.31:0/474928496 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f82867fc700 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b58490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f82877fe700 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 -- 172.21.15.31:0/474928496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 msgr2=0x55c096b589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 -- 172.21.15.31:0/474928496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 msgr2=0x55c096b58490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b58490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.016+0000 7f8286ffd700 1 -- 172.21.15.31:0/474928496 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c096a61ec0 con 0x55c096b49380 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.017+0000 7f8286ffd700 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b6f2e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8278000c00 tx=0x7f8278022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ce78a4836d6d21 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.017+0000 7f826e7fc700 1 -- 172.21.15.31:0/474928496 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827802c040 con 0x55c096b49380 2023-12-14T18:29:54.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.017+0000 7f826e7fc700 1 -- 172.21.15.31:0/474928496 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827802d040 con 0x55c096b49380 2023-12-14T18:29:54.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.017+0000 7f826e7fc700 1 -- 172.21.15.31:0/474928496 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82780044f0 con 0x55c096b49380 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 -- 172.21.15.31:0/474928496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 msgr2=0x55c096b6f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b6f2e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8278000c00 tx=0x7f8278022000 comp rx=0 tx=0).stop 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 -- 172.21.15.31:0/474928496 shutdown_connections 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49380 0x55c096b6f2e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/474928496 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49c90 0x55c096b58490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 -- 172.21.15.31:0/474928496 >> 172.21.15.31:0/474928496 conn(0x55c096b3fca0 msgr2=0x55c096b41210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 -- 172.21.15.31:0/474928496 shutdown_connections 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 -- 172.21.15.31:0/474928496 wait complete. 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 -- start start 2023-12-14T18:29:54.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.018+0000 7f8299bdbb80 1 Processor -- start 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49380 0x55c096b5f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b5dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b5e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096b5e790 con 0x55c096b49c90 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096b5e8d0 con 0x55c096b49380 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c096b5ea90 con 0x55c096b58f60 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8299bdbb80 10 client.0 _ll_register_callbacks cb 0x7f8280009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b5dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b5dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42482/0 (socket says 172.21.15.31:42482) 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 -- 172.21.15.31:0/3567417099 learned_addr learned my addr 172.21.15.31:0/3567417099 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8286ffd700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49380 0x55c096b5f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82877fe700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b5e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 -- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 msgr2=0x55c096b5e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b5e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 -- 172.21.15.31:0/3567417099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49380 msgr2=0x55c096b5f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49380 0x55c096b5f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f82867fc700 1 -- 172.21.15.31:0/3567417099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828810d910 con 0x55c096b49c90 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.019+0000 7f8286ffd700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49380 0x55c096b5f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f82867fc700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b5dcc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f827c0cacc0 tx=0x7f827c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da127fe500ce975f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8266ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8266ffd700 1 -- 172.21.15.31:0/3567417099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c096400 con 0x55c096b49c90 2023-12-14T18:29:54.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8266ffd700 1 -- 172.21.15.31:0/3567417099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f827c0a2510 con 0x55c096b49c90 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f827c0cae90 con 0x55c096b49c90 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8266ffd700 1 -- 172.21.15.31:0/3567417099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f827c099a20 con 0x55c096b49c90 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 10 client.16173 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c096a489a0 con 0x55c096b49c90 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8266ffd700 1 -- 172.21.15.31:0/3567417099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f827c0cae90 con 0x55c096b49c90 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8266ffd700 1 -- 172.21.15.31:0/3567417099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f827c09a320 con 0x55c096b49c90 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 10 client.16173 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 20 client.16173 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.020+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096bbd430 0x55c096bbf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.021+0000 7f8299bdbb80 4 client.16173 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.021+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c096a94a00 con 0x55c096bbd430 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.021+0000 7f8286ffd700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096bbd430 0x55c096bbf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.021+0000 7f8266ffd700 10 client.16173 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.021+0000 7f8286ffd700 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096bbd430 0x55c096bbf810 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b23ecdefaceee0 server_cookie=633e26a8fdc155e5 in_seq=0 out_seq=0 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8266ffd700 1 -- 172.21.15.31:0/3567417099 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c096a94a00 con 0x55c096bbd430 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8266ffd700 10 client.16173 handle_command_reply: tid=0 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 client.16173 shutdown 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 msgr2=0x55c096b5dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b5dcc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f827c0cacc0 tx=0x7f827c0092d0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 shutdown_connections 2023-12-14T18:29:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c096bbd430 0x55c096bbf810 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c096b58f60 0x55c096b5e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c096b49380 0x55c096b5f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8266ffd700 0 client.16173 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 --2- 172.21.15.31:0/3567417099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c096b49c90 0x55c096b5dcc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 >> 172.21.15.31:0/3567417099 conn(0x55c096b3fca0 msgr2=0x55c096b51020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 shutdown_connections 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 1 -- 172.21.15.31:0/3567417099 wait complete. 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.022+0000 7f8299bdbb80 20 client.16173 trim_cache size 0 max 16384 2023-12-14T18:29:54.031 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:29:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f82700a12c0 msgr2=0x7f82700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f82700a12c0 0x7f82700a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8278009990 tx=0x7f8278005fb0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 msgr2=0x7f82881969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 0x7f82881969c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f827c00e910 tx=0x7f827c00ec20 comp rx=0 tx=0).stop 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 shutdown_connections 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8288105550 0x7f828819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f82700a12c0 0x7f82700a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8288100c50 0x7f8288196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 --2- 172.21.15.31:0/1265711140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82881015f0 0x7f82881969c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 >> 172.21.15.31:0/1265711140 conn(0x7f8288077570 msgr2=0x7f82880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 shutdown_connections 2023-12-14T18:29:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:54.023+0000 7f828da4d700 1 -- 172.21.15.31:0/1265711140 wait complete. 2023-12-14T18:29:54.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:53 smithi204 ceph-mon[43866]: pgmap v65: 129 pgs: 129 active+clean; 2.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 7 op/s 2023-12-14T18:29:54.277 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:54.277 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:54.614 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:54.615 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:54.707 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:54.707 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:54.707 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 77 -lt 100 ']' 2023-12-14T18:29:54.707 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 77 + 1 2023-12-14T18:29:54.743 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=78 2023-12-14T18:29:54.743 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:54.772 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:54.772 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:54.774 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:54.774 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:54.774 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:55.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:29:55.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:29:55.085 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:55.085 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:55.233 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:55.233 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:55.239 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:55.239 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:55.239 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 78 -lt 100 ']' 2023-12-14T18:29:55.240 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 78 + 1 2023-12-14T18:29:55.297 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=79 2023-12-14T18:29:55.297 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:55.333 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:55.333 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:55.335 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:55.335 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:55.335 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:55.780 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:55.780 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:55.897 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:55.897 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:55.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:55.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:55.923 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 79 -lt 100 ']' 2023-12-14T18:29:55.923 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 79 + 1 2023-12-14T18:29:55.955 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=80 2023-12-14T18:29:55.955 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:55.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:55 smithi031 ceph-mon[42085]: pgmap v66: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 16 op/s 2023-12-14T18:29:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:55 smithi176 ceph-mon[43506]: pgmap v66: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 16 op/s 2023-12-14T18:29:56.036 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:56.036 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:56.038 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:56.038 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:56.038 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:55 smithi204 ceph-mon[43866]: pgmap v66: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 16 op/s 2023-12-14T18:29:56.851 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:56.851 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:57.039 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:57.039 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:57.056 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:57.057 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:57.057 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 80 -lt 100 ']' 2023-12-14T18:29:57.057 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 80 + 1 2023-12-14T18:29:57.092 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=81 2023-12-14T18:29:57.092 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:57.174 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:57.174 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:57.175 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:57.176 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:57.176 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:57.926 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:57.926 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:57.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:57 smithi031 ceph-mon[42085]: pgmap v67: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-12-14T18:29:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:57 smithi176 ceph-mon[43506]: pgmap v67: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-12-14T18:29:58.080 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:58.080 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:58.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:57 smithi204 ceph-mon[43866]: pgmap v67: 129 pgs: 129 active+clean; 2.5 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 KiB/s wr, 12 op/s 2023-12-14T18:29:58.090 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:58.090 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:58.090 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 81 -lt 100 ']' 2023-12-14T18:29:58.090 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 81 + 1 2023-12-14T18:29:58.147 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=82 2023-12-14T18:29:58.147 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:58.272 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:58.272 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:58.273 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:58.273 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:58.274 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 -- 172.21.15.31:0/3407168199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 msgr2=0x7fc15010cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 --2- 172.21.15.31:0/3407168199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15010cc60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc140009990 tx=0x7fc140009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 -- 172.21.15.31:0/3407168199 shutdown_connections 2023-12-14T18:29:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 --2- 172.21.15.31:0/3407168199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15010cc60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 --2- 172.21.15.31:0/3407168199 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc150101340 0x7fc150101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 --2- 172.21.15.31:0/3407168199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 0x7fc15010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.306+0000 7fc15689d700 1 -- 172.21.15.31:0/3407168199 >> 172.21.15.31:0/3407168199 conn(0x7fc150075fc0 msgr2=0x7fc1500763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.307+0000 7fc15689d700 1 -- 172.21.15.31:0/3407168199 shutdown_connections 2023-12-14T18:29:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.307+0000 7fc15689d700 1 -- 172.21.15.31:0/3407168199 wait complete. 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.307+0000 7fc15689d700 1 Processor -- start 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.307+0000 7fc15689d700 1 -- start start 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc15689d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc150101340 0x7fc150196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc15689d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 0x7fc150196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc15689d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc15689d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15019b2e0 con 0x7fc15010a860 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc15689d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15019b450 con 0x7fc150101340 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc15689d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15019b610 con 0x7fc150101c50 2023-12-14T18:29:58.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc154e3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc154e3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51110/0 (socket says 172.21.15.31:51110) 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc154e3a700 1 -- 172.21.15.31:0/1526436148 learned_addr learned my addr 172.21.15.31:0/1526436148 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 0x7fc150196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.308+0000 7fc14ffff700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc150101340 0x7fc150196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14f7fe700 1 -- 172.21.15.31:0/1526436148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc150101340 msgr2=0x7fc150196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc150101340 0x7fc150196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14f7fe700 1 -- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 msgr2=0x7fc15019aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15019aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14f7fe700 1 -- 172.21.15.31:0/1526436148 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1400095f0 con 0x7fc150101c50 2023-12-14T18:29:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc154e3a700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 0x7fc150196840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc13800e9f0 tx=0x7fc13800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14d7fa700 1 -- 172.21.15.31:0/1526436148 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800c9c0 con 0x7fc150101c50 2023-12-14T18:29:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc150110290 con 0x7fc150101c50 2023-12-14T18:29:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14d7fa700 1 -- 172.21.15.31:0/1526436148 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc138010430 con 0x7fc150101c50 2023-12-14T18:29:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc14d7fa700 1 -- 172.21.15.31:0/1526436148 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138018680 con 0x7fc150101c50 2023-12-14T18:29:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.309+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc150110100 con 0x7fc150101c50 2023-12-14T18:29:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.312+0000 7fc14d7fa700 1 -- 172.21.15.31:0/1526436148 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc138010860 con 0x7fc150101c50 2023-12-14T18:29:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.313+0000 7fc14d7fa700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc13c0a12c0 0x7fc13c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.313+0000 7fc14ffff700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc13c0a12c0 0x7fc13c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.313+0000 7fc14d7fa700 1 -- 172.21.15.31:0/1526436148 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc138014070 con 0x7fc150101c50 2023-12-14T18:29:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.313+0000 7fc14ffff700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc13c0a12c0 0x7fc13c0a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc14800f540 tx=0x7fc14800b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 Processor -- start 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 -- start start 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 0x5643f5c207d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20d10 0x5643f5c2faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c323c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5b0a660 con 0x5643f5c20400 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5b798f0 con 0x5643f5c20d10 2023-12-14T18:29:58.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5b72d80 con 0x5643f5c2ffe0 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc14ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 0x5643f5c207d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc14ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 0x5643f5c207d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51124/0 (socket says 172.21.15.31:51124) 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.396+0000 7fc14ffff700 1 -- 172.21.15.31:0/461851521 learned_addr learned my addr 172.21.15.31:0/461851521 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20d10 0x5643f5c2faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc154e3a700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 -- 172.21.15.31:0/461851521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 msgr2=0x5643f5c323c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c323c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 -- 172.21.15.31:0/461851521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 msgr2=0x5643f5c207d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 0x5643f5c207d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 -- 172.21.15.31:0/461851521 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643f59f4430 con 0x5643f5c20d10 2023-12-14T18:29:58.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc154e3a700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c323c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14ffff700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 0x5643f5c207d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20d10 0x5643f5c2faa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1380ca550 tx=0x7fc138099dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896f520231271318 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc1467fc700 1 -- 172.21.15.31:0/461851521 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380a2320 con 0x5643f5c20d10 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc1467fc700 1 -- 172.21.15.31:0/461851521 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1380105a0 con 0x5643f5c20d10 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.397+0000 7fc1467fc700 1 -- 172.21.15.31:0/461851521 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380a1e90 con 0x5643f5c20d10 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 -- 172.21.15.31:0/461851521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20d10 msgr2=0x5643f5c2faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20d10 0x5643f5c2faa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1380ca550 tx=0x7fc138099dd0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 -- 172.21.15.31:0/461851521 shutdown_connections 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c323c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20400 0x5643f5c207d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/461851521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20d10 0x5643f5c2faa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 -- 172.21.15.31:0/461851521 >> 172.21.15.31:0/461851521 conn(0x5643f5c16d20 msgr2=0x5643f5c1c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.398+0000 7fc162a2bb80 1 -- 172.21.15.31:0/461851521 shutdown_connections 2023-12-14T18:29:58.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.399+0000 7fc162a2bb80 1 -- 172.21.15.31:0/461851521 wait complete. 2023-12-14T18:29:58.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.399+0000 7fc162a2bb80 1 -- start start 2023-12-14T18:29:58.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.399+0000 7fc162a2bb80 1 Processor -- start 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5b0f010 con 0x5643f5c20d10 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5b0ecb0 con 0x5643f5c20400 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f59f6720 con 0x5643f5c2ffe0 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc162a2bb80 10 client.0 _ll_register_callbacks cb 0x5643f5b32990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51138/0 (socket says 172.21.15.31:51138) 2023-12-14T18:29:58.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 -- 172.21.15.31:0/3119238366 learned_addr learned my addr 172.21.15.31:0/3119238366 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14ffff700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc154e3a700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 -- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 msgr2=0x5643f5c2d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c2d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 -- 172.21.15.31:0/3119238366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 msgr2=0x5643f5c2fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.400+0000 7fc14f7fe700 1 -- 172.21.15.31:0/3119238366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc150110100 con 0x5643f5c20d10 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2cd30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc1380a1df0 tx=0x7fc138099670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f09795b31d7fcf9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc12f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc12f7fe700 1 -- 172.21.15.31:0/3119238366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380a2320 con 0x5643f5c20d10 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc12f7fe700 1 -- 172.21.15.31:0/3119238366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc1380adde0 con 0x5643f5c20d10 2023-12-14T18:29:58.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643f5c14330 con 0x5643f5c20d10 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc12f7fe700 1 -- 172.21.15.31:0/3119238366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1380a1e90 con 0x5643f5c20d10 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc162a2bb80 10 client.16185 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5643f5b391f0 con 0x5643f5c20d10 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.401+0000 7fc12f7fe700 1 -- 172.21.15.31:0/3119238366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643f5c14330 con 0x5643f5c20d10 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc12f7fe700 1 -- 172.21.15.31:0/3119238366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc138095cc0 con 0x5643f5c20d10 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc162a2bb80 10 client.16185 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc162a2bb80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc162a2bb80 20 client.16185 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c3f500 0x5643f5c41910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc162a2bb80 4 client.16185 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643f5b6bd60 con 0x5643f5c3f500 2023-12-14T18:29:58.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.402+0000 7fc14ffff700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c3f500 0x5643f5c41910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.403+0000 7fc14ffff700 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c3f500 0x5643f5c41910 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920ad26140cb1608 server_cookie=a13f873a9e92b900 in_seq=0 out_seq=0 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.403+0000 7fc12f7fe700 10 client.16185 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.403+0000 7fc12f7fe700 1 -- 172.21.15.31:0/3119238366 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5643f5b6bd60 con 0x5643f5c3f500 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.403+0000 7fc12f7fe700 10 client.16185 handle_command_reply: tid=0 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.403+0000 7fc162a2bb80 1 client.16185 shutdown 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 msgr2=0x5643f5c2cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2cd30 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc1380a1df0 tx=0x7fc138099670 comp rx=0 tx=0).stop 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 shutdown_connections 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c3f500 0x5643f5c41910 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc12f7fe700 0 client.16185 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:58.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/3119238366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2cd30 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 >> 172.21.15.31:0/3119238366 conn(0x5643f5c16d20 msgr2=0x5643f5c1c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 shutdown_connections 2023-12-14T18:29:58.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 1 -- 172.21.15.31:0/3119238366 wait complete. 2023-12-14T18:29:58.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.404+0000 7fc162a2bb80 20 client.16185 trim_cache size 0 max 16384 2023-12-14T18:29:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 Processor -- start 2023-12-14T18:29:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 -- start start 2023-12-14T18:29:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c29fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5c2e2a0 con 0x5643f5c20d10 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5c23410 con 0x5643f5c20400 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5c1d350 con 0x5643f5c2ffe0 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51154/0 (socket says 172.21.15.31:51154) 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 -- 172.21.15.31:0/596405636 learned_addr learned my addr 172.21.15.31:0/596405636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14ffff700 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc154e3a700 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c29fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 -- 172.21.15.31:0/596405636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 msgr2=0x5643f5c29fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c29fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 -- 172.21.15.31:0/596405636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 msgr2=0x5643f5c2f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.424+0000 7fc14f7fe700 1 -- 172.21.15.31:0/596405636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643f5b38f40 con 0x5643f5c20d10 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.425+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2fa80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc138099bd0 tx=0x7fc1380107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263c50bc6ac07682 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.425+0000 7fc146ffd700 1 -- 172.21.15.31:0/596405636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380b82c0 con 0x5643f5c20d10 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.425+0000 7fc146ffd700 1 -- 172.21.15.31:0/596405636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1380a2320 con 0x5643f5c20d10 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.425+0000 7fc146ffd700 1 -- 172.21.15.31:0/596405636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380a1e90 con 0x5643f5c20d10 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.425+0000 7fc162a2bb80 1 -- 172.21.15.31:0/596405636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 msgr2=0x5643f5c2fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.425+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2fa80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc138099bd0 tx=0x7fc1380107d0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 -- 172.21.15.31:0/596405636 shutdown_connections 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c2ffe0 0x5643f5c29fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c2f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/596405636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c20d10 0x5643f5c2fa80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 -- 172.21.15.31:0/596405636 >> 172.21.15.31:0/596405636 conn(0x5643f5c16d20 msgr2=0x5643f5c28040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 -- 172.21.15.31:0/596405636 shutdown_connections 2023-12-14T18:29:58.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 -- 172.21.15.31:0/596405636 wait complete. 2023-12-14T18:29:58.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.426+0000 7fc162a2bb80 1 -- start start 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 Processor -- start 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c35770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c20d10 0x5643f5c35cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c2ffe0 0x5643f5c933d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5c363c0 con 0x5643f5c2ffe0 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5c93910 con 0x5643f5c20400 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643f5c93a50 con 0x5643f5c20d10 2023-12-14T18:29:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc162a2bb80 10 client.0 _ll_register_callbacks cb 0x7fc14800fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc154e3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c2ffe0 0x5643f5c933d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc14ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c35770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc154e3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c2ffe0 0x5643f5c933d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51168/0 (socket says 172.21.15.31:51168) 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc14ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c35770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:52370/0 (socket says 172.21.15.31:52370) 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc154e3a700 1 -- 172.21.15.31:0/4135719615 learned_addr learned my addr 172.21.15.31:0/4135719615 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.427+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c20d10 0x5643f5c35cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc14ffff700 1 -- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c20d10 msgr2=0x5643f5c35cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc14ffff700 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c20d10 0x5643f5c35cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc14ffff700 1 -- 172.21.15.31:0/4135719615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c2ffe0 msgr2=0x5643f5c933d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc14ffff700 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c2ffe0 0x5643f5c933d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc14ffff700 1 -- 172.21.15.31:0/4135719615 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643f5b391f0 con 0x5643f5c20400 2023-12-14T18:29:58.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc14ffff700 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c35770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5643f5c421f0 tx=0x7fc148004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6e8355e527a9bc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc12f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc12f7fe700 1 -- 172.21.15.31:0/4135719615 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14802b070 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc12f7fe700 1 -- 172.21.15.31:0/4135719615 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fc1480048a0 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1380105a0 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc12f7fe700 1 -- 172.21.15.31:0/4135719615 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc14800e040 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc162a2bb80 10 client.26036 fetch_fsmap learned FSMap version 24 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.428+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5643f5b1fa20 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc12f7fe700 1 -- 172.21.15.31:0/4135719615 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1380105a0 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc12f7fe700 1 -- 172.21.15.31:0/4135719615 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc148025940 con 0x5643f5c20400 2023-12-14T18:29:58.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc162a2bb80 10 client.26036 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc162a2bb80 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc162a2bb80 20 client.26036 populate_metadata read hostname 'smithi031' 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c951a0 0x5643f5c975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc162a2bb80 4 client.26036 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5643f5b6ba80 con 0x5643f5c951a0 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c951a0 0x5643f5c975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.429+0000 7fc14f7fe700 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c951a0 0x5643f5c975b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b0b0dcf197afe6 server_cookie=1ecf3c9473e7e1ae in_seq=0 out_seq=0 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.430+0000 7fc12f7fe700 10 client.26036 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc12f7fe700 1 -- 172.21.15.31:0/4135719615 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5643f5b6ba80 con 0x5643f5c951a0 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc12f7fe700 10 client.26036 handle_command_reply: tid=0 2023-12-14T18:29:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 client.26036 shutdown 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 msgr2=0x5643f5c35770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c35770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5643f5c421f0 tx=0x7fc148004ad0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 shutdown_connections 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5643f5c951a0 0x5643f5c975b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5643f5c2ffe0 0x5643f5c933d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5643f5c20400 0x5643f5c35770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 --2- 172.21.15.31:0/4135719615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643f5c20d10 0x5643f5c35cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc12f7fe700 0 client.26036 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 >> 172.21.15.31:0/4135719615 conn(0x5643f5c16d20 msgr2=0x5643f5c222e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 shutdown_connections 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 1 -- 172.21.15.31:0/4135719615 wait complete. 2023-12-14T18:29:58.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.431+0000 7fc162a2bb80 20 client.26036 trim_cache size 0 max 16384 2023-12-14T18:29:58.440 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:29:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:29:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "99d61672-4149-4b11-ae4c-785b4e8bffd9", 2023-12-14T18:29:58.440 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:29:58.440 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc13c0a12c0 msgr2=0x7fc13c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc13c0a12c0 0x7fc13c0a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc14800f540 tx=0x7fc14800b040 comp rx=0 tx=0).stop 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 msgr2=0x7fc150196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 0x7fc150196840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc13800e9f0 tx=0x7fc13800ed00 comp rx=0 tx=0).stop 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 shutdown_connections 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc15010a860 0x7fc15019aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc150101340 0x7fc150196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc13c0a12c0 0x7fc13c0a3770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 --2- 172.21.15.31:0/1526436148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc150101c50 0x7fc150196840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 >> 172.21.15.31:0/1526436148 conn(0x7fc150075fc0 msgr2=0x7fc1500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 shutdown_connections 2023-12-14T18:29:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:29:58.432+0000 7fc15689d700 1 -- 172.21.15.31:0/1526436148 wait complete. 2023-12-14T18:29:58.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:29:58.901 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:58.901 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:58.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:29:58.984 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:29:58.996 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:58.996 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:29:58.996 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 82 -lt 100 ']' 2023-12-14T18:29:58.997 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 82 + 1 2023-12-14T18:29:59.021 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=83 2023-12-14T18:29:59.021 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:29:59.049 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:29:59.049 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:29:59.051 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:29:59.051 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:29:59.051 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:29:59.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:59 smithi031 ceph-mon[42085]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-12-14T18:29:59.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:59 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:29:59.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:59 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:59.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:59 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:29:59.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:59 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:29:59.879 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:29:59 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:29:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:59 smithi176 ceph-mon[43506]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-12-14T18:30:00.092 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:59 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:30:00.092 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:59 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:00.092 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:59 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:30:00.092 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:59 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:00.092 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:29:59 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:30:00.093 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:59 smithi204 ceph-mon[43866]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-12-14T18:30:00.093 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:59 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:30:00.093 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:59 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:00.093 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:59 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:30:00.093 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:59 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:00.093 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:29:59 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:30:00.093 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:00.093 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:00.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:00.157 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:00.164 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:00.164 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:00.165 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 83 -lt 100 ']' 2023-12-14T18:30:00.165 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 83 + 1 2023-12-14T18:30:00.221 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=84 2023-12-14T18:30:00.221 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:00.301 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:00.301 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:00.302 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:00.302 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:00.303 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:00.776 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:00.776 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:00.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:00 smithi031 ceph-mon[42085]: overall HEALTH_OK 2023-12-14T18:30:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:00 smithi176 ceph-mon[43506]: overall HEALTH_OK 2023-12-14T18:30:01.021 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:01.022 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:01.030 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:01.030 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:01.030 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 84 -lt 100 ']' 2023-12-14T18:30:01.030 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 84 + 1 2023-12-14T18:30:01.080 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=85 2023-12-14T18:30:01.080 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:00 smithi204 ceph-mon[43866]: overall HEALTH_OK 2023-12-14T18:30:01.122 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:01.122 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:01.124 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:01.124 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:01.124 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:01.909 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:01.909 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:01.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:01 smithi031 ceph-mon[42085]: pgmap v69: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-12-14T18:30:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:01 smithi176 ceph-mon[43506]: pgmap v69: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-12-14T18:30:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:01 smithi204 ceph-mon[43866]: pgmap v69: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 13 op/s 2023-12-14T18:30:02.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:02.155 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:02.189 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:02.189 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:02.190 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 85 -lt 100 ']' 2023-12-14T18:30:02.190 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 85 + 1 2023-12-14T18:30:02.221 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=86 2023-12-14T18:30:02.221 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:02.280 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:02.280 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:02.281 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:02.281 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:02.281 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 -- 172.21.15.31:0/1157329660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 msgr2=0x7f6ef410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 --2- 172.21.15.31:0/1157329660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef410d0c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00b0d0 tx=0x7f6eec00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 -- 172.21.15.31:0/1157329660 shutdown_connections 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 --2- 172.21.15.31:0/1157329660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef410d0c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 --2- 172.21.15.31:0/1157329660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 0x7f6ef4100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 --2- 172.21.15.31:0/1157329660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ef4100e50 0x7f6ef4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 -- 172.21.15.31:0/1157329660 >> 172.21.15.31:0/1157329660 conn(0x7f6ef40fbd20 msgr2=0x7f6ef40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 -- 172.21.15.31:0/1157329660 shutdown_connections 2023-12-14T18:30:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.556+0000 7f6ef93be700 1 -- 172.21.15.31:0/1157329660 wait complete. 2023-12-14T18:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.557+0000 7f6ef93be700 1 Processor -- start 2023-12-14T18:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 -- start start 2023-12-14T18:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 0x7f6ef4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ef4100e50 0x7f6ef41969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef419b440 con 0x7f6ef41054f0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef419b5b0 con 0x7f6ef4100e50 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef93be700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef419b770 con 0x7f6ef41004b0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51174/0 (socket says 172.21.15.31:51174) 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef37fe700 1 -- 172.21.15.31:0/159160834 learned_addr learned my addr 172.21.15.31:0/159160834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ef4100e50 0x7f6ef41969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.558+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 0x7f6ef4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef37fe700 1 -- 172.21.15.31:0/159160834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 msgr2=0x7f6ef4196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 0x7f6ef4196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef37fe700 1 -- 172.21.15.31:0/159160834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ef4100e50 msgr2=0x7f6ef41969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ef4100e50 0x7f6ef41969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef37fe700 1 -- 172.21.15.31:0/159160834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec009d00 con 0x7f6ef41054f0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 0x7f6ef4196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef419ae00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007870 tx=0x7f6eec0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:02.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6edbfff700 1 -- 172.21.15.31:0/159160834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec00e040 con 0x7f6ef41054f0 2023-12-14T18:30:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6edbfff700 1 -- 172.21.15.31:0/159160834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6eec005070 con 0x7f6ef41054f0 2023-12-14T18:30:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef41102c0 con 0x7f6ef41054f0 2023-12-14T18:30:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6edbfff700 1 -- 172.21.15.31:0/159160834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec01b6c0 con 0x7f6ef41054f0 2023-12-14T18:30:02.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.559+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef4110130 con 0x7f6ef41054f0 2023-12-14T18:30:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.562+0000 7f6edbfff700 1 -- 172.21.15.31:0/159160834 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6eec0051e0 con 0x7f6ef41054f0 2023-12-14T18:30:02.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.563+0000 7f6edbfff700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6edc0a1340 0x7f6edc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.563+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6edc0a1340 0x7f6edc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.563+0000 7f6edbfff700 1 -- 172.21.15.31:0/159160834 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6eec0ce160 con 0x7f6ef41054f0 2023-12-14T18:30:02.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.563+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6edc0a1340 0x7f6edc0a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011490 tx=0x7f6ee4009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:02.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.608+0000 7f6f0554cb80 1 Processor -- start 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 -- start start 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3e3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f2cd5e0 con 0x56235f3eeb50 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f33c870 con 0x56235f3e3c90 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f335d00 con 0x56235f3e3380 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6ef2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6ef2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3e3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:41236/0 (socket says 172.21.15.31:41236) 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6ef37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/2699845197 learned_addr learned my addr 172.21.15.31:0/2699845197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.609+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.610+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/2699845197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 msgr2=0x56235f3ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.610+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.610+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/2699845197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 msgr2=0x56235f3f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.610+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.610+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/2699845197 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56235f1b73b0 con 0x56235f3e3380 2023-12-14T18:30:02.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.610+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3e3750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40043c0 tx=0x7f6ee4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b57f9a82c302461 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.613+0000 7f6ed97fa700 1 -- 172.21.15.31:0/2699845197 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4003d10 con 0x56235f3e3380 2023-12-14T18:30:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.613+0000 7f6ed97fa700 1 -- 172.21.15.31:0/2699845197 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee4003e70 con 0x56235f3e3380 2023-12-14T18:30:02.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.613+0000 7f6ed97fa700 1 -- 172.21.15.31:0/2699845197 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee401e7e0 con 0x56235f3e3380 2023-12-14T18:30:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2699845197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 msgr2=0x56235f3e3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3e3750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40043c0 tx=0x7f6ee4014000 comp rx=0 tx=0).stop 2023-12-14T18:30:02.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2699845197 shutdown_connections 2023-12-14T18:30:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3e3750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2699845197 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2699845197 >> 172.21.15.31:0/2699845197 conn(0x56235f3d9ca0 msgr2=0x56235f3df7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2699845197 shutdown_connections 2023-12-14T18:30:02.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.614+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2699845197 wait complete. 2023-12-14T18:30:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 -- start start 2023-12-14T18:30:02.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 Processor -- start 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3380 0x56235f3ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f2d1f90 con 0x56235f3eeb50 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f2d1c30 con 0x56235f3e3380 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.615+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f1b96a0 con 0x56235f3e3c90 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6f0554cb80 10 client.0 _ll_register_callbacks cb 0x56235f2f5910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3ebfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51200/0 (socket says 172.21.15.31:51200) 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 -- 172.21.15.31:0/810510687 learned_addr learned my addr 172.21.15.31:0/810510687 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3380 0x56235f3ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 -- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 msgr2=0x56235f3eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 -- 172.21.15.31:0/810510687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3380 msgr2=0x56235f3ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3380 0x56235f3ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 -- 172.21.15.31:0/810510687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee4017070 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3ebfa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007910 tx=0x7f6eec09a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de6767a5340b84c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.616+0000 7f6ed6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/810510687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec09b330 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/810510687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6eec0a3290 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56235f3d72b0 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/810510687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eec0b9140 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 10 client.16203 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56235f2fc170 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/810510687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56235f3d72b0 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/810510687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6eec003a00 con 0x56235f3eeb50 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 10 client.16203 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 20 client.16203 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:02.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3fe180 0x56235f400590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 4 client.16203 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:02.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.617+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56235f32ece0 con 0x56235f3fe180 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.618+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3fe180 0x56235f400590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.618+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3fe180 0x56235f400590 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bea92ccbaedc4f5 server_cookie=e029199953fe694b in_seq=0 out_seq=0 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.618+0000 7f6ed6ff5700 10 client.16203 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/810510687 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56235f32ece0 con 0x56235f3fe180 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6ed6ff5700 10 client.16203 handle_command_reply: tid=0 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 client.16203 shutdown 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 msgr2=0x56235f3ebfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3ebfa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007910 tx=0x7f6eec09a8b0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 shutdown_connections 2023-12-14T18:30:02.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3fe180 0x56235f400590 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3eeb50 0x56235f3ebfa0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3380 0x56235f3ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/810510687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6ed6ff5700 0 client.16203 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 >> 172.21.15.31:0/810510687 conn(0x56235f3d9ca0 msgr2=0x56235f3df3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.619+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 shutdown_connections 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.620+0000 7f6f0554cb80 1 -- 172.21.15.31:0/810510687 wait complete. 2023-12-14T18:30:02.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.620+0000 7f6f0554cb80 20 client.16203 trim_cache size 0 max 16384 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.638+0000 7f6f0554cb80 1 Processor -- start 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 -- start start 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 0x56235f3f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f3d9ae0 con 0x56235f3d9ca0 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f3e43c0 con 0x56235f3e3c90 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f3e4d80 con 0x56235f3e3380 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 0x56235f3f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 0x56235f3f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51212/0 (socket says 172.21.15.31:51212) 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/3181484551 learned_addr learned my addr 172.21.15.31:0/3181484551 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/3181484551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 msgr2=0x56235f3f9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3f9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/3181484551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 msgr2=0x56235f3ee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef2ffd700 1 -- 172.21.15.31:0/3181484551 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56235f2fbec0 con 0x56235f3d9ca0 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.639+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.640+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 0x56235f3f5690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4004390 tx=0x7f6ee4003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b459b7a60b1c05 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.640+0000 7f6ed9ffb700 1 -- 172.21.15.31:0/3181484551 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee402a040 con 0x56235f3d9ca0 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.640+0000 7f6ed9ffb700 1 -- 172.21.15.31:0/3181484551 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee402b040 con 0x56235f3d9ca0 2023-12-14T18:30:02.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.640+0000 7f6ed9ffb700 1 -- 172.21.15.31:0/3181484551 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40054b0 con 0x56235f3d9ca0 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 -- 172.21.15.31:0/3181484551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 msgr2=0x56235f3f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 0x56235f3f5690 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4004390 tx=0x7f6ee4003db0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 -- 172.21.15.31:0/3181484551 shutdown_connections 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3e3c90 0x56235f3ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3d9ca0 0x56235f3f5690 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/3181484551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3380 0x56235f3f9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 -- 172.21.15.31:0/3181484551 >> 172.21.15.31:0/3181484551 conn(0x56235f3f1200 msgr2=0x56235f3fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:02.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 -- 172.21.15.31:0/3181484551 shutdown_connections 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 -- 172.21.15.31:0/3181484551 wait complete. 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 -- start start 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 Processor -- start 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.641+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3d9ca0 0x56235f3f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 0x56235f3f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6f0554cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f3f43c0 con 0x56235f3e3380 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f3f4530 con 0x56235f3d9ca0 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6f0554cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56235f452050 con 0x56235f3e3c90 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6f0554cb80 10 client.0 _ll_register_callbacks cb 0x7f6eec09aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 0x56235f3f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 0x56235f3f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51224/0 (socket says 172.21.15.31:51224) 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 -- 172.21.15.31:0/2420623096 learned_addr learned my addr 172.21.15.31:0/2420623096 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:02.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3d9ca0 0x56235f3f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef37fe700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 -- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 msgr2=0x56235f3f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 -- 172.21.15.31:0/2420623096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3d9ca0 msgr2=0x56235f3f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3d9ca0 0x56235f3f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 -- 172.21.15.31:0/2420623096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56235f1b73b0 con 0x56235f3e3380 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ef27fc700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 0x56235f3f37b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6ef406a1e0 tx=0x7f6ee8013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447135dcb74ab40a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ed6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.642+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/2420623096 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8020070 con 0x56235f3e3380 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/2420623096 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6ee80130d0 con 0x56235f3e3380 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee4003e70 con 0x56235f3e3380 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/2420623096 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee8016040 con 0x56235f3e3380 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 10 client.16209 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56235f2e29a0 con 0x56235f3e3380 2023-12-14T18:30:02.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/2420623096 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee4003e70 con 0x56235f3e3380 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/2420623096 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ee8023020 con 0x56235f3e3380 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 10 client.16209 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 20 client.16209 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3eeb50 0x56235f455710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 4 client.16209 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56235f32ea00 con 0x56235f3eeb50 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.643+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3eeb50 0x56235f455710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6ef2ffd700 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3eeb50 0x56235f455710 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca47bd932eee810 server_cookie=9bb32081ab6d9f2 in_seq=0 out_seq=0 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6ed6ff5700 10 client.16209 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6ed6ff5700 1 -- 172.21.15.31:0/2420623096 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56235f32ea00 con 0x56235f3eeb50 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6ed6ff5700 10 client.16209 handle_command_reply: tid=0 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 client.16209 shutdown 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 msgr2=0x56235f3f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 0x56235f3f37b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6ef406a1e0 tx=0x7f6ee8013740 comp rx=0 tx=0).stop 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 shutdown_connections 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56235f3eeb50 0x56235f455710 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56235f3e3c90 0x56235f3f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56235f3d9ca0 0x56235f3f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6ed6ff5700 0 client.16209 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 --2- 172.21.15.31:0/2420623096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56235f3e3380 0x56235f3f37b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 >> 172.21.15.31:0/2420623096 conn(0x56235f3f1200 msgr2=0x56235f3f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 shutdown_connections 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.644+0000 7f6f0554cb80 1 -- 172.21.15.31:0/2420623096 wait complete. 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6f0554cb80 20 client.16209 trim_cache size 0 max 16384 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:30:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6edc0a1340 msgr2=0x7f6edc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6edc0a1340 0x7f6edc0a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4011490 tx=0x7f6ee4009260 comp rx=0 tx=0).stop 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 msgr2=0x7f6ef419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef419ae00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6eec007870 tx=0x7f6eec0078a0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 shutdown_connections 2023-12-14T18:30:02.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ef41054f0 0x7f6ef419ae00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6edc0a1340 0x7f6edc0a37f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef41004b0 0x7f6ef4196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 --2- 172.21.15.31:0/159160834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6ef4100e50 0x7f6ef41969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:02.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 >> 172.21.15.31:0/159160834 conn(0x7f6ef40fbd20 msgr2=0x7f6ef4103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:02.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 shutdown_connections 2023-12-14T18:30:02.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:02.645+0000 7f6ef93be700 1 -- 172.21.15.31:0/159160834 wait complete. 2023-12-14T18:30:02.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:30:02.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:30:02.842 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:02.842 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:02.948 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:02.948 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:02.962 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:02.962 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:02.963 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 86 -lt 100 ']' 2023-12-14T18:30:02.963 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 86 + 1 2023-12-14T18:30:03.004 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=87 2023-12-14T18:30:03.004 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:03.027 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:03.027 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:03.028 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:03.029 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:03.029 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:03.675 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:03.675 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:03.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:03 smithi031 ceph-mon[42085]: pgmap v70: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-12-14T18:30:03.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:03 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:30:03.792 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:03.792 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:03.823 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:03.823 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:03.823 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 87 -lt 100 ']' 2023-12-14T18:30:03.823 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 87 + 1 2023-12-14T18:30:03.871 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=88 2023-12-14T18:30:03.871 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:03.920 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:03.920 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:03.922 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:03.922 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:03.922 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:03 smithi176 ceph-mon[43506]: pgmap v70: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-12-14T18:30:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:03 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:30:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:03 smithi204 ceph-mon[43866]: pgmap v70: 129 pgs: 129 active+clean; 2.6 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-12-14T18:30:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:03 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:30:05.125 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:05.125 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:05.504 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:05.505 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:05.539 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:05.539 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:05.539 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 88 -lt 100 ']' 2023-12-14T18:30:05.539 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 88 + 1 2023-12-14T18:30:05.571 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=89 2023-12-14T18:30:05.571 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:05.594 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:05.594 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:05.596 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:05.596 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:05.596 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:05.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:05 smithi031 ceph-mon[42085]: pgmap v71: 129 pgs: 129 active+clean; 2.7 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 30 op/s 2023-12-14T18:30:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:05 smithi176 ceph-mon[43506]: pgmap v71: 129 pgs: 129 active+clean; 2.7 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 30 op/s 2023-12-14T18:30:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:05 smithi204 ceph-mon[43866]: pgmap v71: 129 pgs: 129 active+clean; 2.7 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 30 op/s 2023-12-14T18:30:06.550 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:06.550 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 -- 172.21.15.31:0/3248324465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 msgr2=0x7f4c24108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 --2- 172.21.15.31:0/3248324465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c24108840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c140098f0 tx=0x7f4c14009c00 comp rx=0 tx=0).stop 2023-12-14T18:30:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 -- 172.21.15.31:0/1997449064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c101390 msgr2=0x7fd92c10d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 --2- 172.21.15.31:0/1997449064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c101390 0x7fd92c10d060 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd924009f90 tx=0x7fd92400b330 comp rx=0 tx=0).stop 2023-12-14T18:30:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 -- 172.21.15.31:0/1997449064 shutdown_connections 2023-12-14T18:30:06.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 --2- 172.21.15.31:0/1997449064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c101390 0x7fd92c10d060 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 --2- 172.21.15.31:0/1997449064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd92c1051b0 0x7fd92c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 --2- 172.21.15.31:0/1997449064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd92c1009e0 0x7fd92c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 -- 172.21.15.31:0/3248324465 shutdown_connections 2023-12-14T18:30:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 --2- 172.21.15.31:0/3248324465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c24102e10 0x7f4c2410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 --2- 172.21.15.31:0/3248324465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c24108840 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 --2- 172.21.15.31:0/3248324465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c24102460 0x7f4c241028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 -- 172.21.15.31:0/3248324465 >> 172.21.15.31:0/3248324465 conn(0x7f4c240fdf60 msgr2=0x7f4c24100370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:06.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 -- 172.21.15.31:0/1997449064 >> 172.21.15.31:0/1997449064 conn(0x7fd92c077520 msgr2=0x7fd92c077920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:06.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.603+0000 7fd930bdf700 1 -- 172.21.15.31:0/1997449064 shutdown_connections 2023-12-14T18:30:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.603+0000 7f4c28934700 1 -- 172.21.15.31:0/3248324465 shutdown_connections 2023-12-14T18:30:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.604+0000 7f4c28934700 1 -- 172.21.15.31:0/3248324465 wait complete. 2023-12-14T18:30:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.604+0000 7fd930bdf700 1 -- 172.21.15.31:0/1997449064 wait complete. 2023-12-14T18:30:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 Processor -- start 2023-12-14T18:30:06.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 Processor -- start 2023-12-14T18:30:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 -- start start 2023-12-14T18:30:06.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 -- start start 2023-12-14T18:30:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c24102460 0x7f4c24198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c24102e10 0x7f4c24198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 0x7fd92c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd92c101390 0x7fd92c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd92c1051b0 0x7fd92c19bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c19c580 con 0x7fd92c1009e0 2023-12-14T18:30:06.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c19c6f0 con 0x7fd92c101390 2023-12-14T18:30:06.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd930bdf700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c19c8b0 con 0x7fd92c1051b0 2023-12-14T18:30:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c2419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.726 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2419d560 con 0x7f4c24102460 2023-12-14T18:30:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2419d6d0 con 0x7f4c24102e10 2023-12-14T18:30:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c28934700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2419d890 con 0x7f4c24108470 2023-12-14T18:30:06.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd92a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 0x7fd92c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c24102460 0x7f4c24198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd92a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 0x7fd92c198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42636/0 (socket says 172.21.15.31:42636) 2023-12-14T18:30:06.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd92a59c700 1 -- 172.21.15.31:0/2658171831 learned_addr learned my addr 172.21.15.31:0/2658171831 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.722+0000 7f4c22d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c2419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.722+0000 7fd92ad9d700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd92c1051b0 0x7fd92c19bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c21d9b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c24102e10 0x7f4c24198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c2419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:40694/0 (socket says 172.21.15.31:40694) 2023-12-14T18:30:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 -- 172.21.15.31:0/3610127629 learned_addr learned my addr 172.21.15.31:0/3610127629 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:06.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd929d9b700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd92c101390 0x7fd92c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd92a59c700 1 -- 172.21.15.31:0/2658171831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd92c1051b0 msgr2=0x7fd92c19bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:06.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd92a59c700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd92c1051b0 0x7fd92c19bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd92a59c700 1 -- 172.21.15.31:0/2658171831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd92c101390 msgr2=0x7fd92c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:06.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd92a59c700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd92c101390 0x7fd92c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 -- 172.21.15.31:0/3610127629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c24102e10 msgr2=0x7f4c24198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c24102e10 0x7f4c24198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 -- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c24102460 msgr2=0x7f4c24198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:06.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd92a59c700 1 -- 172.21.15.31:0/2658171831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920009520 con 0x7fd92c1009e0 2023-12-14T18:30:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c24102460 0x7f4c24198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 -- 172.21.15.31:0/3610127629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c180095f0 con 0x7f4c24108470 2023-12-14T18:30:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.723+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c2419cf20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ef30 tx=0x7f4c1c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:06.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.723+0000 7fd92a59c700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 0x7fd92c198690 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00e9c0 tx=0x7fd91c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:06.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.724+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004a30 con 0x7fd92c1009e0 2023-12-14T18:30:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.724+0000 7f4c137fe700 1 -- 172.21.15.31:0/3610127629 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c018070 con 0x7f4c24108470 2023-12-14T18:30:06.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.724+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd91c00cc80 con 0x7fd92c1009e0 2023-12-14T18:30:06.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.724+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd924009c40 con 0x7fd92c1009e0 2023-12-14T18:30:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.724+0000 7f4c137fe700 1 -- 172.21.15.31:0/3610127629 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1c00f040 con 0x7f4c24108470 2023-12-14T18:30:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.724+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c14009520 con 0x7f4c24108470 2023-12-14T18:30:06.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.724+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0054e0 con 0x7fd92c1009e0 2023-12-14T18:30:06.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.724+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c110320 con 0x7fd92c1009e0 2023-12-14T18:30:06.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.725+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c110190 con 0x7fd92c1009e0 2023-12-14T18:30:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.724+0000 7f4c137fe700 1 -- 172.21.15.31:0/3610127629 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c00cd00 con 0x7f4c24108470 2023-12-14T18:30:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.724+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c24112460 con 0x7f4c24108470 2023-12-14T18:30:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.726+0000 7f4c137fe700 1 -- 172.21.15.31:0/3610127629 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4c1c003a00 con 0x7f4c24108470 2023-12-14T18:30:06.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.727+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd91c004b90 con 0x7fd92c1009e0 2023-12-14T18:30:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.727+0000 7f4c137fe700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4c0c0a1340 0x7f4c0c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.727+0000 7f4c2259c700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4c0c0a1340 0x7f4c0c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.727+0000 7f4c137fe700 1 -- 172.21.15.31:0/3610127629 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4c1c0ccb20 con 0x7f4c24108470 2023-12-14T18:30:06.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.728+0000 7fd91b7fe700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9140a12c0 0x7fd9140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:06.728+0000 7f4c2259c700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4c0c0a1340 0x7f4c0c0a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009fd0 tx=0x7f4c14019040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:06.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.728+0000 7fd929d9b700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9140a12c0 0x7fd9140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:06.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.728+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd91c014070 con 0x7fd92c1009e0 2023-12-14T18:30:06.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.729+0000 7fd929d9b700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9140a12c0 0x7fd9140a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd92c199ca0 tx=0x7fd920009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:06.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.732+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd91c096360 con 0x7fd92c1009e0 2023-12-14T18:30:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.998+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd92c1104b0 con 0x7fd92c1009e0 2023-12-14T18:30:07.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:06.999+0000 7fd91b7fe700 1 -- 172.21.15.31:0/2658171831 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10813 (secure 0 0 0) 0x7fd91c099980 con 0x7fd92c1009e0 2023-12-14T18:30:07.000 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:30:07.000 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:26:22.578285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:30:07.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9140a12c0 msgr2=0x7fd9140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9140a12c0 0x7fd9140a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd92c199ca0 tx=0x7fd920009260 comp rx=0 tx=0).stop 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 msgr2=0x7fd92c198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 0x7fd92c198690 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00e9c0 tx=0x7fd91c00ecd0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 shutdown_connections 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9140a12c0 0x7fd9140a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd92c1051b0 0x7fd92c19bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd92c1009e0 0x7fd92c198690 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 --2- 172.21.15.31:0/2658171831 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd92c101390 0x7fd92c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 >> 172.21.15.31:0/2658171831 conn(0x7fd92c077520 msgr2=0x7fd92c103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:07.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 shutdown_connections 2023-12-14T18:30:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:07.001+0000 7fd930bdf700 1 -- 172.21.15.31:0/2658171831 wait complete. 2023-12-14T18:30:07.004 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2023-12-14T18:30:07.004 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:07.004 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:07.022 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:07.022 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:07.022 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 89 -lt 100 ']' 2023-12-14T18:30:07.022 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 89 + 1 2023-12-14T18:30:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.028+0000 7f4c34ac2b80 1 Processor -- start 2023-12-14T18:30:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.028+0000 7f4c34ac2b80 1 -- start start 2023-12-14T18:30:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458beb5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458beb67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458bec10f0 0x56458bec34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bda0580 con 0x56458bd13430 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458be0f810 con 0x56458bec10f0 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458be08ca0 con 0x56458beb6370 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458beb5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458beb5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42654/0 (socket says 172.21.15.31:42654) 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 -- 172.21.15.31:0/1006747946 learned_addr learned my addr 172.21.15.31:0/1006747946 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458beb67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458bec10f0 0x56458bec34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 -- 172.21.15.31:0/1006747946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 msgr2=0x56458beb67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458beb67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 -- 172.21.15.31:0/1006747946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458bec10f0 msgr2=0x56458bec34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458bec10f0 0x56458bec34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.029+0000 7f4c2259c700 1 -- 172.21.15.31:0/1006747946 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458bc8a350 con 0x56458bd13430 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.030+0000 7f4c2259c700 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458beb5e30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4c14016000 tx=0x7f4c1401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171342d186c2de61 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.030+0000 7f4c10ff9700 1 -- 172.21.15.31:0/1006747946 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400b760 con 0x56458bd13430 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.030+0000 7f4c10ff9700 1 -- 172.21.15.31:0/1006747946 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1401d040 con 0x56458bd13430 2023-12-14T18:30:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.030+0000 7f4c10ff9700 1 -- 172.21.15.31:0/1006747946 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1402f6e0 con 0x56458bd13430 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/1006747946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 msgr2=0x56458beb5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458beb5e30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4c14016000 tx=0x7f4c1401e000 comp rx=0 tx=0).stop 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/1006747946 shutdown_connections 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458bec10f0 0x56458bec34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458beb5e30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/1006747946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458beb67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/1006747946 >> 172.21.15.31:0/1006747946 conn(0x56458beac890 msgr2=0x56458beb2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/1006747946 shutdown_connections 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/1006747946 wait complete. 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 -- start start 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 Processor -- start 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.031+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bebdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 0x56458bebe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bda4f30 con 0x56458bec10f0 2023-12-14T18:30:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bda4bd0 con 0x56458beb6370 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bc8c640 con 0x56458bd13430 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c34ac2b80 10 client.0 _ll_register_callbacks cb 0x56458bdc88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 0x56458bebe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 0x56458bebe300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42656/0 (socket says 172.21.15.31:42656) 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 -- 172.21.15.31:0/753579055 learned_addr learned my addr 172.21.15.31:0/753579055 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c2259c700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bebdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 -- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 msgr2=0x56458bec0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 -- 172.21.15.31:0/753579055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 msgr2=0x56458bebdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bebdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 -- 172.21.15.31:0/753579055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2410fba0 con 0x56458bec10f0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bebdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 0x56458bebe300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c099880 tx=0x7f4c1c0992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93889aab0ca4994e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.032+0000 7f4c09ffb700 1 -- 172.21.15.31:0/753579055 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0998f0 con 0x56458bec10f0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c09ffb700 1 -- 172.21.15.31:0/753579055 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4c1c09d920 con 0x56458bec10f0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56458bea9ea0 con 0x56458bec10f0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c09ffb700 1 -- 172.21.15.31:0/753579055 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c1c0a1cb0 con 0x56458bec10f0 2023-12-14T18:30:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 10 client.16227 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56458bdcf110 con 0x56458bec10f0 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c09ffb700 1 -- 172.21.15.31:0/753579055 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56458bea9ea0 con 0x56458bec10f0 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c09ffb700 1 -- 172.21.15.31:0/753579055 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c1c0b1ef0 con 0x56458bec10f0 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 10 client.16227 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 20 client.16227 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bed0990 0x56458bed2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 4 client.16227 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56458be01c80 con 0x56458bed0990 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.033+0000 7f4c2259c700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bed0990 0x56458bed2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.034+0000 7f4c09ffb700 10 client.16227 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.034+0000 7f4c2259c700 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bed0990 0x56458bed2da0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbec7b00554f9d99 server_cookie=c0b8fb7a5e2b7570 in_seq=0 out_seq=0 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.034+0000 7f4c09ffb700 1 -- 172.21.15.31:0/753579055 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56458be01c80 con 0x56458bed0990 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.034+0000 7f4c09ffb700 10 client.16227 handle_command_reply: tid=0 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 client.16227 shutdown 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 msgr2=0x56458bebe300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 0x56458bebe300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c099880 tx=0x7f4c1c0992d0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 shutdown_connections 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bed0990 0x56458bed2da0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bec10f0 0x56458bebe300 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c09ffb700 0 client.16227 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/753579055 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bebdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 >> 172.21.15.31:0/753579055 conn(0x56458beac890 msgr2=0x56458beb1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 shutdown_connections 2023-12-14T18:30:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/753579055 wait complete. 2023-12-14T18:30:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.035+0000 7f4c34ac2b80 20 client.16227 trim_cache size 0 max 16384 2023-12-14T18:30:07.049 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=90 2023-12-14T18:30:07.049 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 Processor -- start 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 -- start start 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 0x56458bebcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bec0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458becd500 con 0x56458beac890 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458beb78c0 con 0x56458beb6370 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458beb6740 con 0x56458bd13430 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 0x56458bebcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 0x56458bebcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42668/0 (socket says 172.21.15.31:42668) 2023-12-14T18:30:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c2259c700 1 -- 172.21.15.31:0/4025838333 learned_addr learned my addr 172.21.15.31:0/4025838333 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.051+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bec0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c2259c700 1 -- 172.21.15.31:0/4025838333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 msgr2=0x56458bec0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c2259c700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c2259c700 1 -- 172.21.15.31:0/4025838333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 msgr2=0x56458bec0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c2259c700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bec0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c2259c700 1 -- 172.21.15.31:0/4025838333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458beae0d0 con 0x56458beac890 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bec0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c2259c700 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 0x56458bebcc20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009810 tx=0x7f4c1400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7266a9a81a7fc027 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c117fa700 1 -- 172.21.15.31:0/4025838333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14034040 con 0x56458beac890 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c117fa700 1 -- 172.21.15.31:0/4025838333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c140050c0 con 0x56458beac890 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.052+0000 7f4c117fa700 1 -- 172.21.15.31:0/4025838333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14038780 con 0x56458beac890 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/4025838333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 msgr2=0x56458bebcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 0x56458bebcc20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009810 tx=0x7f4c1400b260 comp rx=0 tx=0).stop 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/4025838333 shutdown_connections 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beb6370 0x56458bec0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458beac890 0x56458bebcc20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/4025838333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458bd13430 0x56458bec0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/4025838333 >> 172.21.15.31:0/4025838333 conn(0x56458becd130 msgr2=0x56458bebc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/4025838333 shutdown_connections 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/4025838333 wait complete. 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 -- start start 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 Processor -- start 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458bec7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beac890 0x56458bec7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.053+0000 7f4c34ac2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458bec5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bec6170 con 0x56458bd13430 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bec62e0 con 0x56458beac890 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c34ac2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56458bec64a0 con 0x56458beb6370 2023-12-14T18:30:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c34ac2b80 10 client.0 _ll_register_callbacks cb 0x7f4c1c099770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458bec7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458bec7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42678/0 (socket says 172.21.15.31:42678) 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 -- 172.21.15.31:0/893668394 learned_addr learned my addr 172.21.15.31:0/893668394 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beac890 0x56458bec7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c22d9d700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458bec5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 -- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 msgr2=0x56458bec5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458bec5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 -- 172.21.15.31:0/893668394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beac890 msgr2=0x56458bec7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beac890 0x56458bec7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 -- 172.21.15.31:0/893668394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56458bc8a350 con 0x56458bd13430 2023-12-14T18:30:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beac890 0x56458bec7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c2259c700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458bec7640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4c140383c0 tx=0x7f4c140383f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78018440771eff7c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c09ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.054+0000 7f4c09ffb700 1 -- 172.21.15.31:0/893668394 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14034040 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c1401d040 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c09ffb700 1 -- 172.21.15.31:0/893668394 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f4c1400b970 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c09ffb700 1 -- 172.21.15.31:0/893668394 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c1400bb30 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 10 client.16233 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56458bc36b00 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c09ffb700 1 -- 172.21.15.31:0/893668394 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c1401d040 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c09ffb700 1 -- 172.21.15.31:0/893668394 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c140388c0 con 0x56458bd13430 2023-12-14T18:30:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 20 client.16233 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bec10f0 0x56458bf27660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 4 client.16233 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.055+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56458be019a0 con 0x56458bec10f0 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.056+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bec10f0 0x56458bf27660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.056+0000 7f4c09ffb700 10 client.16233 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.056+0000 7f4c21d9b700 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bec10f0 0x56458bf27660 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113662294df3efce server_cookie=9f2ecbf975ea1172 in_seq=0 out_seq=0 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c09ffb700 1 -- 172.21.15.31:0/893668394 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56458be019a0 con 0x56458bec10f0 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c09ffb700 10 client.16233 handle_command_reply: tid=0 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 client.16233 shutdown 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 msgr2=0x56458bec7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458bec7640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4c140383c0 tx=0x7f4c140383f0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 shutdown_connections 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56458bec10f0 0x56458bf27660 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56458beb6370 0x56458bec5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c09ffb700 0 client.16233 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56458bd13430 0x56458bec7640 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 --2- 172.21.15.31:0/893668394 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56458beac890 0x56458bec7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 >> 172.21.15.31:0/893668394 conn(0x56458becd130 msgr2=0x56458bebf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 shutdown_connections 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 1 -- 172.21.15.31:0/893668394 wait complete. 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.078+0000 7f4c34ac2b80 20 client.16233 trim_cache size 0 max 16384 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:30:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:30:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:30:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4c0c0a1340 msgr2=0x7f4c0c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4c0c0a1340 0x7f4c0c0a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009fd0 tx=0x7f4c14019040 comp rx=0 tx=0).stop 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 msgr2=0x7f4c2419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c2419cf20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00ef30 tx=0x7f4c1c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 shutdown_connections 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4c0c0a1340 0x7f4c0c0a37f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c24108470 0x7f4c2419cf20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4c24102460 0x7f4c24198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 --2- 172.21.15.31:0/3610127629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c24102e10 0x7f4c24198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 >> 172.21.15.31:0/3610127629 conn(0x7f4c240fdf60 msgr2=0x7f4c240fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 shutdown_connections 2023-12-14T18:30:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:07.079+0000 7f4c28934700 1 -- 172.21.15.31:0/3610127629 wait complete. 2023-12-14T18:30:07.090 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:07.090 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:07.091 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:07.091 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:07.091 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:07.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:30:07.717 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:07.717 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:07.814 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:07.814 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:07.854 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:07.854 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:07.854 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 90 -lt 100 ']' 2023-12-14T18:30:07.854 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 90 + 1 2023-12-14T18:30:07.915 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=91 2023-12-14T18:30:07.915 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:07.941 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:07.941 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:07.942 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:07.943 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:07.943 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:07 smithi031 ceph-mon[42085]: pgmap v72: 129 pgs: 129 active+clean; 2.7 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 21 op/s 2023-12-14T18:30:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2658171831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:30:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:07 smithi176 ceph-mon[43506]: pgmap v72: 129 pgs: 129 active+clean; 2.7 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 21 op/s 2023-12-14T18:30:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2658171831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:30:08.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:07 smithi204 ceph-mon[43866]: pgmap v72: 129 pgs: 129 active+clean; 2.7 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 21 op/s 2023-12-14T18:30:08.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2658171831' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:30:08.516 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:08.516 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:08.658 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:08.658 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:08.671 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:08.671 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:08.671 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 91 -lt 100 ']' 2023-12-14T18:30:08.671 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 91 + 1 2023-12-14T18:30:08.720 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=92 2023-12-14T18:30:08.720 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:08.733 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:08.734 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:08.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:08.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:08.735 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:08 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:30:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:08 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:30:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:08 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:30:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:08 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:30:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:08 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:30:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:08 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:30:09.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:08 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:30:09.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:08 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:30:09.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:08 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:30:09.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:09 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:30:09.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:09 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:30:09.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:09 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:30:09.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:09 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:30:09.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:09 smithi031 ceph-mon[42085]: pgmap v73: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-12-14T18:30:09.974 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:09.974 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:09 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:30:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:09 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:30:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:09 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:30:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:09 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:30:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:09 smithi176 ceph-mon[43506]: pgmap v73: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-12-14T18:30:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:09 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:30:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:09 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:30:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:09 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:30:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:09 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:30:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:09 smithi204 ceph-mon[43866]: pgmap v73: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 55 KiB/s wr, 29 op/s 2023-12-14T18:30:10.228 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:10.228 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:10.238 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:10.238 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:10.238 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 92 -lt 100 ']' 2023-12-14T18:30:10.238 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 92 + 1 2023-12-14T18:30:10.273 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=93 2023-12-14T18:30:10.273 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:10.320 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:10.320 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:10.321 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:10.321 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:10.322 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 -- 172.21.15.31:0/339492521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160104d70 msgr2=0x7f916010b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 --2- 172.21.15.31:0/339492521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160104d70 0x7f916010b750 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f915800b0d0 tx=0x7f915800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 -- 172.21.15.31:0/339492521 shutdown_connections 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 --2- 172.21.15.31:0/339492521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160104d70 0x7f916010b750 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 --2- 172.21.15.31:0/339492521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91600ff6e0 0x7f91600ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 --2- 172.21.15.31:0/339492521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 0x7f9160104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 -- 172.21.15.31:0/339492521 >> 172.21.15.31:0/339492521 conn(0x7f9160074a60 msgr2=0x7f9160074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 -- 172.21.15.31:0/339492521 shutdown_connections 2023-12-14T18:30:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.589+0000 7f9167ec8700 1 -- 172.21.15.31:0/339492521 wait complete. 2023-12-14T18:30:10.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 Processor -- start 2023-12-14T18:30:10.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 -- start start 2023-12-14T18:30:10.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 0x7f9160196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 0x7f9160196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9160104d70 0x7f916019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916019b400 con 0x7f91600ff6e0 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916019b570 con 0x7f9160100080 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9167ec8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916019b730 con 0x7f9160104d70 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9165c64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 0x7f9160196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9165c64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 0x7f9160196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42692/0 (socket says 172.21.15.31:42692) 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.590+0000 7f9165c64700 1 -- 172.21.15.31:0/4107193882 learned_addr learned my addr 172.21.15.31:0/4107193882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9166465700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9160104d70 0x7f916019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165463700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 0x7f9160196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165c64700 1 -- 172.21.15.31:0/4107193882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9160104d70 msgr2=0x7f916019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165c64700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9160104d70 0x7f916019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165c64700 1 -- 172.21.15.31:0/4107193882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 msgr2=0x7f9160196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165c64700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 0x7f9160196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165c64700 1 -- 172.21.15.31:0/4107193882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009d00 con 0x7f91600ff6e0 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165463700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 0x7f9160196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9166465700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9160104d70 0x7f916019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9165c64700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 0x7f9160196420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f915c00eab0 tx=0x7f915c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9156ffd700 1 -- 172.21.15.31:0/4107193882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00cb90 con 0x7f91600ff6e0 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9156ffd700 1 -- 172.21.15.31:0/4107193882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f915c00ccf0 con 0x7f91600ff6e0 2023-12-14T18:30:10.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916010e110 con 0x7f91600ff6e0 2023-12-14T18:30:10.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9156ffd700 1 -- 172.21.15.31:0/4107193882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00b690 con 0x7f91600ff6e0 2023-12-14T18:30:10.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.591+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916010df80 con 0x7f91600ff6e0 2023-12-14T18:30:10.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.594+0000 7f9156ffd700 1 -- 172.21.15.31:0/4107193882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f915c016410 con 0x7f91600ff6e0 2023-12-14T18:30:10.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.595+0000 7f9156ffd700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f914c0a12c0 0x7f914c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.595+0000 7f9156ffd700 1 -- 172.21.15.31:0/4107193882 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f915c0cce70 con 0x7f91600ff6e0 2023-12-14T18:30:10.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.595+0000 7f9165463700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f914c0a12c0 0x7f914c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.596+0000 7f9165463700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f914c0a12c0 0x7f914c0a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9160197a30 tx=0x7f915000b5f0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:10.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.649+0000 7f9174056b80 1 Processor -- start 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 -- start start 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dcd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089ddaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089cb75e0 con 0x55e089dcdc90 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089d26870 con 0x55e089dcd380 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9174056b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089d1fd00 con 0x55e089dd8b50 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9165463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9165463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42694/0 (socket says 172.21.15.31:42694) 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9165c64700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dcd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9165463700 1 -- 172.21.15.31:0/4228401221 learned_addr learned my addr 172.21.15.31:0/4228401221 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.650+0000 7f9166465700 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089ddaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9165c64700 1 -- 172.21.15.31:0/4228401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 msgr2=0x55e089ddaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9165c64700 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089ddaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9165c64700 1 -- 172.21.15.31:0/4228401221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 msgr2=0x55e089dd8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9165c64700 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9165c64700 1 -- 172.21.15.31:0/4228401221 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e089ba13b0 con 0x55e089dcd380 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9166465700 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089ddaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f9165c64700 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dcd750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f915c0ca230 tx=0x7f915c099510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef29e84013f9a9f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f914bfff700 1 -- 172.21.15.31:0/4228401221 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c09dbe0 con 0x55e089dcd380 2023-12-14T18:30:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f914bfff700 1 -- 172.21.15.31:0/4228401221 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f915c0a2000 con 0x55e089dcd380 2023-12-14T18:30:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.651+0000 7f914bfff700 1 -- 172.21.15.31:0/4228401221 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c09a070 con 0x55e089dcd380 2023-12-14T18:30:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 -- 172.21.15.31:0/4228401221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 msgr2=0x55e089dcd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dcd750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f915c0ca230 tx=0x7f915c099510 comp rx=0 tx=0).stop 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 -- 172.21.15.31:0/4228401221 shutdown_connections 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089ddaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dcd750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 --2- 172.21.15.31:0/4228401221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 -- 172.21.15.31:0/4228401221 >> 172.21.15.31:0/4228401221 conn(0x55e089dc3ca0 msgr2=0x55e089dc97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.652+0000 7f9174056b80 1 -- 172.21.15.31:0/4228401221 shutdown_connections 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.653+0000 7f9174056b80 1 -- 172.21.15.31:0/4228401221 wait complete. 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.653+0000 7f9174056b80 1 -- start start 2023-12-14T18:30:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.653+0000 7f9174056b80 1 Processor -- start 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dd8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089dd6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089cbbf90 con 0x55e089dcdc90 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089cbbc30 con 0x55e089dcd380 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089ba36a0 con 0x55e089dd8b50 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9174056b80 10 client.0 _ll_register_callbacks cb 0x55e089cdf910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9165463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9165463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42708/0 (socket says 172.21.15.31:42708) 2023-12-14T18:30:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9165463700 1 -- 172.21.15.31:0/1399984009 learned_addr learned my addr 172.21.15.31:0/1399984009 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9166465700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089dd6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.654+0000 7f9165c64700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dd8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165463700 1 -- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 msgr2=0x55e089dd6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165463700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089dd6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165463700 1 -- 172.21.15.31:0/1399984009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 msgr2=0x55e089dd8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165463700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dd8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165463700 1 -- 172.21.15.31:0/1399984009 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916010df80 con 0x55e089dcdc90 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165c64700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dd8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9165463700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd5920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f915000ffe0 tx=0x7f9150004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ace442bcc839c76 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:10.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f91497fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f91497fa700 1 -- 172.21.15.31:0/1399984009 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150004220 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f91497fa700 1 -- 172.21.15.31:0/1399984009 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9150004380 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.655+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e089dc12b0 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f91497fa700 1 -- 172.21.15.31:0/1399984009 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9150011e50 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 10 client.16245 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e089ce6170 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f91497fa700 1 -- 172.21.15.31:0/1399984009 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e089dc12b0 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f91497fa700 1 -- 172.21.15.31:0/1399984009 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f915001f6a0 con 0x55e089dcdc90 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 10 client.16245 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:10.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 20 client.16245 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089de8580 0x55e089dea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 4 client.16245 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e089d18ce0 con 0x55e089de8580 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.656+0000 7f9165c64700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089de8580 0x55e089dea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.657+0000 7f91497fa700 10 client.16245 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.657+0000 7f9165c64700 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089de8580 0x55e089dea990 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48879ed95d7e46b8 server_cookie=51670497bf227609 in_seq=0 out_seq=0 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f91497fa700 1 -- 172.21.15.31:0/1399984009 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e089d18ce0 con 0x55e089de8580 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f91497fa700 10 client.16245 handle_command_reply: tid=0 2023-12-14T18:30:10.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 client.16245 shutdown 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 msgr2=0x55e089dd5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd5920 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f915000ffe0 tx=0x7f9150004750 comp rx=0 tx=0).stop 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 shutdown_connections 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089de8580 0x55e089dea990 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089dd6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcd380 0x55e089dd8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 --2- 172.21.15.31:0/1399984009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd5920 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f91497fa700 0 client.16245 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 >> 172.21.15.31:0/1399984009 conn(0x55e089dc3ca0 msgr2=0x55e089dc93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 shutdown_connections 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 1 -- 172.21.15.31:0/1399984009 wait complete. 2023-12-14T18:30:10.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.658+0000 7f9174056b80 20 client.16245 trim_cache size 0 max 16384 2023-12-14T18:30:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 Processor -- start 2023-12-14T18:30:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 -- start start 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dcd380 0x55e089ddf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dd8b50 0x55e089dd1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089dce330 con 0x55e089dcdc90 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089de44e0 con 0x55e089dd8b50 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9174056b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089dd1770 con 0x55e089dcd380 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9165463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9165463700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42718/0 (socket says 172.21.15.31:42718) 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9165463700 1 -- 172.21.15.31:0/881139798 learned_addr learned my addr 172.21.15.31:0/881139798 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9165c64700 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dcd380 0x55e089ddf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.673+0000 7f9166465700 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dd8b50 0x55e089dd1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9165463700 1 -- 172.21.15.31:0/881139798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dcd380 msgr2=0x55e089ddf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9165463700 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dcd380 0x55e089ddf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9165463700 1 -- 172.21.15.31:0/881139798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dd8b50 msgr2=0x55e089dd1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9165463700 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dd8b50 0x55e089dd1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9165463700 1 -- 172.21.15.31:0/881139798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e089ce5ec0 con 0x55e089dcdc90 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9165463700 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd0c10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f915001fab0 tx=0x7f915000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a6bba39639fe5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9154ff9700 1 -- 172.21.15.31:0/881139798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150011a00 con 0x55e089dcdc90 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9154ff9700 1 -- 172.21.15.31:0/881139798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9150011e50 con 0x55e089dcdc90 2023-12-14T18:30:10.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.674+0000 7f9154ff9700 1 -- 172.21.15.31:0/881139798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150003680 con 0x55e089dcdc90 2023-12-14T18:30:10.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 -- 172.21.15.31:0/881139798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 msgr2=0x55e089dd0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd0c10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f915001fab0 tx=0x7f915000b840 comp rx=0 tx=0).stop 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 -- 172.21.15.31:0/881139798 shutdown_connections 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dd8b50 0x55e089dd1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dcd380 0x55e089ddf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 --2- 172.21.15.31:0/881139798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcdc90 0x55e089dd0c10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 -- 172.21.15.31:0/881139798 >> 172.21.15.31:0/881139798 conn(0x55e089dc3ca0 msgr2=0x55e089de3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 -- 172.21.15.31:0/881139798 shutdown_connections 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 -- 172.21.15.31:0/881139798 wait complete. 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 -- start start 2023-12-14T18:30:10.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.675+0000 7f9174056b80 1 Processor -- start 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 0x55e089ddd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcdc90 0x55e089ddd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089e3b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089dddec0 con 0x55e089dcd380 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089e3b980 con 0x55e089dcdc90 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e089e3bac0 con 0x55e089dd8b50 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9174056b80 10 client.0 _ll_register_callbacks cb 0x7f915000fb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9165c64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 0x55e089ddd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9165c64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 0x55e089ddd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42724/0 (socket says 172.21.15.31:42724) 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9165c64700 1 -- 172.21.15.31:0/1064369153 learned_addr learned my addr 172.21.15.31:0/1064369153 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:10.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9166465700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089e3b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9165463700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcdc90 0x55e089ddd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9165c64700 1 -- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 msgr2=0x55e089e3b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.676+0000 7f9165c64700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089e3b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9165c64700 1 -- 172.21.15.31:0/1064369153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcdc90 msgr2=0x55e089ddd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9165c64700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcdc90 0x55e089ddd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9165c64700 1 -- 172.21.15.31:0/1064369153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e089ce6170 con 0x55e089dcd380 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9165c64700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 0x55e089ddd070 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c6b0 tx=0x7f915c016d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eceb3484d7fa65d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f91497fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f91497fa700 1 -- 172.21.15.31:0/1064369153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c0166c0 con 0x55e089dcd380 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f91497fa700 1 -- 172.21.15.31:0/1064369153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f915c016960 con 0x55e089dcd380 2023-12-14T18:30:10.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f915c0a2000 con 0x55e089dcd380 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f91497fa700 1 -- 172.21.15.31:0/1064369153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f915c0b2230 con 0x55e089dcd380 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9174056b80 10 client.16251 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e089ccc9a0 con 0x55e089dcd380 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.677+0000 7f91497fa700 1 -- 172.21.15.31:0/1064369153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f915c0a2000 con 0x55e089dcd380 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f91497fa700 1 -- 172.21.15.31:0/1064369153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f915c0a1b70 con 0x55e089dcd380 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9174056b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9174056b80 10 client.16251 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9174056b80 20 client.16251 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9174056b80 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089e3ceb0 0x55e089e3f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9174056b80 4 client.16251 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:30:10.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e089d18a00 con 0x55e089e3ceb0 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.678+0000 7f9165463700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089e3ceb0 0x55e089e3f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f91497fa700 10 client.16251 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9165463700 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089e3ceb0 0x55e089e3f2c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfe9e9064bdf1523 server_cookie=4a4fab02bf8b090b in_seq=0 out_seq=0 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f91497fa700 1 -- 172.21.15.31:0/1064369153 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e089d18a00 con 0x55e089e3ceb0 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f91497fa700 10 client.16251 handle_command_reply: tid=0 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 client.16251 shutdown 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 msgr2=0x55e089ddd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 0x55e089ddd070 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c6b0 tx=0x7f915c016d30 comp rx=0 tx=0).stop 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 shutdown_connections 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55e089e3ceb0 0x55e089e3f2c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e089dd8b50 0x55e089e3b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e089dcd380 0x55e089ddd070 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f91497fa700 0 client.16251 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 --2- 172.21.15.31:0/1064369153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e089dcdc90 0x55e089ddd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 >> 172.21.15.31:0/1064369153 conn(0x55e089dc3ca0 msgr2=0x55e089dcf260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 shutdown_connections 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.679+0000 7f9174056b80 1 -- 172.21.15.31:0/1064369153 wait complete. 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9174056b80 20 client.16251 trim_cache size 0 max 16384 2023-12-14T18:30:10.688 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:30:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f914c0a12c0 msgr2=0x7f914c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f914c0a12c0 0x7f914c0a3770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9160197a30 tx=0x7f915000b5f0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 msgr2=0x7f9160196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 0x7f9160196420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f915c00eab0 tx=0x7f915c00ee70 comp rx=0 tx=0).stop 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 shutdown_connections 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f914c0a12c0 0x7f914c0a3770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9160104d70 0x7f916019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91600ff6e0 0x7f9160196420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 --2- 172.21.15.31:0/4107193882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9160100080 0x7f9160196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 >> 172.21.15.31:0/4107193882 conn(0x7f9160074a60 msgr2=0x7f91600fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 shutdown_connections 2023-12-14T18:30:10.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:10.680+0000 7f9167ec8700 1 -- 172.21.15.31:0/4107193882 wait complete. 2023-12-14T18:30:10.721 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:30:11.066 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:11.066 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:11.147 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:11.147 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:11.178 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:11.178 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:11.178 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 93 -lt 100 ']' 2023-12-14T18:30:11.179 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 93 + 1 2023-12-14T18:30:11.203 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=94 2023-12-14T18:30:11.203 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:11.214 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:11.215 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:11.216 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:11.216 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:11.216 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:11.607 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:11.607 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:11.681 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:11.682 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:11.696 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:11.696 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:11.696 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 94 -lt 100 ']' 2023-12-14T18:30:11.696 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 94 + 1 2023-12-14T18:30:11.761 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=95 2023-12-14T18:30:11.761 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:11.811 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:11.812 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:11.813 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:11.813 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:11.813 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:11 smithi031 ceph-mon[42085]: pgmap v74: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 26 op/s 2023-12-14T18:30:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:11 smithi176 ceph-mon[43506]: pgmap v74: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 26 op/s 2023-12-14T18:30:12.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:11 smithi204 ceph-mon[43866]: pgmap v74: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 26 op/s 2023-12-14T18:30:12.399 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:12.399 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:12.628 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:12.628 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:12.636 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:12.636 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:12.636 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 95 -lt 100 ']' 2023-12-14T18:30:12.636 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 95 + 1 2023-12-14T18:30:12.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=96 2023-12-14T18:30:12.661 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:12.714 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:12.714 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:12.715 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:12.715 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:12.715 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:13.257 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:13.257 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:13.479 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:13.479 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:13.487 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:13.488 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:13.488 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 96 -lt 100 ']' 2023-12-14T18:30:13.488 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 96 + 1 2023-12-14T18:30:13.544 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=97 2023-12-14T18:30:13.544 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:13.593 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:13.593 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:13.594 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:13.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:13.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 -- 172.21.15.31:0/3103128915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4105550 msgr2=0x7f04d410d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 --2- 172.21.15.31:0/3103128915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4105550 0x7f04d410d120 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00b0d0 tx=0x7f04cc00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 -- 172.21.15.31:0/3103128915 shutdown_connections 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 --2- 172.21.15.31:0/3103128915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4105550 0x7f04d410d120 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 --2- 172.21.15.31:0/3103128915 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 0x7f04d41008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 --2- 172.21.15.31:0/3103128915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4100eb0 0x7f04d4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 -- 172.21.15.31:0/3103128915 >> 172.21.15.31:0/3103128915 conn(0x7f04d40fbd80 msgr2=0x7f04d40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 -- 172.21.15.31:0/3103128915 shutdown_connections 2023-12-14T18:30:13.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.913+0000 7f04d90d5700 1 -- 172.21.15.31:0/3103128915 wait complete. 2023-12-14T18:30:13.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.914+0000 7f04d90d5700 1 Processor -- start 2023-12-14T18:30:13.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 -- start start 2023-12-14T18:30:13.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 0x7f04d4196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 0x7f04d4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4105550 0x7f04d419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d419b4f0 con 0x7f04d4100eb0 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d419b660 con 0x7f04d4100510 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d90d5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d419b820 con 0x7f04d4105550 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 0x7f04d4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d2d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 0x7f04d4196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 0x7f04d4196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42730/0 (socket says 172.21.15.31:42730) 2023-12-14T18:30:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d259c700 1 -- 172.21.15.31:0/2930204742 learned_addr learned my addr 172.21.15.31:0/2930204742 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.915+0000 7f04d359e700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4105550 0x7f04d419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d359e700 1 -- 172.21.15.31:0/2930204742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 msgr2=0x7f04d4196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d359e700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 0x7f04d4196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d359e700 1 -- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 msgr2=0x7f04d4196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d359e700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 0x7f04d4196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d359e700 1 -- 172.21.15.31:0/2930204742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc009d00 con 0x7f04d4105550 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 0x7f04d4196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d259c700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 0x7f04d4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d359e700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4105550 0x7f04d419aeb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009fd0 tx=0x7f04cc0077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04cbfff700 1 -- 172.21.15.31:0/2930204742 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00e040 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d4110370 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04cbfff700 1 -- 172.21.15.31:0/2930204742 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04cc00bcb0 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04cbfff700 1 -- 172.21.15.31:0/2930204742 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc005840 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.916+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d41101e0 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.919+0000 7f04cbfff700 1 -- 172.21.15.31:0/2930204742 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f04cc004a50 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.920+0000 7f04cbfff700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04c00a12c0 0x7f04c00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.920+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04c00a12c0 0x7f04c00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.920+0000 7f04cbfff700 1 -- 172.21.15.31:0/2930204742 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f04cc0cde80 con 0x7f04d4105550 2023-12-14T18:30:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.920+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04c00a12c0 0x7f04c00a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04c4011490 tx=0x7f04c4009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:13 smithi031 ceph-mon[42085]: pgmap v75: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 26 op/s 2023-12-14T18:30:13.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 Processor -- start 2023-12-14T18:30:13.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 -- start start 2023-12-14T18:30:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 0x555f2e1e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4c90 0x555f2e1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1f3f60 0x555f2e1f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e0ce5e0 con 0x555f2e1e4380 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e13d870 con 0x555f2e1e4c90 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.986+0000 7f04e5263b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e136d00 con 0x555f2e1f3f60 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 0x555f2e1e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4c90 0x555f2e1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 0x555f2e1e4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42740/0 (socket says 172.21.15.31:42740) 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d2d9d700 1 -- 172.21.15.31:0/1950098239 learned_addr learned my addr 172.21.15.31:0/1950098239 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d359e700 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1f3f60 0x555f2e1f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 -- 172.21.15.31:0/1950098239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1f3f60 msgr2=0x555f2e1f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1f3f60 0x555f2e1f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 -- 172.21.15.31:0/1950098239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 msgr2=0x555f2e1e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 0x555f2e1e4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 -- 172.21.15.31:0/1950098239 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f2dfb83b0 con 0x555f2e1e4c90 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 0x555f2e1e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.987+0000 7f04d259c700 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4c90 0x555f2e1f3a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0114c0 tx=0x7f04bc0117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50315dc0f14035a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.988+0000 7f04c97fa700 1 -- 172.21.15.31:0/1950098239 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc004230 con 0x555f2e1e4c90 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.988+0000 7f04c97fa700 1 -- 172.21.15.31:0/1950098239 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc004390 con 0x555f2e1e4c90 2023-12-14T18:30:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.988+0000 7f04c97fa700 1 -- 172.21.15.31:0/1950098239 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00e870 con 0x555f2e1e4c90 2023-12-14T18:30:13.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 -- 172.21.15.31:0/1950098239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4c90 msgr2=0x555f2e1f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:13.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4c90 0x555f2e1f3a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0114c0 tx=0x7f04bc0117d0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 -- 172.21.15.31:0/1950098239 shutdown_connections 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1f3f60 0x555f2e1f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4380 0x555f2e1e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1950098239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4c90 0x555f2e1f3a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 -- 172.21.15.31:0/1950098239 >> 172.21.15.31:0/1950098239 conn(0x555f2e1daca0 msgr2=0x555f2e1e07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 -- 172.21.15.31:0/1950098239 shutdown_connections 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 -- 172.21.15.31:0/1950098239 wait complete. 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.989+0000 7f04e5263b80 1 -- start start 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 Processor -- start 2023-12-14T18:30:13.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4380 0x555f2e1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4c90 0x555f2e1f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 0x555f2e1f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e0d2f90 con 0x555f2e1f3f60 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e0d2c30 con 0x555f2e1e4380 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2dfba6a0 con 0x555f2e1e4c90 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04e5263b80 10 client.0 _ll_register_callbacks cb 0x555f2e0f6910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04d359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 0x555f2e1f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04d259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4c90 0x555f2e1f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04d359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 0x555f2e1f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42748/0 (socket says 172.21.15.31:42748) 2023-12-14T18:30:13.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04d359e700 1 -- 172.21.15.31:0/1316595175 learned_addr learned my addr 172.21.15.31:0/1316595175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.990+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4380 0x555f2e1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d359e700 1 -- 172.21.15.31:0/1316595175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4c90 msgr2=0x555f2e1f0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d359e700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4c90 0x555f2e1f0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d359e700 1 -- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4380 msgr2=0x555f2e1f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d359e700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4380 0x555f2e1f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d359e700 1 -- 172.21.15.31:0/1316595175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc009c80 con 0x555f2e1f3f60 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4380 0x555f2e1f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04d359e700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 0x555f2e1f1250 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0970f0 tx=0x7f04cc0991a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1756f1b63cff3ab server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1316595175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04cc09a550 con 0x555f2e1f3f60 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1316595175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f04cc0ba040 con 0x555f2e1f3f60 2023-12-14T18:30:13.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f2e1d82b0 con 0x555f2e1f3f60 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1316595175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04cc00be20 con 0x555f2e1f3f60 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04e5263b80 10 client.16263 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.991+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555f2e0fd170 con 0x555f2e1f3f60 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1316595175 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f2e1d82b0 con 0x555f2e1f3f60 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1316595175 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04cc09af70 con 0x555f2e1f3f60 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04e5263b80 10 client.16263 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04e5263b80 10 client.16263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04e5263b80 20 client.16263 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e203620 0x555f2e205a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04e5263b80 4 client.16263 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:14.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f2e12fce0 con 0x555f2e203620 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.992+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e203620 0x555f2e205a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.993+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e203620 0x555f2e205a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29ca1f3382650d8d server_cookie=ed3e216b5f12edd in_seq=0 out_seq=0 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.993+0000 7f04ba7fc700 10 client.16263 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.993+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1316595175 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f2e12fce0 con 0x555f2e203620 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.993+0000 7f04ba7fc700 10 client.16263 handle_command_reply: tid=0 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 client.16263 shutdown 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 msgr2=0x555f2e1f1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 0x555f2e1f1250 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0970f0 tx=0x7f04cc0991a0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 shutdown_connections 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1f3f60 0x555f2e1f1250 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1e4380 0x555f2e1f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e203620 0x555f2e205a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1316595175 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4c90 0x555f2e1f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 >> 172.21.15.31:0/1316595175 conn(0x555f2e1daca0 msgr2=0x555f2e1e03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04ba7fc700 0 client.16263 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 shutdown_connections 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 1 -- 172.21.15.31:0/1316595175 wait complete. 2023-12-14T18:30:14.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:13.994+0000 7f04e5263b80 20 client.16263 trim_cache size 0 max 16384 2023-12-14T18:30:14.003 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:13 smithi176 ceph-mon[43506]: pgmap v75: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 26 op/s 2023-12-14T18:30:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 Processor -- start 2023-12-14T18:30:14.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 -- start start 2023-12-14T18:30:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e205630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e1f67e0 con 0x555f2e1e4c90 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e1ff950 con 0x555f2e1f3f60 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04e5263b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e1ef1a0 con 0x555f2e1e4380 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42754/0 (socket says 172.21.15.31:42754) 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.009+0000 7f04d259c700 1 -- 172.21.15.31:0/816121842 learned_addr learned my addr 172.21.15.31:0/816121842 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d359e700 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e205630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d259c700 1 -- 172.21.15.31:0/816121842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 msgr2=0x555f2e205630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d259c700 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e205630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d259c700 1 -- 172.21.15.31:0/816121842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 msgr2=0x555f2e1f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d259c700 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d259c700 1 -- 172.21.15.31:0/816121842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f2e1dc4e0 con 0x555f2e1e4c90 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04d259c700 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f33e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f04bc01b040 tx=0x7f04bc009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b4dd58338e5821 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04c9ffb700 1 -- 172.21.15.31:0/816121842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc005030 con 0x555f2e1e4c90 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04c9ffb700 1 -- 172.21.15.31:0/816121842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc005650 con 0x555f2e1e4c90 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.010+0000 7f04c9ffb700 1 -- 172.21.15.31:0/816121842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc004500 con 0x555f2e1e4c90 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 -- 172.21.15.31:0/816121842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 msgr2=0x555f2e1f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f33e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f04bc01b040 tx=0x7f04bc009440 comp rx=0 tx=0).stop 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 -- 172.21.15.31:0/816121842 shutdown_connections 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e205630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 --2- 172.21.15.31:0/816121842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f33e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 -- 172.21.15.31:0/816121842 >> 172.21.15.31:0/816121842 conn(0x555f2e1daca0 msgr2=0x555f2e1ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 -- 172.21.15.31:0/816121842 shutdown_connections 2023-12-14T18:30:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.011+0000 7f04e5263b80 1 -- 172.21.15.31:0/816121842 wait complete. 2023-12-14T18:30:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 -- start start 2023-12-14T18:30:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 Processor -- start 2023-12-14T18:30:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e256540 con 0x555f2e1e4c90 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e2566b0 con 0x555f2e1f3f60 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f2e256870 con 0x555f2e1e4380 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04e5263b80 10 client.0 _ll_register_callbacks cb 0x555f2e0f6910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.012+0000 7f04d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42762/0 (socket says 172.21.15.31:42762) 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 -- 172.21.15.31:0/1052770947 learned_addr learned my addr 172.21.15.31:0/1052770947 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d359e700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 -- 172.21.15.31:0/1052770947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 msgr2=0x555f2e1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:14.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 -- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 msgr2=0x555f2e1f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 -- 172.21.15.31:0/1052770947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f2dfb83b0 con 0x555f2e1e4c90 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04d259c700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f87c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893893e71e0c4e12 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04ba7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1052770947 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc003a90 con 0x555f2e1e4c90 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04bc004390 con 0x555f2e1e4c90 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.013+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1052770947 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f04bc0041c0 con 0x555f2e1e4c90 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1052770947 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04bc003bf0 con 0x555f2e1e4c90 2023-12-14T18:30:14.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 10 client.16269 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555f2df64b60 con 0x555f2e1e4c90 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1052770947 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04bc004390 con 0x555f2e1e4c90 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1052770947 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04bc02a020 con 0x555f2e1e4c90 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 10 client.16269 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 10 client.16269 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 20 client.16269 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e257c70 0x555f2e25a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 4 client.16269 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.014+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f2e12fa00 con 0x555f2e257c70 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e257c70 0x555f2e25a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04ba7fc700 10 client.16269 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:14.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04d2d9d700 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e257c70 0x555f2e25a080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7ebc1cf6e585da server_cookie=f72749fc0c3b267f in_seq=0 out_seq=0 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04ba7fc700 1 -- 172.21.15.31:0/1052770947 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f2e12fa00 con 0x555f2e257c70 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04ba7fc700 10 client.16269 handle_command_reply: tid=0 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 client.16269 shutdown 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 msgr2=0x555f2e1f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f87c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc005480 comp rx=0 tx=0).stop 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 shutdown_connections 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f2e1f3f60 0x555f2e1f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555f2e257c70 0x555f2e25a080 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f2e1e4380 0x555f2e1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 --2- 172.21.15.31:0/1052770947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f2e1e4c90 0x555f2e1f87c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04ba7fc700 0 client.16269 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:14.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.015+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 >> 172.21.15.31:0/1052770947 conn(0x555f2e1daca0 msgr2=0x555f2e205630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:14.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 shutdown_connections 2023-12-14T18:30:14.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04e5263b80 1 -- 172.21.15.31:0/1052770947 wait complete. 2023-12-14T18:30:14.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04e5263b80 20 client.16269 trim_cache size 0 max 16384 2023-12-14T18:30:14.024 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:30:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04c00a12c0 msgr2=0x7f04c00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04c00a12c0 0x7f04c00a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04c4011490 tx=0x7f04c4009260 comp rx=0 tx=0).stop 2023-12-14T18:30:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4105550 msgr2=0x7f04d419aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4105550 0x7f04d419aeb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009fd0 tx=0x7f04cc0077f0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 shutdown_connections 2023-12-14T18:30:14.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04c00a12c0 0x7f04c00a3770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04d4105550 0x7f04d419aeb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04d4100510 0x7f04d4196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 --2- 172.21.15.31:0/2930204742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4100eb0 0x7f04d4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:14.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.016+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 >> 172.21.15.31:0/2930204742 conn(0x7f04d40fbd80 msgr2=0x7f04d40fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:14.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.017+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 shutdown_connections 2023-12-14T18:30:14.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:14.017+0000 7f04d90d5700 1 -- 172.21.15.31:0/2930204742 wait complete. 2023-12-14T18:30:14.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:30:14.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:13 smithi204 ceph-mon[43866]: pgmap v75: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 50 KiB/s wr, 26 op/s 2023-12-14T18:30:14.373 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:14.373 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:14.479 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:14.479 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:14.528 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:14.528 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:14.528 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 97 -lt 100 ']' 2023-12-14T18:30:14.528 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 97 + 1 2023-12-14T18:30:14.560 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=98 2023-12-14T18:30:14.560 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:14.588 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:14.589 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:14.590 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:14.590 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:14.590 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:15.057 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:15.057 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:15.197 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:15.197 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:15.227 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:15.227 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:15.227 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 98 -lt 100 ']' 2023-12-14T18:30:15.227 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 98 + 1 2023-12-14T18:30:15.260 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=99 2023-12-14T18:30:15.261 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:15.328 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:15.328 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:15.329 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:15.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:15.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:15.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:15 smithi031 ceph-mon[42085]: pgmap v76: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 29 op/s 2023-12-14T18:30:15.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:15 smithi176 ceph-mon[43506]: pgmap v76: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 29 op/s 2023-12-14T18:30:16.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:15 smithi204 ceph-mon[43866]: pgmap v76: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 53 KiB/s wr, 29 op/s 2023-12-14T18:30:16.448 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:16.448 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:16.703 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:16.703 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:16.720 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:16.720 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:16.720 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 99 -lt 100 ']' 2023-12-14T18:30:16.720 INFO:tasks.workunit.client.0.smithi031.stderr:++ expr 99 + 1 2023-12-14T18:30:16.794 INFO:tasks.workunit.client.0.smithi031.stderr:+ c=100 2023-12-14T18:30:16.794 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:16.841 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:16.841 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:16.842 INFO:tasks.workunit.client.0.smithi031.stderr:+ c1=2 2023-12-14T18:30:16.842 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo 3 2023-12-14T18:30:16.842 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:30:17.423 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfacl d1 2023-12-14T18:30:17.423 INFO:tasks.workunit.client.0.smithi031.stderr:++ grep -c nobody:rw 2023-12-14T18:30:17.644 INFO:tasks.workunit.client.0.smithi031.stderr:+ c2=2 2023-12-14T18:30:17.644 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:17.652 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:17.652 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:30:17.652 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 100 -lt 100 ']' 2023-12-14T18:30:17.669 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir d1 2023-12-14T18:30:17.709 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-12-14T18:30:17.720 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-12-14T18:30:17.724 INFO:tasks.workunit.client.0.smithi031.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-12-14T18:30:17.724 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n system.posix_acl_access d1 2023-12-14T18:30:17.753 INFO:tasks.workunit.client.0.smithi031.stderr:d1: system.posix_acl_access: No such attribute 2023-12-14T18:30:17.753 INFO:tasks.workunit.client.0.smithi031.stderr:+ return 0 2023-12-14T18:30:17.753 INFO:tasks.workunit.client.0.smithi031.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-12-14T18:30:17.753 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n system.posix_acl_default . 2023-12-14T18:30:17.754 INFO:tasks.workunit.client.0.smithi031.stderr:.: system.posix_acl_default: No such attribute 2023-12-14T18:30:17.754 INFO:tasks.workunit.client.0.smithi031.stderr:+ return 0 2023-12-14T18:30:17.754 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir d1 2023-12-14T18:30:17.756 INFO:tasks.workunit.client.0.smithi031.stderr:+ cd .. 2023-12-14T18:30:17.757 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir testdir 2023-12-14T18:30:17.770 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo OK 2023-12-14T18:30:17.771 INFO:tasks.workunit.client.0.smithi031.stdout:OK 2023-12-14T18:30:17.772 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:30:17.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 -- 172.21.15.31:0/1218187982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d00076040 msgr2=0x7f2d000ff440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 --2- 172.21.15.31:0/1218187982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d00076040 0x7f2d000ff440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009990 tx=0x7f2cf0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 -- 172.21.15.31:0/1218187982 shutdown_connections 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 --2- 172.21.15.31:0/1218187982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d0010a860 0x7f2d0010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 --2- 172.21.15.31:0/1218187982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d00076040 0x7f2d000ff440 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 --2- 172.21.15.31:0/1218187982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 0x7f2d000ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 -- 172.21.15.31:0/1218187982 >> 172.21.15.31:0/1218187982 conn(0x7f2d000749f0 msgr2=0x7f2d00074df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 -- 172.21.15.31:0/1218187982 shutdown_connections 2023-12-14T18:30:17.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.954+0000 7f2d05887700 1 -- 172.21.15.31:0/1218187982 wait complete. 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.955+0000 7f2d05887700 1 Processor -- start 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 -- start start 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d00076040 0x7f2d001985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 0x7f2d00198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 0x7f2d0019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0019d580 con 0x7f2d0010a860 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0019d6f0 con 0x7f2d00076040 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2d05887700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0019d8b0 con 0x7f2d000ffa10 2023-12-14T18:30:17.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 0x7f2d0019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 0x7f2d0019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36696/0 (socket says 172.21.15.31:36696) 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2cff7fe700 1 -- 172.21.15.31:0/738713077 learned_addr learned my addr 172.21.15.31:0/738713077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 0x7f2d00198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.956+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d00076040 0x7f2d001985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/738713077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d00076040 msgr2=0x7f2d001985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d00076040 0x7f2d001985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 msgr2=0x7f2d0019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 0x7f2d0019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/738713077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00095f0 con 0x7f2d000ffa10 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 0x7f2d0019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:17.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 0x7f2d00198ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400e9c0 tx=0x7f2cf400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:17.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2d04885700 1 -- 172.21.15.31:0/738713077 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4004220 con 0x7f2d000ffa10 2023-12-14T18:30:17.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2d04885700 1 -- 172.21.15.31:0/738713077 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf4004380 con 0x7f2d000ffa10 2023-12-14T18:30:17.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d001102d0 con 0x7f2d000ffa10 2023-12-14T18:30:17.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2d04885700 1 -- 172.21.15.31:0/738713077 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4003be0 con 0x7f2d000ffa10 2023-12-14T18:30:17.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.957+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d00110140 con 0x7f2d000ffa10 2023-12-14T18:30:17.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.960+0000 7f2d04885700 1 -- 172.21.15.31:0/738713077 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2cf4003d40 con 0x7f2d000ffa10 2023-12-14T18:30:17.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.960+0000 7f2d04885700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ce80a12c0 0x7f2ce80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:17.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.961+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ce80a12c0 0x7f2ce80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:17.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.961+0000 7f2d04885700 1 -- 172.21.15.31:0/738713077 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2cf4014070 con 0x7f2d000ffa10 2023-12-14T18:30:17.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:17.961+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ce80a12c0 0x7f2ce80a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b2e0 tx=0x7f2cf0005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:17.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:17 smithi031 ceph-mon[42085]: pgmap v77: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-12-14T18:30:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:17 smithi176 ceph-mon[43506]: pgmap v77: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-12-14T18:30:18.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.039+0000 7f2d11a15b80 1 Processor -- start 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 -- start start 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955380 0x55fd98955750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98960610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd98962f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9883f5e0 con 0x55fd98960b50 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd988ae870 con 0x55fd98955380 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd988a7d00 con 0x55fd98955c90 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd98962f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd98962f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36706/0 (socket says 172.21.15.31:36706) 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2cff7fe700 1 -- 172.21.15.31:0/4204755418 learned_addr learned my addr 172.21.15.31:0/4204755418 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98960610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.040+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955380 0x55fd98955750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cff7fe700 1 -- 172.21.15.31:0/4204755418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 msgr2=0x55fd98960610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98960610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cff7fe700 1 -- 172.21.15.31:0/4204755418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955380 msgr2=0x55fd98955750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955380 0x55fd98955750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cff7fe700 1 -- 172.21.15.31:0/4204755418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd987293b0 con 0x55fd98960b50 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98960610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:18.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd98962f30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007530 tx=0x7f2cf8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231435917b4197d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:18.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2ce5ffb700 1 -- 172.21.15.31:0/4204755418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8009100 con 0x55fd98960b50 2023-12-14T18:30:18.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2ce5ffb700 1 -- 172.21.15.31:0/4204755418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf800f040 con 0x55fd98960b50 2023-12-14T18:30:18.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.041+0000 7f2ce5ffb700 1 -- 172.21.15.31:0/4204755418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8015e20 con 0x55fd98960b50 2023-12-14T18:30:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 -- 172.21.15.31:0/4204755418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 msgr2=0x55fd98962f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd98962f30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007530 tx=0x7f2cf8009920 comp rx=0 tx=0).stop 2023-12-14T18:30:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 -- 172.21.15.31:0/4204755418 shutdown_connections 2023-12-14T18:30:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd98962f30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955380 0x55fd98955750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/4204755418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98960610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 -- 172.21.15.31:0/4204755418 >> 172.21.15.31:0/4204755418 conn(0x55fd9894bca0 msgr2=0x55fd989517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 -- 172.21.15.31:0/4204755418 shutdown_connections 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.042+0000 7f2d11a15b80 1 -- 172.21.15.31:0/4204755418 wait complete. 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 -- start start 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 Processor -- start 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd9895d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd9895dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98843f90 con 0x55fd98960b50 2023-12-14T18:30:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.043+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98843c30 con 0x55fd98955c90 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9872b6a0 con 0x55fd98955380 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2d11a15b80 10 client.0 _ll_register_callbacks cb 0x55fd98867910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd9895dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cfeffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd9895dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36716/0 (socket says 172.21.15.31:36716) 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 -- 172.21.15.31:0/2795890026 learned_addr learned my addr 172.21.15.31:0/2795890026 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd9895d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 -- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 msgr2=0x55fd989604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 -- 172.21.15.31:0/2795890026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 msgr2=0x55fd9895d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd9895d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 -- 172.21.15.31:0/2795890026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf8009260 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd9895dd20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007500 tx=0x7f2cf8016cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85e81264773dd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.044+0000 7f2cdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2cdeffd700 1 -- 172.21.15.31:0/2795890026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8023070 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2cdeffd700 1 -- 172.21.15.31:0/2795890026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2cf8016eb0 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd989492b0 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2cdeffd700 1 -- 172.21.15.31:0/2795890026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf8004ce0 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2d11a15b80 10 client.16281 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fd9886e170 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2cdeffd700 1 -- 172.21.15.31:0/2795890026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd989492b0 con 0x55fd98960b50 2023-12-14T18:30:18.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.045+0000 7f2cdeffd700 1 -- 172.21.15.31:0/2795890026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cf8030070 con 0x55fd98960b50 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2d11a15b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2d11a15b80 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2d11a15b80 20 client.16281 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98970350 0x55fd98972760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2d11a15b80 4 client.16281 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd988a0ce0 con 0x55fd98970350 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98970350 0x55fd98972760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98970350 0x55fd98972760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfbcdfa513107db server_cookie=89db6a4ba0d3b546 in_seq=0 out_seq=0 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.046+0000 7f2cdeffd700 10 client.16281 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2cdeffd700 1 -- 172.21.15.31:0/2795890026 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd988a0ce0 con 0x55fd98970350 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2cdeffd700 10 client.16281 handle_command_reply: tid=0 2023-12-14T18:30:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 client.16281 shutdown 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 msgr2=0x55fd9895dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd9895dd20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8007500 tx=0x7f2cf8016cc0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 shutdown_connections 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98970350 0x55fd98972760 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98960b50 0x55fd9895dd20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2cdeffd700 0 client.16281 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/2795890026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd9895d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.047+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 >> 172.21.15.31:0/2795890026 conn(0x55fd9894bca0 msgr2=0x55fd989513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.048+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 shutdown_connections 2023-12-14T18:30:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.048+0000 7f2d11a15b80 1 -- 172.21.15.31:0/2795890026 wait complete. 2023-12-14T18:30:18.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.048+0000 7f2d11a15b80 20 client.16281 trim_cache size 0 max 16384 2023-12-14T18:30:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.062+0000 7f2d11a15b80 1 Processor -- start 2023-12-14T18:30:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.062+0000 7f2d11a15b80 1 -- start start 2023-12-14T18:30:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd9894bca0 0x55fd9895ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98958ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98959670 con 0x55fd98955380 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98956d30 con 0x55fd9894bca0 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98972730 con 0x55fd98955c90 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36730/0 (socket says 172.21.15.31:36730) 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/3020219261 learned_addr learned my addr 172.21.15.31:0/3020219261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd9894bca0 0x55fd9895ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98958ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/3020219261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 msgr2=0x55fd98958ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98958ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/3020219261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd9894bca0 msgr2=0x55fd9895ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd9894bca0 0x55fd9895ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfe7fc700 1 -- 172.21.15.31:0/3020219261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd9886dec0 con 0x55fd98955380 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.063+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd9894bca0 0x55fd9895ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.064+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 0x55fd989604d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4099d70 tx=0x7f2cf40cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba579adcb194a737 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.064+0000 7f2ce67fc700 1 -- 172.21.15.31:0/3020219261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf409a070 con 0x55fd98955380 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.064+0000 7f2ce67fc700 1 -- 172.21.15.31:0/3020219261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf40b1070 con 0x55fd98955380 2023-12-14T18:30:18.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.064+0000 7f2ce67fc700 1 -- 172.21.15.31:0/3020219261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf409a530 con 0x55fd98955380 2023-12-14T18:30:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3020219261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 msgr2=0x55fd989604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 0x55fd989604d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4099d70 tx=0x7f2cf40cd040 comp rx=0 tx=0).stop 2023-12-14T18:30:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3020219261 shutdown_connections 2023-12-14T18:30:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955c90 0x55fd98958ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd9894bca0 0x55fd9895ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3020219261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd98955380 0x55fd989604d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3020219261 >> 172.21.15.31:0/3020219261 conn(0x55fd98963200 msgr2=0x55fd9896c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3020219261 shutdown_connections 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3020219261 wait complete. 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.065+0000 7f2d11a15b80 1 -- start start 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 Processor -- start 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 0x55fd98967550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd98965880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd98965dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98966300 con 0x55fd9894bca0 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98966440 con 0x55fd98955c90 2023-12-14T18:30:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd98966600 con 0x55fd98955380 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2d11a15b80 10 client.0 _ll_register_callbacks cb 0x7f2cf0005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2cfeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 0x55fd98967550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2cfeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 0x55fd98967550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36738/0 (socket says 172.21.15.31:36738) 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2cfeffd700 1 -- 172.21.15.31:0/3302223139 learned_addr learned my addr 172.21.15.31:0/3302223139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd98965880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.066+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd98965dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfeffd700 1 -- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 msgr2=0x55fd98965880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd98965880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfeffd700 1 -- 172.21.15.31:0/3302223139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 msgr2=0x55fd98965dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd98965dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfeffd700 1 -- 172.21.15.31:0/3302223139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0010d920 con 0x55fd9894bca0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd98965880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cff7fe700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd98965dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cfeffd700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 0x55fd98967550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00098e0 tx=0x7f2cf0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e548887664d1e05 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cdeffd700 1 -- 172.21.15.31:0/3302223139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf002d040 con 0x55fd9894bca0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cf800f040 con 0x55fd9894bca0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cdeffd700 1 -- 172.21.15.31:0/3302223139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2cf002b070 con 0x55fd9894bca0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2cdeffd700 1 -- 172.21.15.31:0/3302223139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cf0005710 con 0x55fd9894bca0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2d11a15b80 10 client.16287 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.067+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f2d001102d0 con 0x55fd9894bca0 2023-12-14T18:30:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2cdeffd700 1 -- 172.21.15.31:0/3302223139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cf800f040 con 0x55fd9894bca0 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2cdeffd700 1 -- 172.21.15.31:0/3302223139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2cf0004d20 con 0x55fd9894bca0 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2d11a15b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2d11a15b80 10 client.16287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2d11a15b80 20 client.16287 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98960b50 0x55fd989c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2d11a15b80 4 client.16287 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd988a0a00 con 0x55fd98960b50 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.068+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98960b50 0x55fd989c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2cfe7fc700 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98960b50 0x55fd989c75b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1467be447c5a63e2 server_cookie=8d0d928937ead2d4 in_seq=0 out_seq=0 2023-12-14T18:30:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2cdeffd700 10 client.16287 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2cdeffd700 1 -- 172.21.15.31:0/3302223139 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd988a0a00 con 0x55fd98960b50 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2cdeffd700 10 client.16287 handle_command_reply: tid=0 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 client.16287 shutdown 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 msgr2=0x55fd98967550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 0x55fd98967550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00098e0 tx=0x7f2cf0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 shutdown_connections 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55fd98960b50 0x55fd989c75b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55fd98955c90 0x55fd98965dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55fd9894bca0 0x55fd98967550 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 --2- 172.21.15.31:0/3302223139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fd98955380 0x55fd98965880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2cdeffd700 0 client.16287 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 >> 172.21.15.31:0/3302223139 conn(0x55fd98963200 msgr2=0x55fd98958ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 shutdown_connections 2023-12-14T18:30:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.069+0000 7f2d11a15b80 1 -- 172.21.15.31:0/3302223139 wait complete. 2023-12-14T18:30:18.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d11a15b80 20 client.16287 trim_cache size 0 max 16384 2023-12-14T18:30:18.078 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:30:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ce80a12c0 msgr2=0x7f2ce80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ce80a12c0 0x7f2ce80a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2cf000b2e0 tx=0x7f2cf0005fd0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 msgr2=0x7f2d00198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 0x7f2d00198ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400e9c0 tx=0x7f2cf400ecd0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 shutdown_connections 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d0010a860 0x7f2d0019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d00076040 0x7f2d001985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ce80a12c0 0x7f2ce80a3770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 --2- 172.21.15.31:0/738713077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d000ffa10 0x7f2d00198ae0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 >> 172.21.15.31:0/738713077 conn(0x7f2d000749f0 msgr2=0x7f2d00101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 shutdown_connections 2023-12-14T18:30:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:18.070+0000 7f2d05887700 1 -- 172.21.15.31:0/738713077 wait complete. 2023-12-14T18:30:18.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:17 smithi204 ceph-mon[43866]: pgmap v77: 129 pgs: 129 active+clean; 2.9 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 11 op/s 2023-12-14T18:30:18.459 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-12-14T18:30:18.459 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-12-14T18:30:18.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ set -e 2023-12-14T18:30:18.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ file=test_chmod.98866 2023-12-14T18:30:18.595 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo foo 2023-12-14T18:30:18.600 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:30:18.600 INFO:tasks.workunit.client.0.smithi031.stderr:+ chmod 400 test_chmod.98866 2023-12-14T18:30:18.602 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:30:18.602 INFO:tasks.workunit.client.0.smithi031.stderr:+ check_perms test_chmod.98866 -r-------- 2023-12-14T18:30:18.603 INFO:tasks.workunit.client.0.smithi031.stderr:+ file=test_chmod.98866 2023-12-14T18:30:18.603 INFO:tasks.workunit.client.0.smithi031.stderr:++ ls -la test_chmod.98866 2023-12-14T18:30:18.620 INFO:tasks.workunit.client.0.smithi031.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Dec 14 18:30 test_chmod.98866' 2023-12-14T18:30:18.620 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:30:18.620 INFO:tasks.workunit.client.0.smithi031.stderr:+ perms=-r-------- 2023-12-14T18:30:18.620 INFO:tasks.workunit.client.0.smithi031.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Dec 14 18:30 test_chmod.98866 2023-12-14T18:30:18.620 INFO:tasks.workunit.client.0.smithi031.stderr:++ awk '{print $1}' 2023-12-14T18:30:18.684 INFO:tasks.workunit.client.0.smithi031.stderr:+ test -r-------- '!=' -r--------. 2023-12-14T18:30:18.685 INFO:tasks.workunit.client.0.smithi031.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Dec 14 18:30 test_chmod.98866 2023-12-14T18:30:18.685 INFO:tasks.workunit.client.0.smithi031.stderr:++ awk '{print $1}' 2023-12-14T18:30:18.687 INFO:tasks.workunit.client.0.smithi031.stderr:+ test -r--------. '!=' -r--------. 2023-12-14T18:30:18.687 INFO:tasks.workunit.client.0.smithi031.stderr:+ set +e 2023-12-14T18:30:18.687 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo bar 2023-12-14T18:30:18.687 INFO:tasks.workunit.client.0.smithi031.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.98866: Permission denied 2023-12-14T18:30:18.688 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 1 = 0 2023-12-14T18:30:18.688 INFO:tasks.workunit.client.0.smithi031.stderr:+ set -e 2023-12-14T18:30:18.688 INFO:tasks.workunit.client.0.smithi031.stderr:+ chmod 600 test_chmod.98866 2023-12-14T18:30:18.688 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo bar 2023-12-14T18:30:18.689 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:30:18.689 INFO:tasks.workunit.client.0.smithi031.stderr:+ check_perms test_chmod.98866 -rw------- 2023-12-14T18:30:18.689 INFO:tasks.workunit.client.0.smithi031.stderr:+ file=test_chmod.98866 2023-12-14T18:30:18.689 INFO:tasks.workunit.client.0.smithi031.stderr:++ ls -la test_chmod.98866 2023-12-14T18:30:18.690 INFO:tasks.workunit.client.0.smithi031.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Dec 14 18:30 test_chmod.98866' 2023-12-14T18:30:18.690 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:30:18.690 INFO:tasks.workunit.client.0.smithi031.stderr:+ perms=-rw------- 2023-12-14T18:30:18.690 INFO:tasks.workunit.client.0.smithi031.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Dec 14 18:30 test_chmod.98866 2023-12-14T18:30:18.690 INFO:tasks.workunit.client.0.smithi031.stderr:++ awk '{print $1}' 2023-12-14T18:30:18.692 INFO:tasks.workunit.client.0.smithi031.stderr:+ test -rw------- '!=' -rw-------. 2023-12-14T18:30:18.693 INFO:tasks.workunit.client.0.smithi031.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Dec 14 18:30 test_chmod.98866 2023-12-14T18:30:18.693 INFO:tasks.workunit.client.0.smithi031.stderr:++ awk '{print $1}' 2023-12-14T18:30:18.695 INFO:tasks.workunit.client.0.smithi031.stderr:+ test -rw-------. '!=' -rw-------. 2023-12-14T18:30:18.695 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo foo 2023-12-14T18:30:18.695 INFO:tasks.workunit.client.0.smithi031.stderr:+ test 0 '!=' 0 2023-12-14T18:30:18.696 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:30:18.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:30:18.741 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-12-14T18:30:18.741 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-12-14T18:30:18.769 INFO:tasks.workunit.client.0.smithi031.stderr:+ set -e 2023-12-14T18:30:18.769 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p testdir 2023-12-14T18:30:18.771 INFO:tasks.workunit.client.0.smithi031.stderr:+ file=test_chmod.98925 2023-12-14T18:30:18.771 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo foo 2023-12-14T18:30:18.772 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo chmod 600 testdir 2023-12-14T18:30:18.793 INFO:tasks.workunit.client.0.smithi031.stderr:+ expect_failure cat testdir/test_chmod.98925 2023-12-14T18:30:18.793 INFO:tasks.workunit.client.0.smithi031.stderr:+ cat testdir/test_chmod.98925 2023-12-14T18:30:18.794 INFO:tasks.workunit.client.0.smithi031.stderr:cat: testdir/test_chmod.98925: Permission denied 2023-12-14T18:30:18.795 INFO:tasks.workunit.client.0.smithi031.stderr:+ return 0 2023-12-14T18:30:18.795 INFO:tasks.workunit.client.0.smithi031.stderr:+ sudo cat testdir/test_chmod.98925 2023-12-14T18:30:18.807 INFO:tasks.workunit.client.0.smithi031.stdout:foo 2023-12-14T18:30:18.809 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:30:18.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:30:18.904 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-12-14T18:30:18.905 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-12-14T18:30:19.145 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:30:19.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.441+0000 7f0e953d7700 1 -- 172.21.15.31:0/1674812729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e900ff390 msgr2=0x7f0e900ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.441+0000 7f0e953d7700 1 --2- 172.21.15.31:0/1674812729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e900ff390 0x7f0e900ff760 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0e800099c0 tx=0x7f0e80009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 -- 172.21.15.31:0/1674812729 shutdown_connections 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 --2- 172.21.15.31:0/1674812729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e901043d0 0x7f0e9010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 --2- 172.21.15.31:0/1674812729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e900ff390 0x7f0e900ff760 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 --2- 172.21.15.31:0/1674812729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ffd30 0x7f0e90103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 -- 172.21.15.31:0/1674812729 >> 172.21.15.31:0/1674812729 conn(0x7f0e90076370 msgr2=0x7f0e90076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 -- 172.21.15.31:0/1674812729 shutdown_connections 2023-12-14T18:30:19.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 -- 172.21.15.31:0/1674812729 wait complete. 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.442+0000 7f0e953d7700 1 Processor -- start 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 -- start start 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ff390 0x7f0e90100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e900ffd30 0x7f0e901012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901043d0 0x7f0e901a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90101b00 con 0x7f0e901043d0 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901a4b00 con 0x7f0e900ff390 2023-12-14T18:30:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e953d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e901a4c40 con 0x7f0e900ffd30 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901043d0 0x7f0e901a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901043d0 0x7f0e901a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36782/0 (socket says 172.21.15.31:36782) 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ff390 0x7f0e90100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8f7fe700 1 -- 172.21.15.31:0/908709035 learned_addr learned my addr 172.21.15.31:0/908709035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e87fff700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e900ffd30 0x7f0e901012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8effd700 1 -- 172.21.15.31:0/908709035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e900ffd30 msgr2=0x7f0e901012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8effd700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e900ffd30 0x7f0e901012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8effd700 1 -- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901043d0 msgr2=0x7f0e901a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8effd700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901043d0 0x7f0e901a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.443+0000 7f0e8effd700 1 -- 172.21.15.31:0/908709035 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800095f0 con 0x7f0e900ff390 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.444+0000 7f0e8effd700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ff390 0x7f0e90100d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e800098e0 tx=0x7f0e80009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.444+0000 7f0e8cff9700 1 -- 172.21.15.31:0/908709035 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8001d070 con 0x7f0e900ff390 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.444+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9010f190 con 0x7f0e900ff390 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.444+0000 7f0e8cff9700 1 -- 172.21.15.31:0/908709035 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e80005020 con 0x7f0e900ff390 2023-12-14T18:30:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.444+0000 7f0e8cff9700 1 -- 172.21.15.31:0/908709035 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80004a90 con 0x7f0e900ff390 2023-12-14T18:30:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.444+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9010f000 con 0x7f0e900ff390 2023-12-14T18:30:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.447+0000 7f0e8cff9700 1 -- 172.21.15.31:0/908709035 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0e800054b0 con 0x7f0e900ff390 2023-12-14T18:30:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.447+0000 7f0e8cff9700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e7c0a12c0 0x7f0e7c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.448+0000 7f0e8cff9700 1 -- 172.21.15.31:0/908709035 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0e800cdae0 con 0x7f0e900ff390 2023-12-14T18:30:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.448+0000 7f0e87fff700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e7c0a12c0 0x7f0e7c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.448+0000 7f0e87fff700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e7c0a12c0 0x7f0e7c0a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e90102580 tx=0x7f0e78009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.448+0000 7f0ea1565b80 1 Processor -- start 2023-12-14T18:30:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.448+0000 7f0ea1565b80 1 -- start start 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c5365b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c5365dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5353a660 con 0x561c53650400 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c535a98f0 con 0x561c53650d10 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c535a2d80 con 0x561c5365bbd0 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36800/0 (socket says 172.21.15.31:36800) 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0e8effd700 1 -- 172.21.15.31:0/3582533834 learned_addr learned my addr 172.21.15.31:0/3582533834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0e87fff700 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c5365b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.449+0000 7f0e8f7fe700 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c5365dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e8effd700 1 -- 172.21.15.31:0/3582533834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 msgr2=0x561c5365dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e8effd700 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c5365dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e8effd700 1 -- 172.21.15.31:0/3582533834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 msgr2=0x561c5365b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e8effd700 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c5365b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e8effd700 1 -- 172.21.15.31:0/3582533834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c53424430 con 0x561c53650400 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e8effd700 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536507d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0e800a2740 tx=0x7f0e8009a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19515833f1a676a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e857fa700 1 -- 172.21.15.31:0/3582533834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8009e8e0 con 0x561c53650400 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e857fa700 1 -- 172.21.15.31:0/3582533834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e800a2c70 con 0x561c53650400 2023-12-14T18:30:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.450+0000 7f0e857fa700 1 -- 172.21.15.31:0/3582533834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80004bf0 con 0x561c53650400 2023-12-14T18:30:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 -- 172.21.15.31:0/3582533834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 msgr2=0x561c536507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536507d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0e800a2740 tx=0x7f0e8009a210 comp rx=0 tx=0).stop 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 -- 172.21.15.31:0/3582533834 shutdown_connections 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c5365dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536507d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/3582533834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c5365b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 -- 172.21.15.31:0/3582533834 >> 172.21.15.31:0/3582533834 conn(0x561c53646d20 msgr2=0x561c5364c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 -- 172.21.15.31:0/3582533834 shutdown_connections 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.451+0000 7f0ea1565b80 1 -- 172.21.15.31:0/3582533834 wait complete. 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 -- start start 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 Processor -- start 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650400 0x561c536647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 0x561c53664cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53665230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5353f010 con 0x561c53650d10 2023-12-14T18:30:19.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5353ecb0 con 0x561c53650400 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c53426720 con 0x561c5365bbd0 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.452+0000 7f0ea1565b80 10 client.0 _ll_register_callbacks cb 0x561c53562990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 0x561c53664cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650400 0x561c536647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 0x561c53664cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36802/0 (socket says 172.21.15.31:36802) 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 -- 172.21.15.31:0/1866882926 learned_addr learned my addr 172.21.15.31:0/1866882926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 -- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 msgr2=0x561c53665230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53665230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 -- 172.21.15.31:0/1866882926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650400 msgr2=0x561c536647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650400 0x561c536647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 -- 172.21.15.31:0/1866882926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9010f000 con 0x561c53650d10 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e8effd700 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650400 0x561c536647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e87fff700 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 0x561c53664cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0e78004520 tx=0x7f0e780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a2e32aa357c91a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e767fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e767fc700 1 -- 172.21.15.31:0/1866882926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78022070 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0e767fc700 1 -- 172.21.15.31:0/1866882926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0e78004550 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.453+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c53644330 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0e767fc700 1 -- 172.21.15.31:0/1866882926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e78003e20 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 10 client.16317 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561c535691f0 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0e767fc700 1 -- 172.21.15.31:0/1866882926 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c53644330 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0e767fc700 1 -- 172.21.15.31:0/1866882926 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e7801d670 con 0x561c53650d10 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 20 client.16317 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c53673b00 0x561c53675f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 4 client.16317 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:19.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c5359bd60 con 0x561c53673b00 2023-12-14T18:30:19.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.454+0000 7f0e8effd700 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c53673b00 0x561c53675f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.455+0000 7f0e8effd700 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c53673b00 0x561c53675f10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16fc460dbced82d server_cookie=f91ed853cba155f4 in_seq=0 out_seq=0 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.455+0000 7f0e767fc700 10 client.16317 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.455+0000 7f0e767fc700 1 -- 172.21.15.31:0/1866882926 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c5359bd60 con 0x561c53673b00 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.455+0000 7f0e767fc700 10 client.16317 handle_command_reply: tid=0 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 client.16317 shutdown 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 msgr2=0x561c53664cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 0x561c53664cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0e78004520 tx=0x7f0e780047c0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 shutdown_connections 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c53673b00 0x561c53675f10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53665230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650400 0x561c536647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/1866882926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650d10 0x561c53664cf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0e767fc700 0 client.16317 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 >> 172.21.15.31:0/1866882926 conn(0x561c53646d20 msgr2=0x561c5364c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 shutdown_connections 2023-12-14T18:30:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 1 -- 172.21.15.31:0/1866882926 wait complete. 2023-12-14T18:30:19.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.456+0000 7f0ea1565b80 20 client.16317 trim_cache size 0 max 16384 2023-12-14T18:30:19.461 INFO:tasks.workunit.client.0.smithi031.stdout:ok 2023-12-14T18:30:19.466 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:30:19.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:30:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 Processor -- start 2023-12-14T18:30:19.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 -- start start 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c53655300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53655840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5364d480 con 0x561c53650400 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c5365ae50 con 0x561c53650d10 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c53656ba0 con 0x561c5365bbd0 2023-12-14T18:30:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36814/0 (socket says 172.21.15.31:36814) 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0e8effd700 1 -- 172.21.15.31:0/984976525 learned_addr learned my addr 172.21.15.31:0/984976525 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0e87fff700 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c53655300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.475+0000 7f0e8f7fe700 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53655840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e8effd700 1 -- 172.21.15.31:0/984976525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 msgr2=0x561c53655840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e8effd700 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53655840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e8effd700 1 -- 172.21.15.31:0/984976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 msgr2=0x561c53655300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e8effd700 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c53655300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e8effd700 1 -- 172.21.15.31:0/984976525 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c53568f40 con 0x561c53650400 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e8effd700 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536697c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005800 tx=0x7f0e8009a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94adab35d90e43c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e85ffb700 1 -- 172.21.15.31:0/984976525 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800b3040 con 0x561c53650400 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e85ffb700 1 -- 172.21.15.31:0/984976525 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e80098c50 con 0x561c53650400 2023-12-14T18:30:19.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.476+0000 7f0e85ffb700 1 -- 172.21.15.31:0/984976525 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80099da0 con 0x561c53650400 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 -- 172.21.15.31:0/984976525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 msgr2=0x561c536697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536697c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e80005800 tx=0x7f0e8009a210 comp rx=0 tx=0).stop 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 -- 172.21.15.31:0/984976525 shutdown_connections 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53655840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536697c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/984976525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c53655300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 -- 172.21.15.31:0/984976525 >> 172.21.15.31:0/984976525 conn(0x561c53646d20 msgr2=0x561c5364c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 -- 172.21.15.31:0/984976525 shutdown_connections 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.477+0000 7f0ea1565b80 1 -- 172.21.15.31:0/984976525 wait complete. 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 -- start start 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 Processor -- start 2023-12-14T18:30:19.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c536679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53667f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c536684b0 con 0x561c53650400 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c536685f0 con 0x561c53650d10 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c536687b0 con 0x561c5365bbd0 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0ea1565b80 10 client.0 _ll_register_callbacks cb 0x561c53562990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0e8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36828/0 (socket says 172.21.15.31:36828) 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0e8effd700 1 -- 172.21.15.31:0/4184789983 learned_addr learned my addr 172.21.15.31:0/4184789983 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0e8f7fe700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53667f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.478+0000 7f0e87fff700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c536679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e8effd700 1 -- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 msgr2=0x561c53667f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e8effd700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53667f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e8effd700 1 -- 172.21.15.31:0/4184789983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 msgr2=0x561c536679f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e8effd700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c536679f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e8effd700 1 -- 172.21.15.31:0/4184789983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c53424430 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e8effd700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536696c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000c00 tx=0x7f0e800053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a823f739046f0c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e767fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e767fc700 1 -- 172.21.15.31:0/4184789983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e800ba0d0 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e767fc700 1 -- 172.21.15.31:0/4184789983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0e800ba230 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e800a2c70 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e767fc700 1 -- 172.21.15.31:0/4184789983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e80099da0 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0ea1565b80 10 client.16323 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561c5354fa20 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.479+0000 7f0e767fc700 1 -- 172.21.15.31:0/4184789983 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e80064030 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0e767fc700 1 -- 172.21.15.31:0/4184789983 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e800a2870 con 0x561c53650400 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0ea1565b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0ea1565b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0ea1565b80 20 client.16323 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c536c8a30 0x561c536cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0ea1565b80 4 client.16323 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:30:19.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561c5359ba80 con 0x561c536c8a30 2023-12-14T18:30:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0e87fff700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c536c8a30 0x561c536cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0e767fc700 10 client.16323 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.480+0000 7f0e87fff700 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c536c8a30 0x561c536cae10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53dc96ac9edf2ffb server_cookie=58b8a2a020988580 in_seq=0 out_seq=0 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0e767fc700 1 -- 172.21.15.31:0/4184789983 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561c5359ba80 con 0x561c536c8a30 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0e767fc700 10 client.16323 handle_command_reply: tid=0 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 client.16323 shutdown 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 msgr2=0x561c536696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536696c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0e80000c00 tx=0x7f0e800053b0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 shutdown_connections 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561c536c8a30 0x561c536cae10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561c5365bbd0 0x561c53667f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561c53650400 0x561c536696c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 --2- 172.21.15.31:0/4184789983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561c53650d10 0x561c536679f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.482+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 >> 172.21.15.31:0/4184789983 conn(0x561c53646d20 msgr2=0x561c53659500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e767fc700 0 client.16323 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 shutdown_connections 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0ea1565b80 1 -- 172.21.15.31:0/4184789983 wait complete. 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0ea1565b80 20 client.16323 trim_cache size 0 max 16384 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "29c050fb-1757-4b92-a538-8bba25085dc0", 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:30:19.487 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e7c0a12c0 msgr2=0x7f0e7c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e7c0a12c0 0x7f0e7c0a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e90102580 tx=0x7f0e78009260 comp rx=0 tx=0).stop 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ff390 msgr2=0x7f0e90100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ff390 0x7f0e90100d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e800098e0 tx=0x7f0e80009580 comp rx=0 tx=0).stop 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 shutdown_connections 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e901043d0 0x7f0e901a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e900ff390 0x7f0e90100d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e7c0a12c0 0x7f0e7c0a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 --2- 172.21.15.31:0/908709035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e900ffd30 0x7f0e901012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.483+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 >> 172.21.15.31:0/908709035 conn(0x7f0e90076370 msgr2=0x7f0e900691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.484+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 shutdown_connections 2023-12-14T18:30:19.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.484+0000 7f0e953d7700 1 -- 172.21.15.31:0/908709035 wait complete. 2023-12-14T18:30:19.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:30:19.512 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-12-14T18:30:19.512 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-12-14T18:30:19.592 INFO:tasks.workunit.client.0.smithi031.stdout:creating folder hierarchy 2023-12-14T18:30:19.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 -- 172.21.15.31:0/2293774264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21341026c0 msgr2=0x7f2134102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 --2- 172.21.15.31:0/2293774264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21341026c0 0x7f2134102a90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21240099c0 tx=0x7f2124009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 -- 172.21.15.31:0/2293774264 shutdown_connections 2023-12-14T18:30:19.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 --2- 172.21.15.31:0/2293774264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 --2- 172.21.15.31:0/2293774264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21341026c0 0x7f2134102a90 secure :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21240099c0 tx=0x7f2124009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 --2- 172.21.15.31:0/2293774264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134103060 0x7f2134107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 -- 172.21.15.31:0/2293774264 >> 172.21.15.31:0/2293774264 conn(0x7f21340fdf90 msgr2=0x7f21341003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 -- 172.21.15.31:0/2293774264 shutdown_connections 2023-12-14T18:30:19.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.705+0000 7f2138c68700 1 -- 172.21.15.31:0/2293774264 wait complete. 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 Processor -- start 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 -- start start 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2134103060 0x7f213419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134198640 0x7f2134198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21341990b0 con 0x7f2134103060 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21341991f0 con 0x7f2134198640 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.706+0000 7f2138c68700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21341993b0 con 0x7f2134107700 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:56542/0 (socket says 172.21.15.31:56542) 2023-12-14T18:30:19.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 -- 172.21.15.31:0/2703173557 learned_addr learned my addr 172.21.15.31:0/2703173557 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f213259c700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2134103060 0x7f213419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2132d9d700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134198640 0x7f2134198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 -- 172.21.15.31:0/2703173557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134198640 msgr2=0x7f2134198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134198640 0x7f2134198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 -- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2134103060 msgr2=0x7f213419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2134103060 0x7f213419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 -- 172.21.15.31:0/2703173557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240095f0 con 0x7f2134107700 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f213259c700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2134103060 0x7f213419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2132d9d700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134198640 0x7f2134198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.707+0000 7f2131d9b700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213419eb10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f212800e940 tx=0x7f212800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.708+0000 7f21237fe700 1 -- 172.21.15.31:0/2703173557 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128009730 con 0x7f2134107700 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.708+0000 7f21237fe700 1 -- 172.21.15.31:0/2703173557 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f212800ee90 con 0x7f2134107700 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.708+0000 7f21237fe700 1 -- 172.21.15.31:0/2703173557 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128010ea0 con 0x7f2134107700 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.708+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21341124c0 con 0x7f2134107700 2023-12-14T18:30:19.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.708+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2134112330 con 0x7f2134107700 2023-12-14T18:30:19.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.711+0000 7f21237fe700 1 -- 172.21.15.31:0/2703173557 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f212800cbd0 con 0x7f2134107700 2023-12-14T18:30:19.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f21237fe700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f211c0a1340 0x7f211c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f21237fe700 1 -- 172.21.15.31:0/2703173557 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2128014070 con 0x7f2134107700 2023-12-14T18:30:19.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 Processor -- start 2023-12-14T18:30:19.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 -- start start 2023-12-14T18:30:19.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3748750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3753610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e3755f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e36325e0 con 0x5569e3748380 2023-12-14T18:30:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e36a1870 con 0x5569e3753b50 2023-12-14T18:30:19.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.712+0000 7f2144df6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e369ad00 con 0x5569e3748c90 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3753610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f213259c700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f211c0a1340 0x7f211c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3753610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:56554/0 (socket says 172.21.15.31:56554) 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 -- 172.21.15.31:0/1965707166 learned_addr learned my addr 172.21.15.31:0/1965707166 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f213259c700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3748750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2132d9d700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e3755f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 -- 172.21.15.31:0/1965707166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 msgr2=0x5569e3755f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e3755f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 -- 172.21.15.31:0/1965707166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 msgr2=0x5569e3748750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3748750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2131d9b700 1 -- 172.21.15.31:0/1965707166 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e351c3b0 con 0x5569e3748c90 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f2132d9d700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e3755f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f213259c700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3748750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.713+0000 7f213259c700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f211c0a1340 0x7f211c0a37f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f212400b580 tx=0x7f212400bb40 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.714+0000 7f2131d9b700 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3753610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21280a1ec0 tx=0x7f2128099900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60bca49e7efc370 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.714+0000 7f2120ff9700 1 -- 172.21.15.31:0/1965707166 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212809dfd0 con 0x5569e3748c90 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.714+0000 7f2120ff9700 1 -- 172.21.15.31:0/1965707166 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21280a23f0 con 0x5569e3748c90 2023-12-14T18:30:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.714+0000 7f2120ff9700 1 -- 172.21.15.31:0/1965707166 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21280a1f60 con 0x5569e3748c90 2023-12-14T18:30:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 -- 172.21.15.31:0/1965707166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 msgr2=0x5569e3753610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3753610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21280a1ec0 tx=0x7f2128099900 comp rx=0 tx=0).stop 2023-12-14T18:30:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 -- 172.21.15.31:0/1965707166 shutdown_connections 2023-12-14T18:30:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e3755f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3748750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1965707166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3753610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 -- 172.21.15.31:0/1965707166 >> 172.21.15.31:0/1965707166 conn(0x5569e373eca0 msgr2=0x5569e37447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 -- 172.21.15.31:0/1965707166 shutdown_connections 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 -- 172.21.15.31:0/1965707166 wait complete. 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.715+0000 7f2144df6b80 1 -- start start 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 Processor -- start 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 0x5569e37535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748c90 0x5569e37508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e3750fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e3636f90 con 0x5569e3753b50 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e3636c30 con 0x5569e3748c90 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e351e6a0 con 0x5569e3748380 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2144df6b80 10 client.0 _ll_register_callbacks cb 0x5569e365a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f213259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 0x5569e37535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f213259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 0x5569e37535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:56556/0 (socket says 172.21.15.31:56556) 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f213259c700 1 -- 172.21.15.31:0/3005651983 learned_addr learned my addr 172.21.15.31:0/3005651983 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2132d9d700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e3750fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.716+0000 7f2131d9b700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748c90 0x5569e37508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f213259c700 1 -- 172.21.15.31:0/3005651983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748c90 msgr2=0x5569e37508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f213259c700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748c90 0x5569e37508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f213259c700 1 -- 172.21.15.31:0/3005651983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 msgr2=0x5569e3750fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f213259c700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e3750fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f213259c700 1 -- 172.21.15.31:0/3005651983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2134112330 con 0x5569e3748380 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f213259c700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 0x5569e37535f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f212400a010 tx=0x7f2124003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e8739c2a5e0a64 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f2119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f2119ffb700 1 -- 172.21.15.31:0/3005651983 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124004800 con 0x5569e3748380 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569e373c2b0 con 0x5569e3748380 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f2119ffb700 1 -- 172.21.15.31:0/3005651983 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2124004960 con 0x5569e3748380 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.717+0000 7f2119ffb700 1 -- 172.21.15.31:0/3005651983 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2124004b20 con 0x5569e3748380 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 10 client.26185 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569e3661170 con 0x5569e3748380 2023-12-14T18:30:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2119ffb700 1 -- 172.21.15.31:0/3005651983 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569e373c2b0 con 0x5569e3748380 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2119ffb700 1 -- 172.21.15.31:0/3005651983 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2124028460 con 0x5569e3748380 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 10 client.26185 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 10 client.26185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 20 client.26185 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e3763510 0x5569e3765920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 4 client.26185 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569e3693ce0 con 0x5569e3763510 2023-12-14T18:30:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.718+0000 7f2131d9b700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e3763510 0x5569e3765920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.719+0000 7f2131d9b700 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e3763510 0x5569e3765920 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14355e6126b9416b server_cookie=36fa537e7d1699c2 in_seq=0 out_seq=0 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.719+0000 7f2119ffb700 10 client.26185 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.720+0000 7f2119ffb700 1 -- 172.21.15.31:0/3005651983 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5569e3693ce0 con 0x5569e3763510 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.720+0000 7f2119ffb700 10 client.26185 handle_command_reply: tid=0 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.720+0000 7f2144df6b80 1 client.26185 shutdown 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 msgr2=0x5569e37535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 0x5569e37535f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f212400a010 tx=0x7f2124003680 comp rx=0 tx=0).stop 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 shutdown_connections 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e3763510 0x5569e3765920 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e3750fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748380 0x5569e37535f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 --2- 172.21.15.31:0/3005651983 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748c90 0x5569e37508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 >> 172.21.15.31:0/3005651983 conn(0x5569e373eca0 msgr2=0x5569e37443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2119ffb700 0 client.26185 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 shutdown_connections 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 1 -- 172.21.15.31:0/3005651983 wait complete. 2023-12-14T18:30:19.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.721+0000 7f2144df6b80 20 client.26185 trim_cache size 0 max 16384 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.741+0000 7f2144df6b80 1 Processor -- start 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.741+0000 7f2144df6b80 1 -- start start 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748380 0x5569e375a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e37530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e37535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2144df6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e3743650 con 0x5569e3753b50 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2144df6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e3763330 con 0x5569e3748380 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2144df6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e3756400 con 0x5569e3748c90 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2132d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e37535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e37530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e37530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:56562/0 (socket says 172.21.15.31:56562) 2023-12-14T18:30:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 -- 172.21.15.31:0/1094454067 learned_addr learned my addr 172.21.15.31:0/1094454067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f213259c700 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748380 0x5569e375a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 -- 172.21.15.31:0/1094454067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748380 msgr2=0x5569e375a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748380 0x5569e375a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 -- 172.21.15.31:0/1094454067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 msgr2=0x5569e37535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e37535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 -- 172.21.15.31:0/1094454067 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569e3660ec0 con 0x5569e3748c90 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.742+0000 7f2131d9b700 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e37530b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2128099f30 tx=0x7f2128099be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c684d50f5f1d45 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f21217fa700 1 -- 172.21.15.31:0/1094454067 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128099490 con 0x5569e3748c90 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f21217fa700 1 -- 172.21.15.31:0/1094454067 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21280995f0 con 0x5569e3748c90 2023-12-14T18:30:19.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f21217fa700 1 -- 172.21.15.31:0/1094454067 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21280a21c0 con 0x5569e3748c90 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 -- 172.21.15.31:0/1094454067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 msgr2=0x5569e37530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e37530b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2128099f30 tx=0x7f2128099be0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 -- 172.21.15.31:0/1094454067 shutdown_connections 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3753b50 0x5569e37535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3748380 0x5569e375a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 --2- 172.21.15.31:0/1094454067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e37530b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.743+0000 7f2144df6b80 1 -- 172.21.15.31:0/1094454067 >> 172.21.15.31:0/1094454067 conn(0x5569e373eca0 msgr2=0x5569e3765520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.744+0000 7f2144df6b80 1 -- 172.21.15.31:0/1094454067 shutdown_connections 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.744+0000 7f2144df6b80 1 -- 172.21.15.31:0/1094454067 wait complete. 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.744+0000 7f2144df6b80 1 -- start start 2023-12-14T18:30:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.744+0000 7f2144df6b80 1 Processor -- start 2023-12-14T18:30:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.744+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3757e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3758370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2144df6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e37588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2144df6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e37b6490 con 0x5569e3748380 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2144df6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e37b6600 con 0x5569e3753b50 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2144df6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569e37b67c0 con 0x5569e3748c90 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2144df6b80 10 client.0 _ll_register_callbacks cb 0x7f2124005be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3758370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3758370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:56576/0 (socket says 172.21.15.31:56576) 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 -- 172.21.15.31:0/311756370 learned_addr learned my addr 172.21.15.31:0/311756370 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2132d9d700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e37588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f213259c700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3757e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 -- 172.21.15.31:0/311756370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 msgr2=0x5569e37588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e37588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 -- 172.21.15.31:0/311756370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 msgr2=0x5569e3757e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3757e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 -- 172.21.15.31:0/311756370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569e351c3b0 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f213259c700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3757e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2131d9b700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3758370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f212809a160 tx=0x7f212809e0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16af9543e8234a94 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.745+0000 7f2119ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2119ffb700 1 -- 172.21.15.31:0/311756370 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128099900 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2119ffb700 1 -- 172.21.15.31:0/311756370 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f212809a200 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21280a23f0 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2119ffb700 1 -- 172.21.15.31:0/311756370 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f212809a3c0 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 10 client.26191 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5569e36479a0 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2119ffb700 1 -- 172.21.15.31:0/311756370 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21280a23f0 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2119ffb700 1 -- 172.21.15.31:0/311756370 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2128098f00 con 0x5569e3748c90 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 10 client.26191 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 10 client.26191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 20 client.26191 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e37b79e0 0x5569e37b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:19.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 4 client.26191 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:30:19.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.746+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569e3693a00 con 0x5569e37b79e0 2023-12-14T18:30:19.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.747+0000 7f213259c700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e37b79e0 0x5569e37b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:19.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.747+0000 7f2119ffb700 10 client.26191 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.747+0000 7f213259c700 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e37b79e0 0x5569e37b9dc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabaab0b7f802725 server_cookie=33f993d2bcca6c9b in_seq=0 out_seq=0 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.749+0000 7f2119ffb700 1 -- 172.21.15.31:0/311756370 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5569e3693a00 con 0x5569e37b79e0 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.749+0000 7f2119ffb700 10 client.26191 handle_command_reply: tid=0 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.749+0000 7f2144df6b80 1 client.26191 shutdown 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 msgr2=0x5569e3758370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3758370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f212809a160 tx=0x7f212809e0f0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 shutdown_connections 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5569e37b79e0 0x5569e37b9dc0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5569e3753b50 0x5569e37588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5569e3748380 0x5569e3757e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2119ffb700 0 client.26191 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:19.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 --2- 172.21.15.31:0/311756370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569e3748c90 0x5569e3758370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 >> 172.21.15.31:0/311756370 conn(0x5569e373eca0 msgr2=0x5569e374bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 shutdown_connections 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 1 -- 172.21.15.31:0/311756370 wait complete. 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2144df6b80 20 client.26191 trim_cache size 0 max 16384 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: "29c050fb-1757-4b92-a538-8bba25085dc0": { 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "29c050fb-1757-4b92-a538-8bba25085dc0", 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:30:19.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f211c0a1340 msgr2=0x7f211c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f211c0a1340 0x7f211c0a37f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f212400b580 tx=0x7f212400bb40 comp rx=0 tx=0).stop 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 msgr2=0x7f213419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213419eb10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f212800e940 tx=0x7f212800ec50 comp rx=0 tx=0).stop 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 shutdown_connections 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2134198640 0x7f2134198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2134103060 0x7f213419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f211c0a1340 0x7f211c0a37f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 --2- 172.21.15.31:0/2703173557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2134107700 0x7f213419eb10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.750+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 >> 172.21.15.31:0/2703173557 conn(0x7f21340fdf90 msgr2=0x7f21340fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.751+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 shutdown_connections 2023-12-14T18:30:19.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:19.751+0000 7f2138c68700 1 -- 172.21.15.31:0/2703173557 wait complete. 2023-12-14T18:30:19.766 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:19 smithi031 ceph-mon[42085]: pgmap v78: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-12-14T18:30:19.767 INFO:tasks.cephfs.filesystem:scrub status for tag:29c050fb-1757-4b92-a538-8bba25085dc0 - {'path': '/', 'tag': '29c050fb-1757-4b92-a538-8bba25085dc0', 'options': 'recursive,force'} 2023-12-14T18:30:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:19 smithi176 ceph-mon[43506]: pgmap v78: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-12-14T18:30:20.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:19 smithi204 ceph-mon[43866]: pgmap v78: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2023-12-14T18:30:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 -- 172.21.15.31:0/252697289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f470410a8d0 msgr2=0x7f470410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 --2- 172.21.15.31:0/252697289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f470410a8d0 0x7f470410ccb0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00b0a0 tx=0x7f46fc00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 -- 172.21.15.31:0/252697289 shutdown_connections 2023-12-14T18:30:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 --2- 172.21.15.31:0/252697289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f470410a8d0 0x7f470410ccb0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 --2- 172.21.15.31:0/252697289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4704076070 0x7f47040ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 --2- 172.21.15.31:0/252697289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47040ffa00 0x7f47040ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 -- 172.21.15.31:0/252697289 >> 172.21.15.31:0/252697289 conn(0x7f4704074a50 msgr2=0x7f4704074e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 -- 172.21.15.31:0/252697289 shutdown_connections 2023-12-14T18:30:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 -- 172.21.15.31:0/252697289 wait complete. 2023-12-14T18:30:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.251+0000 7f47098fd700 1 Processor -- start 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 -- start start 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704076070 0x7f4704196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 0x7f4704196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f470410a8d0 0x7f470419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470419b400 con 0x7f47040ffa00 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470419b570 con 0x7f470410a8d0 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47098fd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470419b730 con 0x7f4704076070 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 0x7f4704196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 0x7f4704196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36914/0 (socket says 172.21.15.31:36914) 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47037fe700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f470410a8d0 0x7f470419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 -- 172.21.15.31:0/2064689100 learned_addr learned my addr 172.21.15.31:0/2064689100 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f4702ffd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704076070 0x7f4704196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 -- 172.21.15.31:0/2064689100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704076070 msgr2=0x7f4704196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704076070 0x7f4704196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 -- 172.21.15.31:0/2064689100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f470410a8d0 msgr2=0x7f470419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f470410a8d0 0x7f470419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.252+0000 7f47027fc700 1 -- 172.21.15.31:0/2064689100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc009d00 con 0x7f47040ffa00 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47037fe700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f470410a8d0 0x7f470419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47027fc700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 0x7f4704196960 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00e9a0 tx=0x7f46ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47088fb700 1 -- 172.21.15.31:0/2064689100 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec009790 con 0x7f47040ffa00 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47041102a0 con 0x7f47040ffa00 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47088fb700 1 -- 172.21.15.31:0/2064689100 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ec010430 con 0x7f47040ffa00 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47088fb700 1 -- 172.21.15.31:0/2064689100 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec018680 con 0x7f47040ffa00 2023-12-14T18:30:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.253+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4704110110 con 0x7f47040ffa00 2023-12-14T18:30:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.255+0000 7f47088fb700 1 -- 172.21.15.31:0/2064689100 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f46ec0105a0 con 0x7f47040ffa00 2023-12-14T18:30:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.256+0000 7f47088fb700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f46f00a12c0 0x7f46f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.256+0000 7f4702ffd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f46f00a12c0 0x7f46f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.256+0000 7f47088fb700 1 -- 172.21.15.31:0/2064689100 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f46ec014070 con 0x7f47040ffa00 2023-12-14T18:30:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.257+0000 7f4702ffd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f46f00a12c0 0x7f46f00a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46f400ac50 tx=0x7f46f400a380 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.257+0000 7f4715a8bb80 1 Processor -- start 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.257+0000 7f4715a8bb80 1 -- start start 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039720e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x5650397217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x56503972e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503960b580 con 0x56503972c0f0 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503967a810 con 0x565039721370 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039673ca0 con 0x56503957e430 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x56503972e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x56503972e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36928/0 (socket says 172.21.15.31:36928) 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 -- 172.21.15.31:0/243475514 learned_addr learned my addr 172.21.15.31:0/243475514 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f4702ffd700 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039720e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47027fc700 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x5650397217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 -- 172.21.15.31:0/243475514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 msgr2=0x565039720e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039720e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 -- 172.21.15.31:0/243475514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 msgr2=0x5650397217e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x5650397217e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 -- 172.21.15.31:0/243475514 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650394f5350 con 0x56503972c0f0 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47027fc700 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x5650397217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f47037fe700 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x56503972e4d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f46fc015000 tx=0x7f46fc009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f678a050fa5a91 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f46f9ffb700 1 -- 172.21.15.31:0/243475514 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00e040 con 0x56503972c0f0 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.258+0000 7f46f9ffb700 1 -- 172.21.15.31:0/243475514 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46fc00bd30 con 0x56503972c0f0 2023-12-14T18:30:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f46f9ffb700 1 -- 172.21.15.31:0/243475514 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc005260 con 0x56503972c0f0 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 -- 172.21.15.31:0/243475514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 msgr2=0x56503972e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x56503972e4d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f46fc015000 tx=0x7f46fc009410 comp rx=0 tx=0).stop 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 -- 172.21.15.31:0/243475514 shutdown_connections 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x56503972e4d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039720e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/243475514 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x5650397217e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 -- 172.21.15.31:0/243475514 >> 172.21.15.31:0/243475514 conn(0x565039717890 msgr2=0x56503971d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 -- 172.21.15.31:0/243475514 shutdown_connections 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.259+0000 7f4715a8bb80 1 -- 172.21.15.31:0/243475514 wait complete. 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 -- start start 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 Processor -- start 2023-12-14T18:30:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x56503972ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x565039728d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x565039729300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503960ff30 con 0x56503972c0f0 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503960fbd0 con 0x565039721370 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650394f7640 con 0x56503957e430 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4715a8bb80 10 client.0 _ll_register_callbacks cb 0x5650396338b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f47037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x565039729300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f47037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x565039729300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36930/0 (socket says 172.21.15.31:36930) 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f47037fe700 1 -- 172.21.15.31:0/1436536404 learned_addr learned my addr 172.21.15.31:0/1436536404 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.260+0000 7f4702ffd700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x56503972ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47027fc700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x565039728d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47037fe700 1 -- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 msgr2=0x56503972ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47037fe700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x56503972ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47037fe700 1 -- 172.21.15.31:0/1436536404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 msgr2=0x565039728d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47037fe700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x565039728d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47037fe700 1 -- 172.21.15.31:0/1436536404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470410d970 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f47037fe700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x565039729300 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f46fc003ec0 tx=0x7f46fc003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ca2a1eb5ffca01 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f46e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1436536404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00e070 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1436536404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f46fc007940 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565039714ea0 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1436536404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46fc007700 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f4715a8bb80 10 client.16353 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.261+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56503963a110 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1436536404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565039714ea0 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1436536404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46fc00b0d0 con 0x56503972c0f0 2023-12-14T18:30:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4715a8bb80 10 client.16353 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4715a8bb80 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4715a8bb80 20 client.16353 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56503973b670 0x56503973da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4715a8bb80 4 client.16353 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56503966cc80 con 0x56503973b670 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4702ffd700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56503973b670 0x56503973da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f46e2ffd700 10 client.16353 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.262+0000 7f4702ffd700 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56503973b670 0x56503973da50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84e24bbc366a32e server_cookie=51b28291766b8cf5 in_seq=0 out_seq=0 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f46e2ffd700 1 -- 172.21.15.31:0/1436536404 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56503966cc80 con 0x56503973b670 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f46e2ffd700 10 client.16353 handle_command_reply: tid=0 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 client.16353 shutdown 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 msgr2=0x565039729300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x565039729300 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f46fc003ec0 tx=0x7f46fc003ef0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 shutdown_connections 2023-12-14T18:30:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56503973b670 0x56503973da50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503972c0f0 0x565039729300 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x56503972ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f46e2ffd700 0 client.16353 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/1436536404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565039721370 0x565039728d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 >> 172.21.15.31:0/1436536404 conn(0x565039717890 msgr2=0x56503971cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 shutdown_connections 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.263+0000 7f4715a8bb80 1 -- 172.21.15.31:0/1436536404 wait complete. 2023-12-14T18:30:20.270 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.264+0000 7f4715a8bb80 20 client.16353 trim_cache size 0 max 16384 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 Processor -- start 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 -- start start 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 0x565039732c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565039721370 0x565039725fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x5650397264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565039717560 con 0x56503957e430 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650397298e0 con 0x56503972c0f0 2023-12-14T18:30:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503972e4d0 con 0x565039721370 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 0x565039732c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4702ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 0x565039732c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36934/0 (socket says 172.21.15.31:36934) 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f4702ffd700 1 -- 172.21.15.31:0/3871462326 learned_addr learned my addr 172.21.15.31:0/3871462326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f47037fe700 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x5650397264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.280+0000 7f47027fc700 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565039721370 0x565039725fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f4702ffd700 1 -- 172.21.15.31:0/3871462326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565039721370 msgr2=0x565039725fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f4702ffd700 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565039721370 0x565039725fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f4702ffd700 1 -- 172.21.15.31:0/3871462326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 msgr2=0x5650397264f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f4702ffd700 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x5650397264f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f4702ffd700 1 -- 172.21.15.31:0/3871462326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565039639e60 con 0x56503957e430 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f47027fc700 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565039721370 0x565039725fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f4702ffd700 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 0x565039732c20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x56503973e360 tx=0x7f46f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c540e243388d0b30 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f46fa7fc700 1 -- 172.21.15.31:0/3871462326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4024070 con 0x56503957e430 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f46fa7fc700 1 -- 172.21.15.31:0/3871462326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f4026cb0 con 0x56503957e430 2023-12-14T18:30:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.281+0000 7f46fa7fc700 1 -- 172.21.15.31:0/3871462326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4003aa0 con 0x56503957e430 2023-12-14T18:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 -- 172.21.15.31:0/3871462326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 msgr2=0x565039732c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 0x565039732c20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x56503973e360 tx=0x7f46f4012000 comp rx=0 tx=0).stop 2023-12-14T18:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 -- 172.21.15.31:0/3871462326 shutdown_connections 2023-12-14T18:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x5650397264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56503957e430 0x565039732c20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/3871462326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565039721370 0x565039725fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 -- 172.21.15.31:0/3871462326 >> 172.21.15.31:0/3871462326 conn(0x565039717890 msgr2=0x565039723f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 -- 172.21.15.31:0/3871462326 shutdown_connections 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 -- 172.21.15.31:0/3871462326 wait complete. 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.282+0000 7f4715a8bb80 1 -- start start 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 Processor -- start 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039731290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 0x5650397317d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x565039731d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503978f510 con 0x565039721370 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503978f650 con 0x56503972c0f0 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56503978f790 con 0x56503957e430 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4715a8bb80 10 client.0 _ll_register_callbacks cb 0x7f46fc0092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 0x5650397317d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 0x5650397317d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36948/0 (socket says 172.21.15.31:36948) 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 -- 172.21.15.31:0/291840672 learned_addr learned my addr 172.21.15.31:0/291840672 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47037fe700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x565039731d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f4702ffd700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039731290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 -- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 msgr2=0x565039731290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039731290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 -- 172.21.15.31:0/291840672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 msgr2=0x565039731d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x565039731d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.283+0000 7f47027fc700 1 -- 172.21.15.31:0/291840672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f400a8a0 con 0x565039721370 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f47027fc700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 0x5650397317d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f46ec099c20 tx=0x7f46ec099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ef260cd79d231a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f46e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f46e2ffd700 1 -- 172.21.15.31:0/291840672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec09a380 con 0x565039721370 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f46e2ffd700 1 -- 172.21.15.31:0/291840672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f46ec099410 con 0x565039721370 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46fc00bd30 con 0x565039721370 2023-12-14T18:30:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f46e2ffd700 1 -- 172.21.15.31:0/291840672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec0995d0 con 0x565039721370 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 10 client.16359 fetch_fsmap learned FSMap version 24 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565039620940 con 0x565039721370 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f46e2ffd700 1 -- 172.21.15.31:0/291840672 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46fc00bd30 con 0x565039721370 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f46e2ffd700 1 -- 172.21.15.31:0/291840672 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46ec095d10 con 0x565039721370 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 10 client.16359 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 20 client.16359 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565039790f00 0x565039793310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 4 client.16359 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.284+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56503966c9a0 con 0x565039790f00 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.285+0000 7f4702ffd700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565039790f00 0x565039793310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.285+0000 7f4702ffd700 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565039790f00 0x565039793310 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e28be5d91312363e server_cookie=1537667efb76f520 in_seq=0 out_seq=0 2023-12-14T18:30:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.285+0000 7f46e2ffd700 10 client.16359 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f46e2ffd700 1 -- 172.21.15.31:0/291840672 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56503966c9a0 con 0x565039790f00 2023-12-14T18:30:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f46e2ffd700 10 client.16359 handle_command_reply: tid=0 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 client.16359 shutdown 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 msgr2=0x5650397317d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 0x5650397317d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f46ec099c20 tx=0x7f46ec099990 comp rx=0 tx=0).stop 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 shutdown_connections 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565039790f00 0x565039793310 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56503972c0f0 0x565039731d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f46e2ffd700 0 client.16359 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56503957e430 0x565039731290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 --2- 172.21.15.31:0/291840672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565039721370 0x5650397317d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 >> 172.21.15.31:0/291840672 conn(0x565039717890 msgr2=0x565039727d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 shutdown_connections 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 1 -- 172.21.15.31:0/291840672 wait complete. 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.821+0000 7f4715a8bb80 20 client.16359 trim_cache size 0 max 16384 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:30:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:30:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f46f00a12c0 msgr2=0x7f46f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f46f00a12c0 0x7f46f00a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f46f400ac50 tx=0x7f46f400a380 comp rx=0 tx=0).stop 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 msgr2=0x7f4704196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 0x7f4704196960 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00e9a0 tx=0x7f46ec00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 shutdown_connections 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f470410a8d0 0x7f470419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f46f00a12c0 0x7f46f00a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4704076070 0x7f4704196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 --2- 172.21.15.31:0/2064689100 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47040ffa00 0x7f4704196960 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 >> 172.21.15.31:0/2064689100 conn(0x7f4704074a50 msgr2=0x7f47041027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:20.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 shutdown_connections 2023-12-14T18:30:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:20.822+0000 7f47098fd700 1 -- 172.21.15.31:0/2064689100 wait complete. 2023-12-14T18:30:20.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:20 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:30:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:20 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:20 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:30:20.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:20 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:20 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:30:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:20 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:20 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:30:20.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:20 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:20 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:30:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:20 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:20 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:30:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:20 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:21 smithi031 ceph-mon[42085]: pgmap v79: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-12-14T18:30:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:21 smithi176 ceph-mon[43506]: pgmap v79: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-12-14T18:30:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:21 smithi204 ceph-mon[43866]: pgmap v79: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-12-14T18:30:23.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:23 smithi031 ceph-mon[42085]: pgmap v80: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-12-14T18:30:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:23 smithi176 ceph-mon[43506]: pgmap v80: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-12-14T18:30:24.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:23 smithi204 ceph-mon[43866]: pgmap v80: 129 pgs: 129 active+clean; 3.1 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 7 op/s 2023-12-14T18:30:24.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:24 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:30:24.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:24 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:30:25.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:24 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:30:25.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:25 smithi031 ceph-mon[42085]: pgmap v81: 129 pgs: 129 active+clean; 10 MiB data, 113 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 709 KiB/s wr, 222 op/s 2023-12-14T18:30:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:25 smithi176 ceph-mon[43506]: pgmap v81: 129 pgs: 129 active+clean; 10 MiB data, 113 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 709 KiB/s wr, 222 op/s 2023-12-14T18:30:26.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:25 smithi204 ceph-mon[43866]: pgmap v81: 129 pgs: 129 active+clean; 10 MiB data, 113 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 709 KiB/s wr, 222 op/s 2023-12-14T18:30:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:27 smithi031 ceph-mon[42085]: pgmap v82: 129 pgs: 129 active+clean; 10 MiB data, 113 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 706 KiB/s wr, 219 op/s 2023-12-14T18:30:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:27 smithi176 ceph-mon[43506]: pgmap v82: 129 pgs: 129 active+clean; 10 MiB data, 113 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 706 KiB/s wr, 219 op/s 2023-12-14T18:30:28.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:27 smithi204 ceph-mon[43866]: pgmap v82: 129 pgs: 129 active+clean; 10 MiB data, 113 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 706 KiB/s wr, 219 op/s 2023-12-14T18:30:29.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:29 smithi031 ceph-mon[42085]: pgmap v83: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 278 op/s 2023-12-14T18:30:29.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:29 smithi176 ceph-mon[43506]: pgmap v83: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 278 op/s 2023-12-14T18:30:30.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:29 smithi204 ceph-mon[43866]: pgmap v83: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 278 op/s 2023-12-14T18:30:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:30 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:30 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:31.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:30 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:31.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:31 smithi031 ceph-mon[42085]: pgmap v84: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2023-12-14T18:30:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:31 smithi176 ceph-mon[43506]: pgmap v84: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2023-12-14T18:30:32.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:31 smithi204 ceph-mon[43866]: pgmap v84: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2023-12-14T18:30:33.956 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.953+0000 7f2edd13e700 1 -- 172.21.15.31:0/229019853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed810ce50 msgr2=0x7f2ed810f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.953+0000 7f2edd13e700 1 --2- 172.21.15.31:0/229019853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed810ce50 0x7f2ed810f230 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009f90 tx=0x7f2ed000b330 comp rx=0 tx=0).stop 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 -- 172.21.15.31:0/229019853 shutdown_connections 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 --2- 172.21.15.31:0/229019853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed810ce50 0x7f2ed810f230 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 --2- 172.21.15.31:0/229019853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed80746a0 0x7f2ed8072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 --2- 172.21.15.31:0/229019853 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed8074b00 0x7f2ed8073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 -- 172.21.15.31:0/229019853 >> 172.21.15.31:0/229019853 conn(0x7f2ed8077550 msgr2=0x7f2ed8077950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 -- 172.21.15.31:0/229019853 shutdown_connections 2023-12-14T18:30:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.954+0000 7f2edd13e700 1 -- 172.21.15.31:0/229019853 wait complete. 2023-12-14T18:30:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 Processor -- start 2023-12-14T18:30:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 -- start start 2023-12-14T18:30:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 0x7f2ed8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed8074b00 0x7f2ed8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed810ce50 0x7f2ed819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed819c5f0 con 0x7f2ed80746a0 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed819c760 con 0x7f2ed810ce50 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.955+0000 7f2edd13e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed819c920 con 0x7f2ed8074b00 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 0x7f2ed8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed8074b00 0x7f2ed8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 0x7f2ed8198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54624/0 (socket says 172.21.15.31:54624) 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/508201230 learned_addr learned my addr 172.21.15.31:0/508201230 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed759e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed810ce50 0x7f2ed819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 -- 172.21.15.31:0/508201230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed810ce50 msgr2=0x7f2ed819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed810ce50 0x7f2ed819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 -- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 msgr2=0x7f2ed8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 0x7f2ed8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 -- 172.21.15.31:0/508201230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80095f0 con 0x7f2ed8074b00 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed759e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed810ce50 0x7f2ed819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 0x7f2ed8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ed659c700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed8074b00 0x7f2ed8198c40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00e9f0 tx=0x7f2ecc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ebffff700 1 -- 172.21.15.31:0/508201230 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00c9c0 con 0x7f2ed8074b00 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed0009c40 con 0x7f2ed8074b00 2023-12-14T18:30:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ebffff700 1 -- 172.21.15.31:0/508201230 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ecc010430 con 0x7f2ed8074b00 2023-12-14T18:30:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.956+0000 7f2ebffff700 1 -- 172.21.15.31:0/508201230 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc018680 con 0x7f2ed8074b00 2023-12-14T18:30:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.957+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed8112550 con 0x7f2ed8074b00 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.959+0000 7f2ebffff700 1 -- 172.21.15.31:0/508201230 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2ecc010860 con 0x7f2ed8074b00 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.959+0000 7f2ebffff700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ec00a12c0 0x7f2ec00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.960+0000 7f2ebffff700 1 -- 172.21.15.31:0/508201230 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2ecc014070 con 0x7f2ed8074b00 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.960+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ec00a12c0 0x7f2ec00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.960+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ec00a12c0 0x7f2ec00a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005d10 tx=0x7f2ec8009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.960+0000 7f2ee92ccb80 1 Processor -- start 2023-12-14T18:30:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 -- start start 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f0293e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f029f0f0 0x5624f02a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f017e580 con 0x5624f0294370 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f01ed810 con 0x5624f00f1430 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f01e6ca0 con 0x5624f029f0f0 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f0293e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed759e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f029f0f0 0x5624f02a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f0293e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:47660/0 (socket says 172.21.15.31:47660) 2023-12-14T18:30:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/2338922196 learned_addr learned my addr 172.21.15.31:0/2338922196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/2338922196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f029f0f0 msgr2=0x5624f02a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f029f0f0 0x5624f02a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/2338922196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 msgr2=0x5624f02947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.961+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/2338922196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f0068350 con 0x5624f00f1430 2023-12-14T18:30:33.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:33 smithi031 ceph-mon[42085]: pgmap v85: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2023-12-14T18:30:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.962+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f0293e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8011810 tx=0x7f2ec8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa335690f825f4b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.962+0000 7f2ebd7fa700 1 -- 172.21.15.31:0/2338922196 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8026630 con 0x5624f00f1430 2023-12-14T18:30:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.962+0000 7f2ebd7fa700 1 -- 172.21.15.31:0/2338922196 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ec8021070 con 0x5624f00f1430 2023-12-14T18:30:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.962+0000 7f2ebd7fa700 1 -- 172.21.15.31:0/2338922196 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec801d7f0 con 0x5624f00f1430 2023-12-14T18:30:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/2338922196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 msgr2=0x5624f0293e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f0293e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8011810 tx=0x7f2ec8014000 comp rx=0 tx=0).stop 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/2338922196 shutdown_connections 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f029f0f0 0x5624f02a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f0293e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/2338922196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/2338922196 >> 172.21.15.31:0/2338922196 conn(0x5624f028a890 msgr2=0x5624f0290200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/2338922196 shutdown_connections 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/2338922196 wait complete. 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 -- start start 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.963+0000 7f2ee92ccb80 1 Processor -- start 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f029e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f029bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 0x5624f029c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0182f30 con 0x5624f029f0f0 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f0182bd0 con 0x5624f00f1430 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f006a640 con 0x5624f0294370 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ee92ccb80 10 client.0 _ll_register_callbacks cb 0x5624f01a68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f029bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 0x5624f029c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f029bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45426/0 (socket says 172.21.15.31:45426) 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 0x5624f029c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54656/0 (socket says 172.21.15.31:54656) 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 -- 172.21.15.31:0/351360171 learned_addr learned my addr 172.21.15.31:0/351360171 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f029e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 -- 172.21.15.31:0/351360171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 msgr2=0x5624f029e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f029e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 -- 172.21.15.31:0/351360171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 msgr2=0x5624f029c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 0x5624f029c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed659c700 1 -- 172.21.15.31:0/351360171 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec8017070 con 0x5624f0294370 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.964+0000 7f2ed759e700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 0x5624f029c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ed659c700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f029bcb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0963d0 tx=0x7f2ecc09a010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffb0cddbe276ee01 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ebaff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ebaff5700 1 -- 172.21.15.31:0/351360171 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0995c0 con 0x5624f0294370 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ebaff5700 1 -- 172.21.15.31:0/351360171 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2ecc0a2320 con 0x5624f0294370 2023-12-14T18:30:33.974 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624f0287ea0 con 0x5624f0294370 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ebaff5700 1 -- 172.21.15.31:0/351360171 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ecc0adbd0 con 0x5624f0294370 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ee92ccb80 10 client.26221 fetch_fsmap learned FSMap version 25 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.965+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5624f01ad110 con 0x5624f0294370 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ebaff5700 1 -- 172.21.15.31:0/351360171 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624f0287ea0 con 0x5624f0294370 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ebaff5700 1 -- 172.21.15.31:0/351360171 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2ecc0b7fa0 con 0x5624f0294370 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ee92ccb80 10 client.26221 fetch_fsmap finished waiting for FSMap version 25 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ee92ccb80 10 client.26221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ee92ccb80 20 client.26221 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f02aeb60 0x5624f02b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ee92ccb80 4 client.26221 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624f01dfc80 con 0x5624f02aeb60 2023-12-14T18:30:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.966+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f02aeb60 0x5624f02b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.967+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f02aeb60 0x5624f02b0f70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a898dea4f62184d3 server_cookie=da000ac7898aa445 in_seq=0 out_seq=0 2023-12-14T18:30:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.967+0000 7f2ebaff5700 10 client.26221 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.969+0000 7f2ebaff5700 1 -- 172.21.15.31:0/351360171 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624f01dfc80 con 0x5624f02aeb60 2023-12-14T18:30:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ebaff5700 10 client.26221 handle_command_reply: tid=0 2023-12-14T18:30:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 client.26221 shutdown 2023-12-14T18:30:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 msgr2=0x5624f029bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f029bcb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0963d0 tx=0x7f2ecc09a010 comp rx=0 tx=0).stop 2023-12-14T18:30:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 shutdown_connections 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f02aeb60 0x5624f02b0f70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f029f0f0 0x5624f029c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f00f1430 0x5624f029e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/351360171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f029bcb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ebaff5700 0 client.26221 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 >> 172.21.15.31:0/351360171 conn(0x5624f028a890 msgr2=0x5624f028fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 shutdown_connections 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/351360171 wait complete. 2023-12-14T18:30:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.970+0000 7f2ee92ccb80 20 client.26221 trim_cache size 0 max 16384 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.987+0000 7f2ee92ccb80 1 Processor -- start 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.987+0000 7f2ee92ccb80 1 -- start start 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.987+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f00f1430 0x5624f02a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.987+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f028fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.987+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f0290210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f02950a0 con 0x5624f00f1430 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f02958c0 con 0x5624f029f0f0 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f02a1a70 con 0x5624f0294370 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f028fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f00f1430 0x5624f02a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f028fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45428/0 (socket says 172.21.15.31:45428) 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed659c700 1 -- 172.21.15.31:0/389974555 learned_addr learned my addr 172.21.15.31:0/389974555 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed759e700 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f0290210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/389974555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 msgr2=0x5624f028fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f028fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/389974555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 msgr2=0x5624f0290210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f0290210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 -- 172.21.15.31:0/389974555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624f01ace60 con 0x5624f00f1430 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed659c700 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f028fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.988+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f00f1430 0x5624f02a5cf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8013fd0 tx=0x7f2ec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dea5611cfc8da03 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ebdffb700 1 -- 172.21.15.31:0/389974555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034b50 con 0x5624f00f1430 2023-12-14T18:30:33.992 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ebdffb700 1 -- 172.21.15.31:0/389974555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ec8038070 con 0x5624f00f1430 2023-12-14T18:30:33.993 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ebdffb700 1 -- 172.21.15.31:0/389974555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004550 con 0x5624f00f1430 2023-12-14T18:30:33.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:33 smithi176 ceph-mon[43506]: pgmap v85: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/389974555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f00f1430 msgr2=0x5624f02a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f00f1430 0x5624f02a5cf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8013fd0 tx=0x7f2ec8017000 comp rx=0 tx=0).stop 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/389974555 shutdown_connections 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f0290210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f00f1430 0x5624f02a5cf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.989+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/389974555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f0294370 0x5624f028fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/389974555 >> 172.21.15.31:0/389974555 conn(0x5624f028a890 msgr2=0x5624f0296fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/389974555 shutdown_connections 2023-12-14T18:30:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/389974555 wait complete. 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- start start 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 Processor -- start 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f00f1430 0x5624f02f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f02fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f02fbf00 con 0x5624f0294370 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f02fc040 con 0x5624f029f0f0 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624f02fc200 con 0x5624f00f1430 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.990+0000 7f2ee92ccb80 10 client.0 _ll_register_callbacks cb 0x7f2ecc099830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54688/0 (socket says 172.21.15.31:54688) 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 -- 172.21.15.31:0/1075058445 learned_addr learned my addr 172.21.15.31:0/1075058445 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f00f1430 0x5624f02f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed759e700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f02fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 -- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f00f1430 msgr2=0x5624f02f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f00f1430 0x5624f02f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 -- 172.21.15.31:0/1075058445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 msgr2=0x5624f02fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f02fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 -- 172.21.15.31:0/1075058445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624f01ad110 con 0x5624f0294370 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ed659c700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02f70a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc000c00 tx=0x7f2ecc010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccf5ae72f757f18 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ebaff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ebaff5700 1 -- 172.21.15.31:0/1075058445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc0addd0 con 0x5624f0294370 2023-12-14T18:30:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ebaff5700 1 -- 172.21.15.31:0/1075058445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f2ecc0b2ac0 con 0x5624f0294370 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.991+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ec8021070 con 0x5624f0294370 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ebaff5700 1 -- 172.21.15.31:0/1075058445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ecc0b8170 con 0x5624f0294370 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 10 client.16377 fetch_fsmap learned FSMap version 25 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f2ec802a040 con 0x5624f0294370 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ebaff5700 1 -- 172.21.15.31:0/1075058445 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ec8021070 con 0x5624f0294370 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ebaff5700 1 -- 172.21.15.31:0/1075058445 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2ecc094ae0 con 0x5624f0294370 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 10 client.16377 fetch_fsmap finished waiting for FSMap version 25 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 20 client.16377 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f03037e0 0x5624f0305bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 4 client.16377 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5624f01df9a0 con 0x5624f03037e0 2023-12-14T18:30:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.992+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f03037e0 0x5624f0305bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.993+0000 7f2ed6d9d700 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f03037e0 0x5624f0305bf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926000776fffaabb server_cookie=de32e7dd4760d2f0 in_seq=0 out_seq=0 2023-12-14T18:30:34.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:33.993+0000 7f2ebaff5700 10 client.16377 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:34.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:33 smithi204 ceph-mon[43866]: pgmap v85: 129 pgs: 129 active+clean; 21 MiB data, 134 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2023-12-14T18:30:35.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:35 smithi176 ceph-mon[43506]: pgmap v86: 129 pgs: 129 active+clean; 36 MiB data, 146 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 294 op/s 2023-12-14T18:30:36.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:35 smithi204 ceph-mon[43866]: pgmap v86: 129 pgs: 129 active+clean; 36 MiB data, 146 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 294 op/s 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ebaff5700 1 -- 172.21.15.31:0/1075058445 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5624f01df9a0 con 0x5624f03037e0 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ebaff5700 10 client.16377 handle_command_reply: tid=0 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 client.16377 shutdown 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 msgr2=0x5624f02f70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02f70a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc000c00 tx=0x7f2ecc010c60 comp rx=0 tx=0).stop 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 shutdown_connections 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5624f03037e0 0x5624f0305bf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5624f029f0f0 0x5624f02fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624f00f1430 0x5624f02f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 --2- 172.21.15.31:0/1075058445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5624f0294370 0x5624f02f70a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ebaff5700 0 client.16377 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 >> 172.21.15.31:0/1075058445 conn(0x5624f028a890 msgr2=0x5624f02aa490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 shutdown_connections 2023-12-14T18:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.166+0000 7f2ee92ccb80 1 -- 172.21.15.31:0/1075058445 wait complete. 2023-12-14T18:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2ee92ccb80 20 client.16377 trim_cache size 0 max 16384 2023-12-14T18:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:30:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ec00a12c0 msgr2=0x7f2ec00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ec00a12c0 0x7f2ec00a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005d10 tx=0x7f2ec8009310 comp rx=0 tx=0).stop 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed8074b00 msgr2=0x7f2ed8198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed8074b00 0x7f2ed8198c40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00e9f0 tx=0x7f2ecc00ed00 comp rx=0 tx=0).stop 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 shutdown_connections 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2ed810ce50 0x7f2ed819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ed80746a0 0x7f2ed8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2ec00a12c0 0x7f2ec00a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 --2- 172.21.15.31:0/508201230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ed8074b00 0x7f2ed8198c40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.167+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 >> 172.21.15.31:0/508201230 conn(0x7f2ed8077550 msgr2=0x7f2ed8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.168+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 shutdown_connections 2023-12-14T18:30:36.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:36.168+0000 7f2edd13e700 1 -- 172.21.15.31:0/508201230 wait complete. 2023-12-14T18:30:36.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:35 smithi031 ceph-mon[42085]: pgmap v86: 129 pgs: 129 active+clean; 36 MiB data, 146 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 294 op/s 2023-12-14T18:30:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:37 smithi176 ceph-mon[43506]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 146 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 79 op/s 2023-12-14T18:30:38.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:37 smithi204 ceph-mon[43866]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 146 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 79 op/s 2023-12-14T18:30:38.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:37 smithi031 ceph-mon[42085]: pgmap v87: 129 pgs: 129 active+clean; 36 MiB data, 146 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 79 op/s 2023-12-14T18:30:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:39 smithi176 ceph-mon[43506]: pgmap v88: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 1.52k op/s 2023-12-14T18:30:40.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:39 smithi204 ceph-mon[43866]: pgmap v88: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 1.52k op/s 2023-12-14T18:30:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:39 smithi031 ceph-mon[42085]: pgmap v88: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 1.52k op/s 2023-12-14T18:30:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:41 smithi176 ceph-mon[43506]: pgmap v89: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.46k op/s 2023-12-14T18:30:42.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:41 smithi204 ceph-mon[43866]: pgmap v89: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.46k op/s 2023-12-14T18:30:42.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:41 smithi031 ceph-mon[42085]: pgmap v89: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.46k op/s 2023-12-14T18:30:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:43 smithi176 ceph-mon[43506]: pgmap v90: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.46k op/s 2023-12-14T18:30:44.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:43 smithi204 ceph-mon[43866]: pgmap v90: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.46k op/s 2023-12-14T18:30:44.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:43 smithi031 ceph-mon[42085]: pgmap v90: 129 pgs: 129 active+clean; 15 MiB data, 245 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 1.46k op/s 2023-12-14T18:30:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:45 smithi176 ceph-mon[43506]: pgmap v91: 129 pgs: 129 active+clean; 34 MiB data, 253 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 1.47k op/s 2023-12-14T18:30:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:45 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:46.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:45 smithi204 ceph-mon[43866]: pgmap v91: 129 pgs: 129 active+clean; 34 MiB data, 253 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 1.47k op/s 2023-12-14T18:30:46.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:45 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:46.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:45 smithi031 ceph-mon[42085]: pgmap v91: 129 pgs: 129 active+clean; 34 MiB data, 253 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 1.47k op/s 2023-12-14T18:30:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:45 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:47 smithi176 ceph-mon[43506]: pgmap v92: 129 pgs: 129 active+clean; 34 MiB data, 253 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.45k op/s 2023-12-14T18:30:48.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:47 smithi204 ceph-mon[43866]: pgmap v92: 129 pgs: 129 active+clean; 34 MiB data, 253 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.45k op/s 2023-12-14T18:30:48.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:47 smithi031 ceph-mon[42085]: pgmap v92: 129 pgs: 129 active+clean; 34 MiB data, 253 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 1.45k op/s 2023-12-14T18:30:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1467787764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c41084a0 msgr2=0x7f67c4113ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1467787764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c41084a0 0x7f67c4113ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b0d0 tx=0x7f67bc00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1467787764 shutdown_connections 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1467787764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c41084a0 0x7f67c4113ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1467787764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c410c060 0x7f67c410c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1467787764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c4107af0 0x7f67c4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1467787764 >> 172.21.15.31:0/1467787764 conn(0x7f67c406c510 msgr2=0x7f67c40716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1467787764 shutdown_connections 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.326+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1467787764 wait complete. 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.327+0000 7f67ca7b7700 1 Processor -- start 2023-12-14T18:30:49.330 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.327+0000 7f67ca7b7700 1 -- start start 2023-12-14T18:30:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.327+0000 7f67ca7b7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 0x7f67c41a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.327+0000 7f67ca7b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 0x7f67c41a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67ca7b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c410c060 0x7f67c41a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67ca7b7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41a5d40 con 0x7f67c41084a0 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67ca7b7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41a5eb0 con 0x7f67c4107af0 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67ca7b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41a6070 con 0x7f67c410c060 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 0x7f67c41a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 0x7f67c41a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 0x7f67c41a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:58394/0 (socket says 172.21.15.31:58394) 2023-12-14T18:30:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c8d54700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c410c060 0x7f67c41a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 -- 172.21.15.31:0/1760800764 learned_addr learned my addr 172.21.15.31:0/1760800764 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 0x7f67c41a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38314/0 (socket says 172.21.15.31:38314) 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 -- 172.21.15.31:0/1760800764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c410c060 msgr2=0x7f67c41a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c410c060 0x7f67c41a5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 -- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 msgr2=0x7f67c41a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 0x7f67c41a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 -- 172.21.15.31:0/1760800764 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b400d590 con 0x7f67c4107af0 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c8d54700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c410c060 0x7f67c41a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 0x7f67c41a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.328+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 0x7f67c41a0dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67b40099f0 tx=0x7f67b4009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.329+0000 7f67c17fa700 1 -- 172.21.15.31:0/1760800764 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4012070 con 0x7f67c4107af0 2023-12-14T18:30:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.329+0000 7f67c17fa700 1 -- 172.21.15.31:0/1760800764 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b4009e30 con 0x7f67c4107af0 2023-12-14T18:30:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.329+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc009d00 con 0x7f67c4107af0 2023-12-14T18:30:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.329+0000 7f67c17fa700 1 -- 172.21.15.31:0/1760800764 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4016e70 con 0x7f67c4107af0 2023-12-14T18:30:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.329+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c41168f0 con 0x7f67c4107af0 2023-12-14T18:30:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.332+0000 7f67c17fa700 1 -- 172.21.15.31:0/1760800764 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f67b4016510 con 0x7f67c4107af0 2023-12-14T18:30:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.333+0000 7f67c17fa700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f67ac0a12c0 0x7f67ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.333+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f67ac0a12c0 0x7f67ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.333+0000 7f67c17fa700 1 -- 172.21.15.31:0/1760800764 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f67b40cc520 con 0x7f67c4107af0 2023-12-14T18:30:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.333+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f67ac0a12c0 0x7f67ac0a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67c41a2370 tx=0x7f67b8009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 Processor -- start 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 -- start start 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846b8370 0x5609846b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609845a2580 con 0x560984515430 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560984611810 con 0x5609846b8370 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67d6945b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098460aca0 con 0x5609846c30f0 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38316/0 (socket says 172.21.15.31:38316) 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 -- 172.21.15.31:0/1782117590 learned_addr learned my addr 172.21.15.31:0/1782117590 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846b8370 0x5609846b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c8d54700 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 -- 172.21.15.31:0/1782117590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 msgr2=0x5609846c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 -- 172.21.15.31:0/1782117590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846b8370 msgr2=0x5609846b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.334+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846b8370 0x5609846b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.335+0000 7f67c3fff700 1 -- 172.21.15.31:0/1782117590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56098448c350 con 0x560984515430 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.335+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846b8370 0x5609846b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.335+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846b7e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f67b4095cd0 tx=0x7f67b4098e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d143ab221e469b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.335+0000 7f67aa7fc700 1 -- 172.21.15.31:0/1782117590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b409d290 con 0x560984515430 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.335+0000 7f67aa7fc700 1 -- 172.21.15.31:0/1782117590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b40a16b0 con 0x560984515430 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.335+0000 7f67aa7fc700 1 -- 172.21.15.31:0/1782117590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40989b0 con 0x560984515430 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 -- 172.21.15.31:0/1782117590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 msgr2=0x5609846b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846b7e30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f67b4095cd0 tx=0x7f67b4098e70 comp rx=0 tx=0).stop 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 -- 172.21.15.31:0/1782117590 shutdown_connections 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846b7e30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1782117590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846b8370 0x5609846b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 -- 172.21.15.31:0/1782117590 >> 172.21.15.31:0/1782117590 conn(0x5609846ae890 msgr2=0x5609846b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 -- 172.21.15.31:0/1782117590 shutdown_connections 2023-12-14T18:30:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 -- 172.21.15.31:0/1782117590 wait complete. 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.336+0000 7f67d6945b80 1 -- start start 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 Processor -- start 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846b8370 0x5609846bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609845a6f30 con 0x560984515430 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609845a6bd0 con 0x5609846c30f0 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56098448e640 con 0x5609846b8370 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67d6945b80 10 client.0 _ll_register_callbacks cb 0x5609845ca8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846b8370 0x5609846bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38324/0 (socket says 172.21.15.31:38324) 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c3fff700 1 -- 172.21.15.31:0/2679531729 learned_addr learned my addr 172.21.15.31:0/2679531729 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c8d54700 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c37fe700 1 -- 172.21.15.31:0/2679531729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 msgr2=0x5609846c0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.337+0000 7f67c37fe700 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67c37fe700 1 -- 172.21.15.31:0/2679531729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 msgr2=0x5609846c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67c37fe700 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67c37fe700 1 -- 172.21.15.31:0/2679531729 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c41168f0 con 0x5609846b8370 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67c37fe700 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846b8370 0x5609846bfd70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67b8011bb0 tx=0x7f67b80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f62ec4931835230 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67a37fe700 1 -- 172.21.15.31:0/2679531729 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b801f740 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67a37fe700 1 -- 172.21.15.31:0/2679531729 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f67b801f8a0 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609846abea0 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67a37fe700 1 -- 172.21.15.31:0/2679531729 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b801fa60 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67d6945b80 10 client.26239 fetch_fsmap learned FSMap version 26 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5609845d1110 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.338+0000 7f67a37fe700 1 -- 172.21.15.31:0/2679531729 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609846abea0 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67a37fe700 1 -- 172.21.15.31:0/2679531729 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 26) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f67b800e3b0 con 0x5609846b8370 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67d6945b80 10 client.26239 fetch_fsmap finished waiting for FSMap version 26 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67d6945b80 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67d6945b80 20 client.26239 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67d6945b80 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5609846d2ca0 0x5609846d50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67d6945b80 4 client.26239 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560984603c80 con 0x5609846d2ca0 2023-12-14T18:30:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67c3fff700 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5609846d2ca0 0x5609846d50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67c3fff700 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5609846d2ca0 0x5609846d50b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af167b731dbf35e server_cookie=67c143bae387f7ba in_seq=0 out_seq=0 2023-12-14T18:30:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.339+0000 7f67a37fe700 10 client.26239 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67a37fe700 1 -- 172.21.15.31:0/2679531729 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560984603c80 con 0x5609846d2ca0 2023-12-14T18:30:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67a37fe700 10 client.26239 handle_command_reply: tid=0 2023-12-14T18:30:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 client.26239 shutdown 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846b8370 msgr2=0x5609846bfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846b8370 0x5609846bfd70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67b8011bb0 tx=0x7f67b80047c0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 shutdown_connections 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5609846d2ca0 0x5609846d50b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 --2- 172.21.15.31:0/2679531729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846b8370 0x5609846bfd70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67a37fe700 0 client.26239 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 >> 172.21.15.31:0/2679531729 conn(0x5609846ae890 msgr2=0x5609846b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 shutdown_connections 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 1 -- 172.21.15.31:0/2679531729 wait complete. 2023-12-14T18:30:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.342+0000 7f67d6945b80 20 client.26239 trim_cache size 0 max 16384 2023-12-14T18:30:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 Processor -- start 2023-12-14T18:30:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 -- start start 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560984515430 0x5609846c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 0x5609846bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609846b5840 con 0x5609846bb7a0 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609846be300 con 0x560984515430 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67d6945b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609846bbbc0 con 0x5609846c30f0 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67c8d54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 0x5609846bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.362+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560984515430 0x5609846c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 0x5609846bfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38340/0 (socket says 172.21.15.31:38340) 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560984515430 0x5609846c2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:58428/0 (socket says 172.21.15.31:58428) 2023-12-14T18:30:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 -- 172.21.15.31:0/286951502 learned_addr learned my addr 172.21.15.31:0/286951502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 -- 172.21.15.31:0/286951502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 msgr2=0x5609846bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 -- 172.21.15.31:0/286951502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560984515430 msgr2=0x5609846c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560984515430 0x5609846c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 -- 172.21.15.31:0/286951502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609845d0e60 con 0x5609846bb7a0 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67c8d54700 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 0x5609846bfa70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009390 tx=0x7f67bc0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56875a8bc6772fd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67aaffd700 1 -- 172.21.15.31:0/286951502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc021070 con 0x5609846bb7a0 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67aaffd700 1 -- 172.21.15.31:0/286951502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67bc009460 con 0x5609846bb7a0 2023-12-14T18:30:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.363+0000 7f67aaffd700 1 -- 172.21.15.31:0/286951502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00e040 con 0x5609846bb7a0 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 -- 172.21.15.31:0/286951502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 msgr2=0x5609846bfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 0x5609846bfa70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f67bc009390 tx=0x7f67bc0093c0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 -- 172.21.15.31:0/286951502 shutdown_connections 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609846bb7a0 0x5609846bfa70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560984515430 0x5609846c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 --2- 172.21.15.31:0/286951502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c30f0 0x5609846bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 -- 172.21.15.31:0/286951502 >> 172.21.15.31:0/286951502 conn(0x5609846ae890 msgr2=0x5609846bb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 -- 172.21.15.31:0/286951502 shutdown_connections 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.364+0000 7f67d6945b80 1 -- 172.21.15.31:0/286951502 wait complete. 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 -- start start 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 Processor -- start 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c8d50 0x560984726b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609846ca250 con 0x560984515430 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560984727070 con 0x5609846c30f0 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.365+0000 7f67d6945b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609847271b0 con 0x5609846c8d50 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67d6945b80 10 client.0 _ll_register_callbacks cb 0x7f67b80045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38350/0 (socket says 172.21.15.31:38350) 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 -- 172.21.15.31:0/1265784403 learned_addr learned my addr 172.21.15.31:0/1265784403 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c8d54700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c8d50 0x560984726b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 -- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c8d50 msgr2=0x560984726b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c8d50 0x560984726b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 -- 172.21.15.31:0/1265784403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 msgr2=0x5609846c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 -- 172.21.15.31:0/1265784403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c4071db0 con 0x560984515430 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67c3fff700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c82d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f67b40c9990 tx=0x7f67b40993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f126207c3bca8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67a37fe700 1 -- 172.21.15.31:0/1265784403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4079070 con 0x560984515430 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67a37fe700 1 -- 172.21.15.31:0/1265784403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f67b40cc350 con 0x560984515430 2023-12-14T18:30:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.366+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67b40a16b0 con 0x560984515430 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67a37fe700 1 -- 172.21.15.31:0/1265784403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b40b7040 con 0x560984515430 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 10 client.16395 fetch_fsmap learned FSMap version 26 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5609845b7940 con 0x560984515430 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67a37fe700 1 -- 172.21.15.31:0/1265784403 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b40a16b0 con 0x560984515430 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67a37fe700 1 -- 172.21.15.31:0/1265784403 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 26) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f67b40994c0 con 0x560984515430 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 26 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 20 client.16395 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560984728650 0x56098472aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 4 client.16395 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609846039a0 con 0x560984728650 2023-12-14T18:30:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.367+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560984728650 0x56098472aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.368+0000 7f67a37fe700 10 client.16395 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:49.368+0000 7f67c37fe700 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560984728650 0x56098472aa60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a328a553f65c7ad5 server_cookie=5c1d5570a334eac0 in_seq=0 out_seq=0 2023-12-14T18:30:49.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:30:50.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.051+0000 7f07a7767700 1 -- 172.21.15.31:0/2286914388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a0102be0 msgr2=0x7f07a0103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:50.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.051+0000 7f07a7767700 1 --2- 172.21.15.31:0/2286914388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a0102be0 0x7f07a0103080 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0798009f90 tx=0x7f079800b330 comp rx=0 tx=0).stop 2023-12-14T18:30:50.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.055+0000 7f07a7767700 1 -- 172.21.15.31:0/2286914388 shutdown_connections 2023-12-14T18:30:50.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.055+0000 7f07a7767700 1 --2- 172.21.15.31:0/2286914388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a0102be0 0x7f07a0103080 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.055+0000 7f07a7767700 1 --2- 172.21.15.31:0/2286914388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a01050e0 0x7f07a0067e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.055+0000 7f07a7767700 1 --2- 172.21.15.31:0/2286914388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a0068390 0x7f07a0068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.055+0000 7f07a7767700 1 -- 172.21.15.31:0/2286914388 >> 172.21.15.31:0/2286914388 conn(0x7f07a0075fd0 msgr2=0x7f07a00763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:50.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.061+0000 7f07a7767700 1 -- 172.21.15.31:0/2286914388 shutdown_connections 2023-12-14T18:30:50.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.061+0000 7f07a7767700 1 -- 172.21.15.31:0/2286914388 wait complete. 2023-12-14T18:30:50.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.062+0000 7f07a7767700 1 Processor -- start 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 -- start start 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a0068390 0x7f07a010c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a0102be0 0x7f07a010c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 0x7f07a01b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01b1490 con 0x7f07a01050e0 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a006a540 con 0x7f07a0068390 2023-12-14T18:30:50.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.063+0000 7f07a7767700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a006a700 con 0x7f07a0102be0 2023-12-14T18:30:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.064+0000 7f07a5d04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 0x7f07a01b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.064+0000 7f07a5d04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 0x7f07a01b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38376/0 (socket says 172.21.15.31:38376) 2023-12-14T18:30:50.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.064+0000 7f07a5d04700 1 -- 172.21.15.31:0/904399870 learned_addr learned my addr 172.21.15.31:0/904399870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:50.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.069+0000 7f07a5d04700 1 -- 172.21.15.31:0/904399870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a0102be0 msgr2=0x7f07a010c720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:30:50.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.069+0000 7f07a5d04700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a0102be0 0x7f07a010c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.069+0000 7f07a5d04700 1 -- 172.21.15.31:0/904399870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a0068390 msgr2=0x7f07a010c1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:30:50.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.069+0000 7f07a5d04700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a0068390 0x7f07a010c1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.069+0000 7f07a5d04700 1 -- 172.21.15.31:0/904399870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0798009c40 con 0x7f07a01050e0 2023-12-14T18:30:50.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.069+0000 7f07a5d04700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 0x7f07a01b0f20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0798004c60 tx=0x7f0798004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.072+0000 7f07927fc700 1 -- 172.21.15.31:0/904399870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079800e040 con 0x7f07a01050e0 2023-12-14T18:30:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.072+0000 7f07927fc700 1 -- 172.21.15.31:0/904399870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0798007910 con 0x7f07a01050e0 2023-12-14T18:30:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.072+0000 7f07927fc700 1 -- 172.21.15.31:0/904399870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079801b860 con 0x7f07a01050e0 2023-12-14T18:30:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.072+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a0116950 con 0x7f07a01050e0 2023-12-14T18:30:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.072+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a01167c0 con 0x7f07a01050e0 2023-12-14T18:30:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.080+0000 7f07b38f5b80 1 Processor -- start 2023-12-14T18:30:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.080+0000 7f07927fc700 1 -- 172.21.15.31:0/904399870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f079801d470 con 0x7f07a01050e0 2023-12-14T18:30:50.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 -- start start 2023-12-14T18:30:50.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299b380 0x55e34299b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e3429a6b50 0x55e3429a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3428855e0 con 0x55e34299bc90 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3428f4870 con 0x55e3429a6b50 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.081+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3428edd00 con 0x55e34299b380 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.082+0000 7f07927fc700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f078c0a1400 0x7f078c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.082+0000 7f07927fc700 1 -- 172.21.15.31:0/904399870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f07980ce520 con 0x7f07a01050e0 2023-12-14T18:30:50.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.083+0000 7f07a5d04700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f078c0a1400 0x7f078c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.087+0000 7f07a4d02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299b380 0x55e34299b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.087+0000 7f07a4d02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299b380 0x55e34299b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45080/0 (socket says 172.21.15.31:45080) 2023-12-14T18:30:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.087+0000 7f07a4d02700 1 -- 172.21.15.31:0/3193211149 learned_addr learned my addr 172.21.15.31:0/3193211149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.087+0000 7f07a4d02700 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e3429a6b50 0x55e3429a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.088+0000 7f07a5d04700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f078c0a1400 0x7f078c0a38b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0798097790 tx=0x7f0798009bd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.091+0000 7f07a5503700 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.092+0000 7f07a4d02700 1 -- 172.21.15.31:0/3193211149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299b380 msgr2=0x55e34299b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.092+0000 7f07a4d02700 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299b380 0x55e34299b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.092+0000 7f07a4d02700 1 -- 172.21.15.31:0/3193211149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 msgr2=0x55e3429a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.092+0000 7f07a4d02700 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.092+0000 7f07a4d02700 1 -- 172.21.15.31:0/3193211149 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e34276f3b0 con 0x55e3429a6b50 2023-12-14T18:30:50.097 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:49 smithi204 ceph-mon[43866]: pgmap v93: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 1.54k op/s 2023-12-14T18:30:50.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.098+0000 7f07a4d02700 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e3429a6b50 0x55e3429a8f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f079c0121e0 tx=0x7f079c006960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e60999416857e5e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.099+0000 7f078b7fe700 1 -- 172.21.15.31:0/3193211149 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c012c80 con 0x55e3429a6b50 2023-12-14T18:30:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.099+0000 7f078b7fe700 1 -- 172.21.15.31:0/3193211149 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079c012de0 con 0x55e3429a6b50 2023-12-14T18:30:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.099+0000 7f078b7fe700 1 -- 172.21.15.31:0/3193211149 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c0173c0 con 0x55e3429a6b50 2023-12-14T18:30:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.101+0000 7f07b38f5b80 1 -- 172.21.15.31:0/3193211149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e3429a6b50 msgr2=0x55e3429a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.101+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e3429a6b50 0x55e3429a8f30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f079c0121e0 tx=0x7f079c006960 comp rx=0 tx=0).stop 2023-12-14T18:30:50.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.104+0000 7f07b38f5b80 1 -- 172.21.15.31:0/3193211149 shutdown_connections 2023-12-14T18:30:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.104+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e3429a6b50 0x55e3429a8f30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.104+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299b380 0x55e34299b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.104+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/3193211149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.104+0000 7f07b38f5b80 1 -- 172.21.15.31:0/3193211149 >> 172.21.15.31:0/3193211149 conn(0x55e342991ca0 msgr2=0x55e3429977e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:50.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.108+0000 7f07b38f5b80 1 -- 172.21.15.31:0/3193211149 shutdown_connections 2023-12-14T18:30:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.109+0000 7f07b38f5b80 1 -- 172.21.15.31:0/3193211149 wait complete. 2023-12-14T18:30:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.110+0000 7f07b38f5b80 1 -- start start 2023-12-14T18:30:50.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.110+0000 7f07b38f5b80 1 Processor -- start 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 0x55e3429a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 0x55e3429a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3429a6b50 0x55e3429a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e342889f90 con 0x55e3429a6b50 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e342889c30 con 0x55e34299b380 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3427716a0 con 0x55e34299bc90 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07b38f5b80 10 client.0 _ll_register_callbacks cb 0x55e3428ad910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07a5503700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 0x55e3429a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07a5503700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 0x55e3429a37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45086/0 (socket says 172.21.15.31:45086) 2023-12-14T18:30:50.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.111+0000 7f07a5503700 1 -- 172.21.15.31:0/1739679273 learned_addr learned my addr 172.21.15.31:0/1739679273 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:50.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.112+0000 7f07a4d02700 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3429a6b50 0x55e3429a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.115+0000 7f07a5503700 1 -- 172.21.15.31:0/1739679273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 msgr2=0x55e3429a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:50.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.115+0000 7f07a5503700 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 0x55e3429a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.115+0000 7f07a5503700 1 -- 172.21.15.31:0/1739679273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3429a6b50 msgr2=0x55e3429a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:50.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.115+0000 7f07a5503700 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3429a6b50 0x55e3429a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:50.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.115+0000 7f07a5503700 1 -- 172.21.15.31:0/1739679273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c013040 con 0x55e34299bc90 2023-12-14T18:30:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.120+0000 7f07a5503700 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 0x55e3429a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f079400be30 tx=0x7f079400f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8989676811a474af server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:50.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.120+0000 7f0788ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:50.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.121+0000 7f0788ff9700 1 -- 172.21.15.31:0/1739679273 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0794004a30 con 0x55e34299bc90 2023-12-14T18:30:50.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.121+0000 7f0788ff9700 1 -- 172.21.15.31:0/1739679273 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f079400fe60 con 0x55e34299bc90 2023-12-14T18:30:50.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.121+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e34298f2b0 con 0x55e34299bc90 2023-12-14T18:30:50.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.122+0000 7f0788ff9700 1 -- 172.21.15.31:0/1739679273 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0794004b90 con 0x55e34299bc90 2023-12-14T18:30:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.130+0000 7f07b38f5b80 10 client.26248 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:50.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.130+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e3428b4170 con 0x55e34299bc90 2023-12-14T18:30:50.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f0788ff9700 1 -- 172.21.15.31:0/1739679273 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e34298f2b0 con 0x55e34299bc90 2023-12-14T18:30:50.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f0788ff9700 1 -- 172.21.15.31:0/1739679273 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0794011760 con 0x55e34299bc90 2023-12-14T18:30:50.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f07b38f5b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f07b38f5b80 10 client.26248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f07b38f5b80 20 client.26248 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429b65f0 0x55e3429b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f07b38f5b80 4 client.26248 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:50.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.136+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3428e6ce0 con 0x55e3429b65f0 2023-12-14T18:30:50.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.150+0000 7f07a4d02700 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429b65f0 0x55e3429b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:50.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.154+0000 7f07a4d02700 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429b65f0 0x55e3429b8a00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fcc2d6f0c4a205 server_cookie=156cb9bcd479bf43 in_seq=0 out_seq=0 2023-12-14T18:30:50.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:50.154+0000 7f0788ff9700 10 client.26248 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:50.224 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:49 smithi031 ceph-mon[42085]: pgmap v93: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 1.54k op/s 2023-12-14T18:30:50.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:49 smithi176 ceph-mon[43506]: pgmap v93: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.3 MiB/s wr, 1.54k op/s 2023-12-14T18:30:51.095 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:50 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:51.222 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:50 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:51.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:50 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:52.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:51 smithi204 ceph-mon[43866]: pgmap v94: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 96 op/s 2023-12-14T18:30:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:51 smithi031 ceph-mon[42085]: pgmap v94: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 96 op/s 2023-12-14T18:30:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:51 smithi176 ceph-mon[43506]: pgmap v94: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 96 op/s 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67a37fe700 1 -- 172.21.15.31:0/1265784403 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609846039a0 con 0x560984728650 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67a37fe700 10 client.16395 handle_command_reply: tid=0 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 client.16395 shutdown 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 msgr2=0x5609846c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c82d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f67b40c9990 tx=0x7f67b40993e0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 shutdown_connections 2023-12-14T18:30:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x560984728650 0x56098472aa60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609846c8d50 0x560984726b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560984515430 0x5609846c82d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 --2- 172.21.15.31:0/1265784403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609846c30f0 0x5609846c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67a37fe700 0 client.16395 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 >> 172.21.15.31:0/1265784403 conn(0x5609846ae890 msgr2=0x5609846ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f0788ff9700 1 -- 172.21.15.31:0/1739679273 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e3428e6ce0 con 0x55e3429b65f0 2023-12-14T18:30:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f0788ff9700 10 client.26248 handle_command_reply: tid=0 2023-12-14T18:30:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 client.26248 shutdown 2023-12-14T18:30:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 shutdown_connections 2023-12-14T18:30:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 1 -- 172.21.15.31:0/1265784403 wait complete. 2023-12-14T18:30:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 msgr2=0x55e3429a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 0x55e3429a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f079400be30 tx=0x7f079400f640 comp rx=0 tx=0).stop 2023-12-14T18:30:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 shutdown_connections 2023-12-14T18:30:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429b65f0 0x55e3429b8a00 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e3429a6b50 0x55e3429a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 0x55e3429a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f67d6945b80 20 client.16395 trim_cache size 0 max 16384 2023-12-14T18:30:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f0788ff9700 0 client.26248 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/1739679273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e34299bc90 0x55e3429a37f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.810+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 >> 172.21.15.31:0/1739679273 conn(0x55e342991ca0 msgr2=0x55e3429973d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:30:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:30:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:30:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:30:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 shutdown_connections 2023-12-14T18:30:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f07b38f5b80 1 -- 172.21.15.31:0/1739679273 wait complete. 2023-12-14T18:30:52.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f07b38f5b80 20 client.26248 trim_cache size 0 max 16384 2023-12-14T18:30:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f67ac0a12c0 msgr2=0x7f67ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f67ac0a12c0 0x7f67ac0a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67c41a2370 tx=0x7f67b8009260 comp rx=0 tx=0).stop 2023-12-14T18:30:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 msgr2=0x7f67c41a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 0x7f67c41a0dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67b40099f0 tx=0x7f67b4009d00 comp rx=0 tx=0).stop 2023-12-14T18:30:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 shutdown_connections 2023-12-14T18:30:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f67ac0a12c0 0x7f67ac0a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c410c060 0x7f67c41a5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f67c4107af0 0x7f67c41a0dd0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67b40099f0 tx=0x7f67b4009d00 comp rx=0 tx=0).stop 2023-12-14T18:30:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 --2- 172.21.15.31:0/1760800764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67c41084a0 0x7f67c41a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.811+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 >> 172.21.15.31:0/1760800764 conn(0x7f67c406c510 msgr2=0x7f67c406f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.812+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 shutdown_connections 2023-12-14T18:30:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:30:52.812+0000 7f67ca7b7700 1 -- 172.21.15.31:0/1760800764 wait complete. 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.833+0000 7f07b38f5b80 1 Processor -- start 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.833+0000 7f07b38f5b80 1 -- start start 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 0x55e3429a6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299bc90 0x55e3429a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3429a4be0 con 0x55e34299b380 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3429a1d70 con 0x55e34299bc90 2023-12-14T18:30:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e34299f680 con 0x55e342991ca0 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a5503700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 0x55e3429a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a5503700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 0x55e3429a6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38396/0 (socket says 172.21.15.31:38396) 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45098/0 (socket says 172.21.15.31:45098) 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a5503700 1 -- 172.21.15.31:0/870308385 learned_addr learned my addr 172.21.15.31:0/870308385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299bc90 0x55e3429a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 -- 172.21.15.31:0/870308385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299bc90 msgr2=0x55e3429a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299bc90 0x55e3429a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 -- 172.21.15.31:0/870308385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 msgr2=0x55e3429a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 0x55e3429a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 -- 172.21.15.31:0/870308385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3429a3360 con 0x55e342991ca0 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a5503700 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 0x55e3429a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.834+0000 7f07a4d02700 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429b28f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f079c00b4a0 tx=0x7f079c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f95f4f9ddb52051 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.835+0000 7f078bfff700 1 -- 172.21.15.31:0/870308385 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c022350 con 0x55e342991ca0 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.835+0000 7f078bfff700 1 -- 172.21.15.31:0/870308385 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079c00b4f0 con 0x55e342991ca0 2023-12-14T18:30:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.835+0000 7f078bfff700 1 -- 172.21.15.31:0/870308385 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c006640 con 0x55e342991ca0 2023-12-14T18:30:52.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.835+0000 7f07b38f5b80 1 -- 172.21.15.31:0/870308385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 msgr2=0x55e3429b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429b28f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f079c00b4a0 tx=0x7f079c01a040 comp rx=0 tx=0).stop 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 -- 172.21.15.31:0/870308385 shutdown_connections 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299bc90 0x55e3429a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429b28f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/870308385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299b380 0x55e3429a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 -- 172.21.15.31:0/870308385 >> 172.21.15.31:0/870308385 conn(0x55e3429a9290 msgr2=0x55e342995f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 -- 172.21.15.31:0/870308385 shutdown_connections 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 -- 172.21.15.31:0/870308385 wait complete. 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 -- start start 2023-12-14T18:30:52.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.836+0000 7f07b38f5b80 1 Processor -- start 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429ad540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 0x55e3429ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429abde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3429ac320 con 0x55e34299bc90 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3429ac490 con 0x55e34299b380 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3429ac600 con 0x55e342991ca0 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07b38f5b80 10 client.0 _ll_register_callbacks cb 0x7f079400b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45108/0 (socket says 172.21.15.31:45108) 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 -- 172.21.15.31:0/313002999 learned_addr learned my addr 172.21.15.31:0/313002999 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:52.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 -- 172.21.15.31:0/313002999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 msgr2=0x55e3429ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 0x55e3429ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 -- 172.21.15.31:0/313002999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 msgr2=0x55e3429abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.837+0000 7f07a4d02700 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f07a4d02700 1 -- 172.21.15.31:0/313002999 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3428b4170 con 0x55e342991ca0 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f07a4d02700 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429ad540 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f079c004fd0 tx=0x7f079c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e37b3c5e518748a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f0788ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f0788ff9700 1 -- 172.21.15.31:0/313002999 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c006780 con 0x55e342991ca0 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f0788ff9700 1 -- 172.21.15.31:0/313002999 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f079c0229f0 con 0x55e342991ca0 2023-12-14T18:30:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f079c012de0 con 0x55e342991ca0 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f0788ff9700 1 -- 172.21.15.31:0/313002999 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f079c022bb0 con 0x55e342991ca0 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f07b38f5b80 10 client.26254 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.838+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f079c01c040 con 0x55e342991ca0 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.839+0000 7f0788ff9700 1 -- 172.21.15.31:0/313002999 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f079c012de0 con 0x55e342991ca0 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f0788ff9700 1 -- 172.21.15.31:0/313002999 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f079c004860 con 0x55e342991ca0 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07b38f5b80 10 client.26254 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07b38f5b80 10 client.26254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07b38f5b80 20 client.26254 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429a6b50 0x55e342a0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07b38f5b80 4 client.26254 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:30:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e3428e6a00 con 0x55e3429a6b50 2023-12-14T18:30:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07a5503700 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429a6b50 0x55e342a0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f0788ff9700 10 client.26254 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.840+0000 7f07a5503700 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429a6b50 0x55e342a0d480 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487f50331ad898b4 server_cookie=515e0e52a687810 in_seq=0 out_seq=0 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f0788ff9700 1 -- 172.21.15.31:0/313002999 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e3428e6a00 con 0x55e3429a6b50 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f0788ff9700 10 client.26254 handle_command_reply: tid=0 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 client.26254 shutdown 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 msgr2=0x55e3429ad540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429ad540 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f079c004fd0 tx=0x7f079c01a040 comp rx=0 tx=0).stop 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 shutdown_connections 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e3429a6b50 0x55e342a0d480 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e34299bc90 0x55e3429abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e342991ca0 0x55e3429ad540 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f079c004fd0 tx=0x7f079c01a040 comp rx=0 tx=0).stop 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 --2- 172.21.15.31:0/313002999 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e34299b380 0x55e3429ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f0788ff9700 0 client.26254 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 >> 172.21.15.31:0/313002999 conn(0x55e3429a9290 msgr2=0x55e34299eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 shutdown_connections 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.843+0000 7f07b38f5b80 1 -- 172.21.15.31:0/313002999 wait complete. 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07b38f5b80 20 client.26254 trim_cache size 0 max 16384 2023-12-14T18:30:52.849 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:30:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:30:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:30:52.850 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:30:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f078c0a1400 msgr2=0x7f078c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f078c0a1400 0x7f078c0a38b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0798097790 tx=0x7f0798009bd0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 msgr2=0x7f07a01b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 0x7f07a01b0f20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0798004c60 tx=0x7f0798004c90 comp rx=0 tx=0).stop 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 shutdown_connections 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a01050e0 0x7f07a01b0f20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07a0068390 0x7f07a010c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f078c0a1400 0x7f078c0a38b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 --2- 172.21.15.31:0/904399870 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a0102be0 0x7f07a010c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 >> 172.21.15.31:0/904399870 conn(0x7f07a0075fd0 msgr2=0x7f07a0101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.844+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 shutdown_connections 2023-12-14T18:30:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:52.845+0000 7f07a7767700 1 -- 172.21.15.31:0/904399870 wait complete. 2023-12-14T18:30:52.863 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:30:52.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:30:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 -- 172.21.15.31:0/3647563338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ffdd0 msgr2=0x7f3228103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 --2- 172.21.15.31:0/3647563338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ffdd0 0x7f3228103d90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f321c0099c0 tx=0x7f321c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 -- 172.21.15.31:0/3647563338 shutdown_connections 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 --2- 172.21.15.31:0/3647563338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3228104470 0x7f322810c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 --2- 172.21.15.31:0/3647563338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32280ff430 0x7f32280ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 --2- 172.21.15.31:0/3647563338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ffdd0 0x7f3228103d90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 -- 172.21.15.31:0/3647563338 >> 172.21.15.31:0/3647563338 conn(0x7f3228076370 msgr2=0x7f3228076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 -- 172.21.15.31:0/3647563338 shutdown_connections 2023-12-14T18:30:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.036+0000 7f322d487700 1 -- 172.21.15.31:0/3647563338 wait complete. 2023-12-14T18:30:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 Processor -- start 2023-12-14T18:30:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 -- start start 2023-12-14T18:30:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ff430 0x7f32281963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 0x7f32281968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3228104470 0x7f322819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819b390 con 0x7f32280ff430 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819b500 con 0x7f32280ffdd0 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.037+0000 7f322d487700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819b6c0 con 0x7f3228104470 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 0x7f32281968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 0x7f32281968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:58506/0 (socket says 172.21.15.31:58506) 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32277fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3228104470 0x7f322819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 -- 172.21.15.31:0/3560112081 learned_addr learned my addr 172.21.15.31:0/3560112081 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f3226ffd700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ff430 0x7f32281963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 -- 172.21.15.31:0/3560112081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3228104470 msgr2=0x7f322819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3228104470 0x7f322819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 -- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ff430 msgr2=0x7f32281963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ff430 0x7f32281963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 -- 172.21.15.31:0/3560112081 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0095f0 con 0x7f32280ffdd0 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f3226ffd700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ff430 0x7f32281963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.038+0000 7f32267fc700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 0x7f32281968f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f321c009f50 tx=0x7f321c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.039+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c003ac0 con 0x7f32280ffdd0 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.039+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321c023070 con 0x7f32280ffdd0 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.039+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c01fdb0 con 0x7f32280ffdd0 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.039+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3228110340 con 0x7f32280ffdd0 2023-12-14T18:30:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.039+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32281101b0 con 0x7f32280ffdd0 2023-12-14T18:30:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.040+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32281104d0 con 0x7f32280ffdd0 2023-12-14T18:30:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.042+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f321c025030 con 0x7f32280ffdd0 2023-12-14T18:30:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.042+0000 7f320ffff700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f32100a1340 0x7f32100a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.043+0000 7f3226ffd700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f32100a1340 0x7f32100a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.043+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f321c0d25f0 con 0x7f32280ffdd0 2023-12-14T18:30:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.043+0000 7f3226ffd700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f32100a1340 0x7f32100a37f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f321800bc10 tx=0x7f321800b5f0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.047+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f321c09b810 con 0x7f32280ffdd0 2023-12-14T18:30:53.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.377+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3228067e40 con 0x7f32280ffdd0 2023-12-14T18:30:53.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.378+0000 7f320ffff700 1 -- 172.21.15.31:0/3560112081 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10819 (secure 0 0 0) 0x7f321c09ee30 con 0x7f32280ffdd0 2023-12-14T18:30:53.380 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:30:53.380 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:30:49.741392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f32100a1340 msgr2=0x7f32100a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f32100a1340 0x7f32100a37f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f321800bc10 tx=0x7f321800b5f0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 msgr2=0x7f32281968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 0x7f32281968f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f321c009f50 tx=0x7f321c005ef0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 shutdown_connections 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f32100a1340 0x7f32100a37f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3228104470 0x7f322819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32280ff430 0x7f32281963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 --2- 172.21.15.31:0/3560112081 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32280ffdd0 0x7f32281968f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 >> 172.21.15.31:0/3560112081 conn(0x7f3228076370 msgr2=0x7f32280fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.381+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 shutdown_connections 2023-12-14T18:30:53.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.383+0000 7f322d487700 1 -- 172.21.15.31:0/3560112081 wait complete. 2023-12-14T18:30:53.386 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-12-14T18:30:53.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:30:53.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 -- 172.21.15.31:0/3080290202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 msgr2=0x7f6e3410d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 --2- 172.21.15.31:0/3080290202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 0x7f6e3410d0d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c00b0d0 tx=0x7f6e2c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 -- 172.21.15.31:0/3080290202 shutdown_connections 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 --2- 172.21.15.31:0/3080290202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 0x7f6e3410d0d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 --2- 172.21.15.31:0/3080290202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e34101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 --2- 172.21.15.31:0/3080290202 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 0x7f6e34104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 -- 172.21.15.31:0/3080290202 >> 172.21.15.31:0/3080290202 conn(0x7f6e34077570 msgr2=0x7f6e34077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 -- 172.21.15.31:0/3080290202 shutdown_connections 2023-12-14T18:30:53.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.561+0000 7f6e39885700 1 -- 172.21.15.31:0/3080290202 wait complete. 2023-12-14T18:30:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.562+0000 7f6e39885700 1 Processor -- start 2023-12-14T18:30:53.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 -- start start 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e341a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 0x7f6e341a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 0x7f6e3410bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3410c530 con 0x7f6e341015f0 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3410c6a0 con 0x7f6e34100c50 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e39885700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3410c860 con 0x7f6e34105550 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e327fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 0x7f6e341a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e341a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e327fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 0x7f6e341a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38444/0 (socket says 172.21.15.31:38444) 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e341a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:58516/0 (socket says 172.21.15.31:58516) 2023-12-14T18:30:53.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.563+0000 7f6e327fc700 1 -- 172.21.15.31:0/54123770 learned_addr learned my addr 172.21.15.31:0/54123770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.564+0000 7f6e32ffd700 1 -- 172.21.15.31:0/54123770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 msgr2=0x7f6e3410bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.564+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 0x7f6e3410bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.564+0000 7f6e32ffd700 1 -- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 msgr2=0x7f6e341a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.564+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 0x7f6e341a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.564+0000 7f6e32ffd700 1 -- 172.21.15.31:0/54123770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24009520 con 0x7f6e34100c50 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.565+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e341a1720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e9f0 tx=0x7f6e2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.565+0000 7f6e38883700 1 -- 172.21.15.31:0/54123770 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400cb70 con 0x7f6e34100c50 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.565+0000 7f6e38883700 1 -- 172.21.15.31:0/54123770 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e240044e0 con 0x7f6e34100c50 2023-12-14T18:30:53.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.565+0000 7f6e38883700 1 -- 172.21.15.31:0/54123770 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24003d50 con 0x7f6e34100c50 2023-12-14T18:30:53.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.565+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c009d00 con 0x7f6e34100c50 2023-12-14T18:30:53.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.565+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34110370 con 0x7f6e34100c50 2023-12-14T18:30:53.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.569+0000 7f6e38883700 1 -- 172.21.15.31:0/54123770 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6e24003ed0 con 0x7f6e34100c50 2023-12-14T18:30:53.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.570+0000 7f6e38883700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6e1c0a1340 0x7f6e1c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.570+0000 7f6e38883700 1 -- 172.21.15.31:0/54123770 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6e24014070 con 0x7f6e34100c50 2023-12-14T18:30:53.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.570+0000 7f6e327fc700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6e1c0a1340 0x7f6e1c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.570+0000 7f6e45a13b80 1 Processor -- start 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.570+0000 7f6e327fc700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6e1c0a1340 0x7f6e1c0a37f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e3410d570 tx=0x7f6e28009210 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 -- start start 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87176750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87181610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87181b50 0x557a87183f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a870605e0 con 0x557a87176380 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a870cf870 con 0x557a87181b50 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a870c8d00 con 0x557a87176c90 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87176750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87176750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38456/0 (socket says 172.21.15.31:38456) 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e32ffd700 1 -- 172.21.15.31:0/1230680805 learned_addr learned my addr 172.21.15.31:0/1230680805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e337fe700 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87181b50 0x557a87183f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e327fc700 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87181610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e32ffd700 1 -- 172.21.15.31:0/1230680805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 msgr2=0x557a87181610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87181610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.571+0000 7f6e32ffd700 1 -- 172.21.15.31:0/1230680805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87181b50 msgr2=0x557a87183f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.572+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87181b50 0x557a87183f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.572+0000 7f6e32ffd700 1 -- 172.21.15.31:0/1230680805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a86f4a3b0 con 0x557a87176380 2023-12-14T18:30:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.572+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87176750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6e2409a0c0 tx=0x7f6e24099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f76df1b8ca0a11 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.572+0000 7f6e19ffb700 1 -- 172.21.15.31:0/1230680805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2409e160 con 0x557a87176380 2023-12-14T18:30:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.572+0000 7f6e19ffb700 1 -- 172.21.15.31:0/1230680805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e240a2580 con 0x557a87176380 2023-12-14T18:30:53.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.572+0000 7f6e19ffb700 1 -- 172.21.15.31:0/1230680805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2409a5f0 con 0x557a87176380 2023-12-14T18:30:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 -- 172.21.15.31:0/1230680805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 msgr2=0x557a87176750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87176750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6e2409a0c0 tx=0x7f6e24099d60 comp rx=0 tx=0).stop 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 -- 172.21.15.31:0/1230680805 shutdown_connections 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87181b50 0x557a87183f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87176750 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/1230680805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87181610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.573+0000 7f6e45a13b80 1 -- 172.21.15.31:0/1230680805 >> 172.21.15.31:0/1230680805 conn(0x557a8716cca0 msgr2=0x557a871727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.574+0000 7f6e45a13b80 1 -- 172.21.15.31:0/1230680805 shutdown_connections 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.574+0000 7f6e45a13b80 1 -- 172.21.15.31:0/1230680805 wait complete. 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.574+0000 7f6e45a13b80 1 -- start start 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.574+0000 7f6e45a13b80 1 Processor -- start 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a871814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a8717e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87181b50 0x557a8717ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87064f90 con 0x557a87176380 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87064c30 con 0x557a87176c90 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a86f4c6a0 con 0x557a87181b50 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e45a13b80 10 client.0 _ll_register_callbacks cb 0x557a87088910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a871814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a871814f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38468/0 (socket says 172.21.15.31:38468) 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e32ffd700 1 -- 172.21.15.31:0/4111254146 learned_addr learned my addr 172.21.15.31:0/4111254146 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e327fc700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a8717e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e337fe700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87181b50 0x557a8717ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e327fc700 1 -- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87181b50 msgr2=0x557a8717ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e327fc700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87181b50 0x557a8717ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e327fc700 1 -- 172.21.15.31:0/4111254146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 msgr2=0x557a871814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e327fc700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a871814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e327fc700 1 -- 172.21.15.31:0/4111254146 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34110370 con 0x557a87176c90 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.575+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a871814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.576+0000 7f6e327fc700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a8717e810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x557a87173b90 tx=0x7f6e28004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a882811b7e79b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.576+0000 7f6e12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.576+0000 7f6e12ffd700 1 -- 172.21.15.31:0/4111254146 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28015af0 con 0x557a87176c90 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a8716a2b0 con 0x557a87176c90 2023-12-14T18:30:53.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e12ffd700 1 -- 172.21.15.31:0/4111254146 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6e28015c50 con 0x557a87176c90 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e12ffd700 1 -- 172.21.15.31:0/4111254146 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e28015e10 con 0x557a87176c90 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 10 client.26261 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557a8708f170 con 0x557a87176c90 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e12ffd700 1 -- 172.21.15.31:0/4111254146 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a8716a2b0 con 0x557a87176c90 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e12ffd700 1 -- 172.21.15.31:0/4111254146 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6e28006760 con 0x557a87176c90 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 10 client.26261 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 10 client.26261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 20 client.26261 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a871915c0 0x557a871939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 4 client.26261 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:53.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.577+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a870c1ce0 con 0x557a871915c0 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.578+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a871915c0 0x557a871939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.578+0000 7f6e12ffd700 10 client.26261 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.578+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a871915c0 0x557a871939d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de531b67cb0f989a server_cookie=2cccdf52c9807699 in_seq=0 out_seq=0 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.579+0000 7f6e12ffd700 1 -- 172.21.15.31:0/4111254146 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557a870c1ce0 con 0x557a871915c0 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.579+0000 7f6e12ffd700 10 client.26261 handle_command_reply: tid=0 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.579+0000 7f6e45a13b80 1 client.26261 shutdown 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.579+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 msgr2=0x557a8717e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.579+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a8717e810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x557a87173b90 tx=0x7f6e28004860 comp rx=0 tx=0).stop 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 shutdown_connections 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a871915c0 0x557a871939d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87181b50 0x557a8717ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a871814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/4111254146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a8717e810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e12ffd700 0 client.26261 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 >> 172.21.15.31:0/4111254146 conn(0x557a8716cca0 msgr2=0x557a87172370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 shutdown_connections 2023-12-14T18:30:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 1 -- 172.21.15.31:0/4111254146 wait complete. 2023-12-14T18:30:53.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.580+0000 7f6e45a13b80 20 client.26261 trim_cache size 0 max 16384 2023-12-14T18:30:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 Processor -- start 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 -- start start 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 0x557a87179270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87170f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a87179ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87177060 con 0x557a87176380 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87177390 con 0x557a87176c90 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.596+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a8718eca0 con 0x557a8716cca0 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 0x557a87179270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e337fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a87179ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 0x557a87179270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45156/0 (socket says 172.21.15.31:45156) 2023-12-14T18:30:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e337fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a87179ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:58546/0 (socket says 172.21.15.31:58546) 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 -- 172.21.15.31:0/76282196 learned_addr learned my addr 172.21.15.31:0/76282196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e327fc700 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87170f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 -- 172.21.15.31:0/76282196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 msgr2=0x557a87179ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a87179ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 -- 172.21.15.31:0/76282196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 msgr2=0x557a87170f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87170f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 -- 172.21.15.31:0/76282196 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a8708eec0 con 0x557a8716cca0 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 0x557a87179270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005cb0 tx=0x7f6e24005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fdab39adf220732 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e1a7fc700 1 -- 172.21.15.31:0/76282196 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240984d0 con 0x557a8716cca0 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.597+0000 7f6e1a7fc700 1 -- 172.21.15.31:0/76282196 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e240cd160 con 0x557a8716cca0 2023-12-14T18:30:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e1a7fc700 1 -- 172.21.15.31:0/76282196 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240ca580 con 0x557a8716cca0 2023-12-14T18:30:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 -- 172.21.15.31:0/76282196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 msgr2=0x557a87179270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 0x557a87179270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e24005cb0 tx=0x7f6e24005c40 comp rx=0 tx=0).stop 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 -- 172.21.15.31:0/76282196 shutdown_connections 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176c90 0x557a87179ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a8716cca0 0x557a87179270 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/76282196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87176380 0x557a87170f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.598+0000 7f6e45a13b80 1 -- 172.21.15.31:0/76282196 >> 172.21.15.31:0/76282196 conn(0x557a87184200 msgr2=0x557a8718d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.599+0000 7f6e45a13b80 1 -- 172.21.15.31:0/76282196 shutdown_connections 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.599+0000 7f6e45a13b80 1 -- 172.21.15.31:0/76282196 wait complete. 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.599+0000 7f6e45a13b80 1 -- start start 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 Processor -- start 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a8716cca0 0x557a87185d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176380 0x557a871862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87186800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a871e4440 con 0x557a8716cca0 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a871e4580 con 0x557a87176380 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a871e46f0 con 0x557a87176c90 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e45a13b80 10 client.0 _ll_register_callbacks cb 0x557a87088910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a8716cca0 0x557a87185d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e32ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a8716cca0 0x557a87185d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38486/0 (socket says 172.21.15.31:38486) 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e32ffd700 1 -- 172.21.15.31:0/3287194793 learned_addr learned my addr 172.21.15.31:0/3287194793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.600+0000 7f6e327fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176380 0x557a871862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e337fe700 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87186800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e327fc700 1 -- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 msgr2=0x557a87186800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e327fc700 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87186800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e327fc700 1 -- 172.21.15.31:0/3287194793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a8716cca0 msgr2=0x557a87185d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e327fc700 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a8716cca0 0x557a87185d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e327fc700 1 -- 172.21.15.31:0/3287194793 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a86f4a3b0 con 0x557a87176380 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e327fc700 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176380 0x557a871862c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e28021000 tx=0x7f6e28015da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0712fa0cc1a9e40 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e12ffd700 1 -- 172.21.15.31:0/3287194793 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28021dd0 con 0x557a87176380 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e12ffd700 1 -- 172.21.15.31:0/3287194793 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6e28035480 con 0x557a87176380 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.601+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e240a2580 con 0x557a87176380 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e12ffd700 1 -- 172.21.15.31:0/3287194793 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e28015760 con 0x557a87176380 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 10 client.26267 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557a870759a0 con 0x557a87176380 2023-12-14T18:30:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e12ffd700 1 -- 172.21.15.31:0/3287194793 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e240a2580 con 0x557a87176380 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e12ffd700 1 -- 172.21.15.31:0/3287194793 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6e2802a560 con 0x557a87176380 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 10 client.26267 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 10 client.26267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 20 client.26267 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a87181b50 0x557a871e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 4 client.26267 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:30:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.602+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557a870c1a00 con 0x557a87181b50 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.603+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a87181b50 0x557a871e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.603+0000 7f6e32ffd700 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a87181b50 0x557a871e7de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6fdf7ac5b2aacda server_cookie=363fd8ab5a551f05 in_seq=0 out_seq=0 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.603+0000 7f6e12ffd700 10 client.26267 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.605+0000 7f6e12ffd700 1 -- 172.21.15.31:0/3287194793 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557a870c1a00 con 0x557a87181b50 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.605+0000 7f6e12ffd700 10 client.26267 handle_command_reply: tid=0 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.605+0000 7f6e45a13b80 1 client.26267 shutdown 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.606+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176380 msgr2=0x557a871862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.606+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176380 0x557a871862c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e28021000 tx=0x7f6e28015da0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 shutdown_connections 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557a87181b50 0x557a871e7de0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87176c90 0x557a87186800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a8716cca0 0x557a87185d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 --2- 172.21.15.31:0/3287194793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87176380 0x557a871862c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 >> 172.21.15.31:0/3287194793 conn(0x557a87184200 msgr2=0x557a87170f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e12ffd700 0 client.26267 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 shutdown_connections 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.607+0000 7f6e45a13b80 1 -- 172.21.15.31:0/3287194793 wait complete. 2023-12-14T18:30:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.608+0000 7f6e45a13b80 20 client.26267 trim_cache size 0 max 16384 2023-12-14T18:30:53.612 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:30:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6e1c0a1340 msgr2=0x7f6e1c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6e1c0a1340 0x7f6e1c0a37f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e3410d570 tx=0x7f6e28009210 comp rx=0 tx=0).stop 2023-12-14T18:30:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 msgr2=0x7f6e341a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e341a1720 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e9f0 tx=0x7f6e2400ed00 comp rx=0 tx=0).stop 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 shutdown_connections 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6e1c0a1340 0x7f6e1c0a37f0 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e3410d570 tx=0x7f6e28009210 comp rx=0 tx=0).stop 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6e34105550 0x7f6e3410bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e34100c50 0x7f6e341a1720 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400e9f0 tx=0x7f6e2400ed00 comp rx=0 tx=0).stop 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 --2- 172.21.15.31:0/54123770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e341015f0 0x7f6e341a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 >> 172.21.15.31:0/54123770 conn(0x7f6e34077570 msgr2=0x7f6e340fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.609+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 shutdown_connections 2023-12-14T18:30:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.610+0000 7f6e39885700 1 -- 172.21.15.31:0/54123770 wait complete. 2023-12-14T18:30:53.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:30:53.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 -- 172.21.15.31:0/3467724413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 msgr2=0x7f9044101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 --2- 172.21.15.31:0/3467724413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f9044101020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f90340099c0 tx=0x7f9034009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 -- 172.21.15.31:0/3467724413 shutdown_connections 2023-12-14T18:30:53.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 --2- 172.21.15.31:0/3467724413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9044105550 0x7f904410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 --2- 172.21.15.31:0/3467724413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f9044101020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 --2- 172.21.15.31:0/3467724413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90441015f0 0x7f9044104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 -- 172.21.15.31:0/3467724413 >> 172.21.15.31:0/3467724413 conn(0x7f9044077570 msgr2=0x7f9044077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.800+0000 7f9048eb3700 1 -- 172.21.15.31:0/3467724413 shutdown_connections 2023-12-14T18:30:53.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.801+0000 7f9048eb3700 1 -- 172.21.15.31:0/3467724413 wait complete. 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 Processor -- start 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 -- start start 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f904406fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90441015f0 0x7f9044070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9044105550 0x7f904406aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904406b570 con 0x7f9044105550 2023-12-14T18:30:53.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904406b6e0 con 0x7f90441015f0 2023-12-14T18:30:53.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f9048eb3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904406b8a0 con 0x7f9044100c50 2023-12-14T18:30:53.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f904406fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.802+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f904406fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45186/0 (socket says 172.21.15.31:45186) 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 -- 172.21.15.31:0/488636933 learned_addr learned my addr 172.21.15.31:0/488636933 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f9042d9d700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9044105550 0x7f904406aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f9041d9b700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90441015f0 0x7f9044070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 -- 172.21.15.31:0/488636933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90441015f0 msgr2=0x7f9044070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90441015f0 0x7f9044070280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 -- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9044105550 msgr2=0x7f904406aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9044105550 0x7f904406aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 -- 172.21.15.31:0/488636933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90340095f0 con 0x7f9044100c50 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f904259c700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f904406fd40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9034006010 tx=0x7f9034005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.803+0000 7f90337fe700 1 -- 172.21.15.31:0/488636933 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903401d070 con 0x7f9044100c50 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.804+0000 7f90337fe700 1 -- 172.21.15.31:0/488636933 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9034004030 con 0x7f9044100c50 2023-12-14T18:30:53.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.804+0000 7f90337fe700 1 -- 172.21.15.31:0/488636933 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034022e10 con 0x7f9044100c50 2023-12-14T18:30:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.804+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90441103f0 con 0x7f9044100c50 2023-12-14T18:30:53.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.804+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9044110260 con 0x7f9044100c50 2023-12-14T18:30:53.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.807+0000 7f90337fe700 1 -- 172.21.15.31:0/488636933 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9034022500 con 0x7f9044100c50 2023-12-14T18:30:53.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.808+0000 7f90337fe700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f902c0a1340 0x7f902c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.808+0000 7f90337fe700 1 -- 172.21.15.31:0/488636933 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f90340ce3c0 con 0x7f9044100c50 2023-12-14T18:30:53.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.808+0000 7f9055041b80 1 Processor -- start 2023-12-14T18:30:53.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.809+0000 7f9041d9b700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f902c0a1340 0x7f902c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.809+0000 7f9055041b80 1 -- start start 2023-12-14T18:30:53.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9055041b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9055041b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9055041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474c3f60 0x55f9474c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9055041b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94739e5e0 con 0x55f9474b4c90 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9055041b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94740d870 con 0x55f9474b4380 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9055041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f947406d00 con 0x55f9474c3f60 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f902c0a1340 0x7f902c0a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f904406c780 tx=0x7f90380091c0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38510/0 (socket says 172.21.15.31:38510) 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 -- 172.21.15.31:0/3757904040 learned_addr learned my addr 172.21.15.31:0/3757904040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9042d9d700 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474c3f60 0x55f9474c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 -- 172.21.15.31:0/3757904040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474c3f60 msgr2=0x55f9474c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474c3f60 0x55f9474c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 -- 172.21.15.31:0/3757904040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 msgr2=0x55f9474b4750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:30:53.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474b4750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.810+0000 7f9041d9b700 1 -- 172.21.15.31:0/3757904040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9472883b0 con 0x55f9474b4c90 2023-12-14T18:30:53.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.811+0000 7f9042d9d700 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474c3f60 0x55f9474c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:30:53.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.811+0000 7f9041d9b700 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c3a20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f903800e920 tx=0x7f9038011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a747a9d206a14c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.815+0000 7f9030ff9700 1 -- 172.21.15.31:0/3757904040 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800ee90 con 0x55f9474b4c90 2023-12-14T18:30:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.815+0000 7f9030ff9700 1 -- 172.21.15.31:0/3757904040 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9038015940 con 0x55f9474b4c90 2023-12-14T18:30:53.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.815+0000 7f9030ff9700 1 -- 172.21.15.31:0/3757904040 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90380139b0 con 0x55f9474b4c90 2023-12-14T18:30:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 -- 172.21.15.31:0/3757904040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 msgr2=0x55f9474c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c3a20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f903800e920 tx=0x7f9038011000 comp rx=0 tx=0).stop 2023-12-14T18:30:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 -- 172.21.15.31:0/3757904040 shutdown_connections 2023-12-14T18:30:53.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474c3f60 0x55f9474c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 --2- 172.21.15.31:0/3757904040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c3a20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.816+0000 7f9055041b80 1 -- 172.21.15.31:0/3757904040 >> 172.21.15.31:0/3757904040 conn(0x55f9474aaca0 msgr2=0x55f9474b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.817+0000 7f9055041b80 1 -- 172.21.15.31:0/3757904040 shutdown_connections 2023-12-14T18:30:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.817+0000 7f9055041b80 1 -- 172.21.15.31:0/3757904040 wait complete. 2023-12-14T18:30:53.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.817+0000 7f9055041b80 1 -- start start 2023-12-14T18:30:53.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.817+0000 7f9055041b80 1 Processor -- start 2023-12-14T18:30:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474c3f60 0x55f9474c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9473a2f90 con 0x55f9474b4c90 2023-12-14T18:30:53.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9473a2c30 con 0x55f9474c3f60 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94728a6a0 con 0x55f9474b4380 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9055041b80 10 client.0 _ll_register_callbacks cb 0x55f9473c6910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45196/0 (socket says 172.21.15.31:45196) 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f904259c700 1 -- 172.21.15.31:0/2981963804 learned_addr learned my addr 172.21.15.31:0/2981963804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.818+0000 7f9041d9b700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f904259c700 1 -- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474c3f60 msgr2=0x55f9474c13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f904259c700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474c3f60 0x55f9474c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f904259c700 1 -- 172.21.15.31:0/2981963804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 msgr2=0x55f9474c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f904259c700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f904259c700 1 -- 172.21.15.31:0/2981963804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903800e690 con 0x55f9474b4380 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f9041d9b700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:30:53.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f904259c700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f90340cbd80 tx=0x7f903409ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0eeebafa7ae5608 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f9029ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f9029ffb700 1 -- 172.21.15.31:0/2981963804 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903409a910 con 0x55f9474b4380 2023-12-14T18:30:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f9029ffb700 1 -- 172.21.15.31:0/2981963804 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f903409f250 con 0x55f9474b4380 2023-12-14T18:30:53.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.819+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9474a82b0 con 0x55f9474b4380 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.820+0000 7f9029ffb700 1 -- 172.21.15.31:0/2981963804 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90340a35e0 con 0x55f9474b4380 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.820+0000 7f9055041b80 10 client.26284 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.820+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9473cd170 con 0x55f9474b4380 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.820+0000 7f9029ffb700 1 -- 172.21.15.31:0/2981963804 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9474a82b0 con 0x55f9474b4380 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.821+0000 7f9029ffb700 1 -- 172.21.15.31:0/2981963804 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90340b3b70 con 0x55f9474b4380 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.821+0000 7f9055041b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.821+0000 7f9055041b80 10 client.26284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.821+0000 7f9055041b80 20 client.26284 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.821+0000 7f9055041b80 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474b9c10 0x55f9474ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.822+0000 7f9055041b80 4 client.26284 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.822+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9473ffce0 con 0x55f9474b9c10 2023-12-14T18:30:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.822+0000 7f9041d9b700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474b9c10 0x55f9474ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.822+0000 7f9041d9b700 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474b9c10 0x55f9474ba010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d9a9e1a2d9aa9c6 server_cookie=d7d95cad1f44bee7 in_seq=0 out_seq=0 2023-12-14T18:30:53.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.823+0000 7f9029ffb700 10 client.26284 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9029ffb700 1 -- 172.21.15.31:0/2981963804 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f9473ffce0 con 0x55f9474b9c10 2023-12-14T18:30:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9029ffb700 10 client.26284 handle_command_reply: tid=0 2023-12-14T18:30:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 client.26284 shutdown 2023-12-14T18:30:53.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 msgr2=0x55f9474c39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f90340cbd80 tx=0x7f903409ab80 comp rx=0 tx=0).stop 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 shutdown_connections 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474c3f60 0x55f9474c13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474b9c10 0x55f9474ba010 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 --2- 172.21.15.31:0/2981963804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 >> 172.21.15.31:0/2981963804 conn(0x55f9474aaca0 msgr2=0x55f9474b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9029ffb700 0 client.26284 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 shutdown_connections 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 1 -- 172.21.15.31:0/2981963804 wait complete. 2023-12-14T18:30:53.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.824+0000 7f9055041b80 20 client.26284 trim_cache size 0 max 16384 2023-12-14T18:30:53.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.852+0000 7f9055041b80 1 Processor -- start 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.852+0000 7f9055041b80 1 -- start start 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9055041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9055041b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9055041b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9055041b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9474c19d0 con 0x55f9474b4c90 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9055041b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9474b53c0 con 0x55f9474b4380 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9055041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9474bb070 con 0x55f9474aaca0 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474c3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45202/0 (socket says 172.21.15.31:45202) 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f904259c700 1 -- 172.21.15.31:0/1253461230 learned_addr learned my addr 172.21.15.31:0/1253461230 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9041d9b700 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.853+0000 7f9042d9d700 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9042d9d700 1 -- 172.21.15.31:0/1253461230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 msgr2=0x55f9474c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9042d9d700 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9042d9d700 1 -- 172.21.15.31:0/1253461230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 msgr2=0x55f9474c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9042d9d700 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9042d9d700 1 -- 172.21.15.31:0/1253461230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9473ccec0 con 0x55f9474b4c90 2023-12-14T18:30:53.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9041d9b700 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.854+0000 7f9042d9d700 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474df810 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f903c00ae50 tx=0x7f903c00ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba206c3064d7404 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.855+0000 7f90317fa700 1 -- 172.21.15.31:0/1253461230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c009b60 con 0x55f9474b4c90 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.855+0000 7f90317fa700 1 -- 172.21.15.31:0/1253461230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f903c010040 con 0x55f9474b4c90 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.855+0000 7f90317fa700 1 -- 172.21.15.31:0/1253461230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c013350 con 0x55f9474b4c90 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 -- 172.21.15.31:0/1253461230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 msgr2=0x55f9474df810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474df810 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f903c00ae50 tx=0x7f903c00ae80 comp rx=0 tx=0).stop 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 -- 172.21.15.31:0/1253461230 shutdown_connections 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4c90 0x55f9474df810 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 --2- 172.21.15.31:0/1253461230 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4380 0x55f9474c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 -- 172.21.15.31:0/1253461230 >> 172.21.15.31:0/1253461230 conn(0x55f9474c6610 msgr2=0x55f9474ba910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 -- 172.21.15.31:0/1253461230 shutdown_connections 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.856+0000 7f9055041b80 1 -- 172.21.15.31:0/1253461230 wait complete. 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.857+0000 7f9055041b80 1 -- start start 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.857+0000 7f9055041b80 1 Processor -- start 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.857+0000 7f9055041b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.857+0000 7f9055041b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4380 0x55f9474dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.857+0000 7f9055041b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4c90 0x55f947524980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9055041b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f947524ef0 con 0x55f9474b4380 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9055041b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f947525060 con 0x55f9474b4c90 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9055041b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f947525220 con 0x55f9474aaca0 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9055041b80 10 client.0 _ll_register_callbacks cb 0x7f903409aff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474da740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45210/0 (socket says 172.21.15.31:45210) 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 -- 172.21.15.31:0/3812011203 learned_addr learned my addr 172.21.15.31:0/3812011203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9041d9b700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4380 0x55f9474dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9042d9d700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4c90 0x55f947524980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 -- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4c90 msgr2=0x55f947524980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4c90 0x55f947524980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 -- 172.21.15.31:0/3812011203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4380 msgr2=0x55f9474dac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4380 0x55f9474dac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 -- 172.21.15.31:0/3812011203 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9472883b0 con 0x55f9474aaca0 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9042d9d700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4c90 0x55f947524980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:53.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f904259c700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474da740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9034009990 tx=0x7f90340b3da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0348bcb12e3aa5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.858+0000 7f9029ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9029ffb700 1 -- 172.21.15.31:0/3812011203 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340973a0 con 0x55f9474aaca0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9029ffb700 1 -- 172.21.15.31:0/3812011203 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f903409f250 con 0x55f9474aaca0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9038015940 con 0x55f9474aaca0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9029ffb700 1 -- 172.21.15.31:0/3812011203 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90340971b0 con 0x55f9474aaca0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 10 client.26290 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f903c00ab30 con 0x55f9474aaca0 2023-12-14T18:30:53.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9029ffb700 1 -- 172.21.15.31:0/3812011203 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90340c9030 con 0x55f9474aaca0 2023-12-14T18:30:53.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9029ffb700 1 -- 172.21.15.31:0/3812011203 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90340b9450 con 0x55f9474aaca0 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 10 client.26290 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 10 client.26290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 20 client.26290 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474c3f60 0x55f94752e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 4 client.26290 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.859+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f9473ffa00 con 0x55f9474c3f60 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.860+0000 7f9041d9b700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474c3f60 0x55f94752e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.860+0000 7f9041d9b700 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474c3f60 0x55f94752e4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a21d546bdec0c84 server_cookie=e7ddec9e163effa5 in_seq=0 out_seq=0 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.860+0000 7f9029ffb700 10 client.26290 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.861+0000 7f9029ffb700 1 -- 172.21.15.31:0/3812011203 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f9473ffa00 con 0x55f9474c3f60 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.861+0000 7f9029ffb700 10 client.26290 handle_command_reply: tid=0 2023-12-14T18:30:53.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.861+0000 7f9055041b80 1 client.26290 shutdown 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.861+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 msgr2=0x55f9474da740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.861+0000 7f9055041b80 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474da740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9034009990 tx=0x7f90340b3da0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 shutdown_connections 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f9474b4c90 0x55f947524980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f9474c3f60 0x55f94752e4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9474aaca0 0x55f9474da740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 --2- 172.21.15.31:0/3812011203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f9474b4380 0x55f9474dac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 >> 172.21.15.31:0/3812011203 conn(0x55f9474c6610 msgr2=0x55f9474bbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9029ffb700 0 client.26290 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 shutdown_connections 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 1 -- 172.21.15.31:0/3812011203 wait complete. 2023-12-14T18:30:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.862+0000 7f9055041b80 20 client.26290 trim_cache size 0 max 16384 2023-12-14T18:30:53.867 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:30:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f902c0a1340 msgr2=0x7f902c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f902c0a1340 0x7f902c0a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f904406c780 tx=0x7f90380091c0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 msgr2=0x7f904406fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f904406fd40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9034006010 tx=0x7f9034005740 comp rx=0 tx=0).stop 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 shutdown_connections 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9044105550 0x7f904406aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f902c0a1340 0x7f902c0a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9044100c50 0x7f904406fd40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 --2- 172.21.15.31:0/488636933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f90441015f0 0x7f9044070280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 >> 172.21.15.31:0/488636933 conn(0x7f9044077570 msgr2=0x7f90440fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 shutdown_connections 2023-12-14T18:30:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:53.863+0000 7f9048eb3700 1 -- 172.21.15.31:0/488636933 wait complete. 2023-12-14T18:30:53.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:30:53.913 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:53 smithi031 ceph-mon[42085]: pgmap v95: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 96 op/s 2023-12-14T18:30:53.913 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:53 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3560112081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.040+0000 7f1c7f13d700 1 -- 172.21.15.31:0/534206393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 msgr2=0x7f1c78107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.040+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/534206393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78107010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009920 tx=0x7f1c68009c30 comp rx=0 tx=0).stop 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 -- 172.21.15.31:0/534206393 shutdown_connections 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/534206393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781076f0 0x7f1c7810f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/534206393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781026b0 0x7f1c78102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/534206393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78107010 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 -- 172.21.15.31:0/534206393 >> 172.21.15.31:0/534206393 conn(0x7f1c780fdf40 msgr2=0x7f1c78100350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 -- 172.21.15.31:0/534206393 shutdown_connections 2023-12-14T18:30:54.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.041+0000 7f1c7f13d700 1 -- 172.21.15.31:0/534206393 wait complete. 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 Processor -- start 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 -- start start 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 0x7f1c78198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781076f0 0x7f1c7819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7819c5f0 con 0x7f1c78103050 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7819c760 con 0x7f1c781076f0 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7f13d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7819c920 con 0x7f1c781026b0 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 0x7f1c78198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38546/0 (socket says 172.21.15.31:38546) 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 0x7f1c78198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45226/0 (socket says 172.21.15.31:45226) 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c77fff700 1 -- 172.21.15.31:0/2749971979 learned_addr learned my addr 172.21.15.31:0/2749971979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.042+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781076f0 0x7f1c7819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7d6da700 1 -- 172.21.15.31:0/2749971979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 msgr2=0x7f1c78198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 0x7f1c78198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7d6da700 1 -- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 msgr2=0x7f1c78198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7d6da700 1 -- 172.21.15.31:0/2749971979 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c70009d00 con 0x7f1c781076f0 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781076f0 0x7f1c7819bff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009cd0 tx=0x7f1c7000c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c75ffb700 1 -- 172.21.15.31:0/2749971979 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70017070 con 0x7f1c781076f0 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c75ffb700 1 -- 172.21.15.31:0/2749971979 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7000f040 con 0x7f1c781076f0 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c68009520 con 0x7f1c781076f0 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c75ffb700 1 -- 172.21.15.31:0/2749971979 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000e050 con 0x7f1c781076f0 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.043+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 0x7f1c78198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:54.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.044+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c781124c0 con 0x7f1c781076f0 2023-12-14T18:30:54.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.046+0000 7f1c75ffb700 1 -- 172.21.15.31:0/2749971979 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1c70003a00 con 0x7f1c781076f0 2023-12-14T18:30:54.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.047+0000 7f1c75ffb700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1c600a13c0 0x7f1c600a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.047+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1c600a13c0 0x7f1c600a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.047+0000 7f1c75ffb700 1 -- 172.21.15.31:0/2749971979 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1c700cc970 con 0x7f1c781076f0 2023-12-14T18:30:54.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.048+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1c600a13c0 0x7f1c600a3870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009cb0 tx=0x7f1c6c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.048+0000 7f1c8b2cbb80 1 Processor -- start 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 -- start start 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c226e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2279610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c227bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c21585e0 con 0x55d4c226e380 2023-12-14T18:30:54.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c21c7870 con 0x55d4c226ec90 2023-12-14T18:30:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c21c0d00 con 0x55d4c2279b50 2023-12-14T18:30:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c226e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c226e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38550/0 (socket says 172.21.15.31:38550) 2023-12-14T18:30:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7ced9700 1 -- 172.21.15.31:0/2013080925 learned_addr learned my addr 172.21.15.31:0/2013080925 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7d6da700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c227bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c77fff700 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2279610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7d6da700 1 -- 172.21.15.31:0/2013080925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 msgr2=0x55d4c2279610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2279610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7d6da700 1 -- 172.21.15.31:0/2013080925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 msgr2=0x55d4c226e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c226e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.049+0000 7f1c7d6da700 1 -- 172.21.15.31:0/2013080925 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4c20423b0 con 0x55d4c2279b50 2023-12-14T18:30:54.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.050+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c227bf30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c700c9cd0 tx=0x7f1c70095390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc56473d17719d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.050+0000 7f1c5effd700 1 -- 172.21.15.31:0/2013080925 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70099650 con 0x55d4c2279b50 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.050+0000 7f1c5effd700 1 -- 172.21.15.31:0/2013080925 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7009d680 con 0x55d4c2279b50 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.050+0000 7f1c5effd700 1 -- 172.21.15.31:0/2013080925 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c700998b0 con 0x55d4c2279b50 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/2013080925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 msgr2=0x55d4c227bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c227bf30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c700c9cd0 tx=0x7f1c70095390 comp rx=0 tx=0).stop 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/2013080925 shutdown_connections 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c227bf30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c226e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/2013080925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2279610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/2013080925 >> 172.21.15.31:0/2013080925 conn(0x55d4c2264ca0 msgr2=0x55d4c226a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/2013080925 shutdown_connections 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/2013080925 wait complete. 2023-12-14T18:30:54.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.051+0000 7f1c8b2cbb80 1 -- start start 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 Processor -- start 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226ec90 0x55d4c2276830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c2279b50 0x55d4c2276eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c215cf90 con 0x55d4c226e380 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c215cc30 con 0x55d4c2279b50 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c20446a0 con 0x55d4c226ec90 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c8b2cbb80 10 client.0 _ll_register_callbacks cb 0x55d4c2180910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226ec90 0x55d4c2276830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38554/0 (socket says 172.21.15.31:38554) 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c7ced9700 1 -- 172.21.15.31:0/72691781 learned_addr learned my addr 172.21.15.31:0/72691781 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:54.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.052+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c2279b50 0x55d4c2276eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c7ced9700 1 -- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226ec90 msgr2=0x55d4c2276830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226ec90 0x55d4c2276830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c7ced9700 1 -- 172.21.15.31:0/72691781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c2279b50 msgr2=0x55d4c2276eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c2279b50 0x55d4c2276eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c7ced9700 1 -- 172.21.15.31:0/72691781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c781124c0 con 0x55d4c226e380 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c77fff700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226ec90 0x55d4c2276830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c011a50 tx=0x7f1c6c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2425d6ffcff58cbb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c57fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c57fff700 1 -- 172.21.15.31:0/72691781 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c004570 con 0x55d4c226e380 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c57fff700 1 -- 172.21.15.31:0/72691781 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1c6c018070 con 0x55d4c226e380 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4c22622b0 con 0x55d4c226e380 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.053+0000 7f1c57fff700 1 -- 172.21.15.31:0/72691781 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6c012450 con 0x55d4c226e380 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 10 client.16458 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d4c2187170 con 0x55d4c226e380 2023-12-14T18:30:54.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c57fff700 1 -- 172.21.15.31:0/72691781 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4c22622b0 con 0x55d4c226e380 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c57fff700 1 -- 172.21.15.31:0/72691781 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1c6c021980 con 0x55d4c226e380 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 10 client.16458 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 20 client.16458 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c228b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 4 client.16458 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4c21b9ce0 con 0x55d4c2289580 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.054+0000 7f1c77fff700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c228b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.055+0000 7f1c57fff700 10 client.16458 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:54.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.055+0000 7f1c77fff700 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c228b990 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefd4fb535056a01 server_cookie=5a8c09585544608f in_seq=0 out_seq=0 2023-12-14T18:30:54.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c57fff700 1 -- 172.21.15.31:0/72691781 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d4c21b9ce0 con 0x55d4c2289580 2023-12-14T18:30:54.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c57fff700 10 client.16458 handle_command_reply: tid=0 2023-12-14T18:30:54.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 client.16458 shutdown 2023-12-14T18:30:54.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 msgr2=0x55d4c2279510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c011a50 tx=0x7f1c6c0047c0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 shutdown_connections 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c228b990 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c2279b50 0x55d4c2276eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279510 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c57fff700 0 client.16458 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/72691781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226ec90 0x55d4c2276830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 >> 172.21.15.31:0/72691781 conn(0x55d4c2264ca0 msgr2=0x55d4c226a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 shutdown_connections 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.057+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/72691781 wait complete. 2023-12-14T18:30:54.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.058+0000 7f1c8b2cbb80 20 client.16458 trim_cache size 0 max 16384 2023-12-14T18:30:54.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.075+0000 7f1c8b2cbb80 1 Processor -- start 2023-12-14T18:30:54.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.075+0000 7f1c8b2cbb80 1 -- start start 2023-12-14T18:30:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.075+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.075+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2271b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c22720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c226f060 con 0x55d4c226e380 2023-12-14T18:30:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c2286b60 con 0x55d4c226ec90 2023-12-14T18:30:54.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c2274dd0 con 0x55d4c2279b50 2023-12-14T18:30:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2271b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38570/0 (socket says 172.21.15.31:38570) 2023-12-14T18:30:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2271b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:58654/0 (socket says 172.21.15.31:58654) 2023-12-14T18:30:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c7ced9700 1 -- 172.21.15.31:0/1772552730 learned_addr learned my addr 172.21.15.31:0/1772552730 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:54.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c22720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 -- 172.21.15.31:0/1772552730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 msgr2=0x55d4c22720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c22720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 -- 172.21.15.31:0/1772552730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 msgr2=0x55d4c2279040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 -- 172.21.15.31:0/1772552730 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4c2186ec0 con 0x55d4c226ec90 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.076+0000 7f1c77fff700 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2271b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c78199d10 tx=0x7f1c68005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746aeaf55d254b20 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.077+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/1772552730 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68028d40 con 0x55d4c226ec90 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.077+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/1772552730 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6801d040 con 0x55d4c226ec90 2023-12-14T18:30:54.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.077+0000 7f1c5f7fe700 1 -- 172.21.15.31:0/1772552730 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68005280 con 0x55d4c226ec90 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/1772552730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 msgr2=0x55d4c2271b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2271b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c78199d10 tx=0x7f1c68005990 comp rx=0 tx=0).stop 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/1772552730 shutdown_connections 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c2279b50 0x55d4c22720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c226e380 0x55d4c2279040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/1772552730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c2271b70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/1772552730 >> 172.21.15.31:0/1772552730 conn(0x55d4c2264ca0 msgr2=0x55d4c2268f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/1772552730 shutdown_connections 2023-12-14T18:30:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/1772552730 wait complete. 2023-12-14T18:30:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.078+0000 7f1c8b2cbb80 1 -- start start 2023-12-14T18:30:54.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 Processor -- start 2023-12-14T18:30:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 0x55d4c227e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c227eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c2279b50 0x55d4c227f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c22e4d60 con 0x55d4c2279b50 2023-12-14T18:30:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c22e4ed0 con 0x55d4c226ec90 2023-12-14T18:30:54.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4c22d3460 con 0x55d4c226e380 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c8b2cbb80 10 client.0 _ll_register_callbacks cb 0x7f1c6c0043d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.079+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 0x55d4c227e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 0x55d4c227e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45264/0 (socket says 172.21.15.31:45264) 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 -- 172.21.15.31:0/4150860975 learned_addr learned my addr 172.21.15.31:0/4150860975 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7d6da700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c2279b50 0x55d4c227f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c77fff700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c227eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 -- 172.21.15.31:0/4150860975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 msgr2=0x55d4c227eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c227eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 -- 172.21.15.31:0/4150860975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c2279b50 msgr2=0x55d4c227f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c2279b50 0x55d4c227f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 -- 172.21.15.31:0/4150860975 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4c2187170 con 0x55d4c226e380 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c77fff700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c227eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c7ced9700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 0x55d4c227e5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c021d60 tx=0x7f1c6c012af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167f6449b128065 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c57fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c57fff700 1 -- 172.21.15.31:0/4150860975 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c021610 con 0x55d4c226e380 2023-12-14T18:30:54.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.080+0000 7f1c57fff700 1 -- 172.21.15.31:0/4150860975 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1c6c018040 con 0x55d4c226e380 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c7009d680 con 0x55d4c226e380 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c57fff700 1 -- 172.21.15.31:0/4150860975 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6c004550 con 0x55d4c226e380 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 10 client.26308 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1c7810fb00 con 0x55d4c226e380 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c57fff700 1 -- 172.21.15.31:0/4150860975 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c7009d680 con 0x55d4c226e380 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c57fff700 1 -- 172.21.15.31:0/4150860975 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1c6c02a590 con 0x55d4c226e380 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 10 client.26308 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 10 client.26308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 20 client.26308 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c22d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 4 client.26308 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:30:54.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.081+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d4c21b9a00 con 0x55d4c2289580 2023-12-14T18:30:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.082+0000 7f1c77fff700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c22d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.082+0000 7f1c57fff700 10 client.26308 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:54.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.083+0000 7f1c77fff700 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c22d6c60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baec611ffe0a402d server_cookie=da5c81d2752faed2 in_seq=0 out_seq=0 2023-12-14T18:30:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c57fff700 1 -- 172.21.15.31:0/4150860975 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d4c21b9a00 con 0x55d4c2289580 2023-12-14T18:30:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c57fff700 10 client.26308 handle_command_reply: tid=0 2023-12-14T18:30:54.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 client.26308 shutdown 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 msgr2=0x55d4c227e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 0x55d4c227e5b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c021d60 tx=0x7f1c6c012af0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 shutdown_connections 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d4c2289580 0x55d4c22d6c60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d4c2279b50 0x55d4c227f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d4c226e380 0x55d4c227e5b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c57fff700 0 client.26308 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 --2- 172.21.15.31:0/4150860975 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d4c226ec90 0x55d4c227eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 >> 172.21.15.31:0/4150860975 conn(0x55d4c2264ca0 msgr2=0x55d4c2271b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 shutdown_connections 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.086+0000 7f1c8b2cbb80 1 -- 172.21.15.31:0/4150860975 wait complete. 2023-12-14T18:30:54.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c8b2cbb80 20 client.26308 trim_cache size 0 max 16384 2023-12-14T18:30:54.090 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:30:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1c600a13c0 msgr2=0x7f1c600a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1c600a13c0 0x7f1c600a3870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009cb0 tx=0x7f1c6c009260 comp rx=0 tx=0).stop 2023-12-14T18:30:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781076f0 msgr2=0x7f1c7819bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781076f0 0x7f1c7819bff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009cd0 tx=0x7f1c7000c620 comp rx=0 tx=0).stop 2023-12-14T18:30:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 shutdown_connections 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c781076f0 0x7f1c7819bff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1c600a13c0 0x7f1c600a3870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c781026b0 0x7f1c78198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 --2- 172.21.15.31:0/2749971979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c78103050 0x7f1c78198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.087+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 >> 172.21.15.31:0/2749971979 conn(0x7f1c780fdf40 msgr2=0x7f1c78105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.088+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 shutdown_connections 2023-12-14T18:30:54.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:54.088+0000 7f1c7f13d700 1 -- 172.21.15.31:0/2749971979 wait complete. 2023-12-14T18:30:54.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:53 smithi204 ceph-mon[43866]: pgmap v95: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 96 op/s 2023-12-14T18:30:54.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:53 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3560112081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:30:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:53 smithi176 ceph-mon[43506]: pgmap v95: 129 pgs: 129 active+clean; 55 MiB data, 282 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.8 MiB/s wr, 96 op/s 2023-12-14T18:30:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:53 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3560112081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:30:55.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:30:55.109 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 -- 172.21.15.31:0/3514722002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 msgr2=0x7fedac111c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3514722002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac111c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feda4009990 tx=0x7feda4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 -- 172.21.15.31:0/3514722002 shutdown_connections 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3514722002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 0x7fedac114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3514722002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3514722002 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac111c70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 -- 172.21.15.31:0/3514722002 >> 172.21.15.31:0/3514722002 conn(0x7fedac06b7e0 msgr2=0x7fedac071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 -- 172.21.15.31:0/3514722002 shutdown_connections 2023-12-14T18:30:55.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.279+0000 7fedb2b61700 1 -- 172.21.15.31:0/3514722002 wait complete. 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 Processor -- start 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 -- start start 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac1a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac1a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 0x7fedac19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac19f080 con 0x7fedac108c00 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac19f1f0 con 0x7fedac109510 2023-12-14T18:30:55.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.280+0000 7fedb2b61700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac19f3b0 con 0x7fedac1121b0 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac1a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedabfff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac1a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb10fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 0x7fedac19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac1a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46078/0 (socket says 172.21.15.31:46078) 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedabfff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac1a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:57498/0 (socket says 172.21.15.31:57498) 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 -- 172.21.15.31:0/3605640062 learned_addr learned my addr 172.21.15.31:0/3605640062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 -- 172.21.15.31:0/3605640062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 msgr2=0x7fedac19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 0x7fedac19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 -- 172.21.15.31:0/3605640062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 msgr2=0x7fedac1a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac1a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 -- 172.21.15.31:0/3605640062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0009520 con 0x7fedac108c00 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb10fe700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 0x7fedac19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedabfff700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac1a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.281+0000 7fedb08fd700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac1a49a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feda000eb30 tx=0x7feda000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.282+0000 7feda9ffb700 1 -- 172.21.15.31:0/3605640062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0009880 con 0x7fedac108c00 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.282+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda40095f0 con 0x7fedac108c00 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.282+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1179c0 con 0x7fedac108c00 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.282+0000 7feda9ffb700 1 -- 172.21.15.31:0/3605640062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda000cbd0 con 0x7fedac108c00 2023-12-14T18:30:55.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.282+0000 7feda9ffb700 1 -- 172.21.15.31:0/3605640062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0021690 con 0x7fedac108c00 2023-12-14T18:30:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.284+0000 7feda9ffb700 1 -- 172.21.15.31:0/3605640062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7feda0018430 con 0x7fedac108c00 2023-12-14T18:30:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.285+0000 7feda9ffb700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fed940a12c0 0x7fed940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.285+0000 7feda9ffb700 1 -- 172.21.15.31:0/3605640062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7feda0014070 con 0x7fedac108c00 2023-12-14T18:30:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.285+0000 7fedabfff700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fed940a12c0 0x7fed940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedabfff700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fed940a12c0 0x7fed940a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedac113aa0 tx=0x7feda4005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 Processor -- start 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 -- start start 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2b97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9d10 0x55e29c2c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c1a3660 con 0x55e29c2c4bd0 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2128f0 con 0x55e29c2b9d10 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.286+0000 7fedbecefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c20bd80 con 0x55e29c2b9400 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2b97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2b97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53556/0 (socket says 172.21.15.31:53556) 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 -- 172.21.15.31:0/896256236 learned_addr learned my addr 172.21.15.31:0/896256236 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb10fe700 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedabfff700 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9d10 0x55e29c2c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 -- 172.21.15.31:0/896256236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9d10 msgr2=0x55e29c2c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9d10 0x55e29c2c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 -- 172.21.15.31:0/896256236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2c4bd0 msgr2=0x55e29c2c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 -- 172.21.15.31:0/896256236 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29c08d430 con 0x55e29c2b9400 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fedb08fd700 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2b97d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feda0096c70 tx=0x7feda009a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8f78ff2a6637fb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fed92ffd700 1 -- 172.21.15.31:0/896256236 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0096910 con 0x55e29c2b9400 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fed92ffd700 1 -- 172.21.15.31:0/896256236 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda0096a70 con 0x55e29c2b9400 2023-12-14T18:30:55.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.287+0000 7fed92ffd700 1 -- 172.21.15.31:0/896256236 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda009a6c0 con 0x55e29c2b9400 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 -- 172.21.15.31:0/896256236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 msgr2=0x55e29c2b97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2b97d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7feda0096c70 tx=0x7feda009a300 comp rx=0 tx=0).stop 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 -- 172.21.15.31:0/896256236 shutdown_connections 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2b97d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 --2- 172.21.15.31:0/896256236 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9d10 0x55e29c2c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 -- 172.21.15.31:0/896256236 >> 172.21.15.31:0/896256236 conn(0x55e29c2afd20 msgr2=0x55e29c2b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 -- 172.21.15.31:0/896256236 shutdown_connections 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.288+0000 7fedbecefb80 1 -- 172.21.15.31:0/896256236 wait complete. 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.289+0000 7fedbecefb80 1 -- start start 2023-12-14T18:30:55.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.289+0000 7fedbecefb80 1 Processor -- start 2023-12-14T18:30:55.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.289+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.289+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.289+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedbecefb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c1a8010 con 0x55e29c2b9d10 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedbecefb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c1a7cb0 con 0x55e29c2c4bd0 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedbecefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c08f720 con 0x55e29c2b9400 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedbecefb80 10 client.0 _ll_register_callbacks cb 0x55e29c1cb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2c1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46104/0 (socket says 172.21.15.31:46104) 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 -- 172.21.15.31:0/1418260896 learned_addr learned my addr 172.21.15.31:0/1418260896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedb10fe700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 -- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 msgr2=0x55e29c2c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 -- 172.21.15.31:0/1418260896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 msgr2=0x55e29c2c1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 -- 172.21.15.31:0/1418260896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1179c0 con 0x55e29c2b9d10 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedb08fd700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedabfff700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2c1910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feda40098e0 tx=0x7feda4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00ff0656ca2e429 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fed8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fed8bfff700 1 -- 172.21.15.31:0/1418260896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400f730 con 0x55e29c2b9d10 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e29c2ad330 con 0x55e29c2b9d10 2023-12-14T18:30:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.290+0000 7fed8bfff700 1 -- 172.21.15.31:0/1418260896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7feda400f890 con 0x55e29c2b9d10 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.291+0000 7fed8bfff700 1 -- 172.21.15.31:0/1418260896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feda400fa50 con 0x55e29c2b9d10 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.291+0000 7fedbecefb80 10 client.16476 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.291+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e29c1d21f0 con 0x55e29c2b9d10 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.291+0000 7fed8bfff700 1 -- 172.21.15.31:0/1418260896 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e29c2ad330 con 0x55e29c2b9d10 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fed8bfff700 1 -- 172.21.15.31:0/1418260896 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7feda402f070 con 0x55e29c2b9d10 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedbecefb80 10 client.16476 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedbecefb80 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedbecefb80 20 client.16476 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedbecefb80 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c2d4680 0x55e29c2d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedbecefb80 4 client.16476 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e29c204d60 con 0x55e29c2d4680 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.292+0000 7fedb08fd700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c2d4680 0x55e29c2d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.293+0000 7fed8bfff700 10 client.16476 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.293+0000 7fedb08fd700 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c2d4680 0x55e29c2d6a90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f478b5a46fdb7bf4 server_cookie=85b2122a44c712d2 in_seq=0 out_seq=0 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fed8bfff700 1 -- 172.21.15.31:0/1418260896 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e29c204d60 con 0x55e29c2d4680 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fed8bfff700 10 client.16476 handle_command_reply: tid=0 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 client.16476 shutdown 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 msgr2=0x55e29c2c1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2c1910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feda40098e0 tx=0x7feda4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 shutdown_connections 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c2d4680 0x55e29c2d6a90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fed8bfff700 0 client.16476 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 --2- 172.21.15.31:0/1418260896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2c1910 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 >> 172.21.15.31:0/1418260896 conn(0x55e29c2afd20 msgr2=0x55e29c2b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 shutdown_connections 2023-12-14T18:30:55.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 1 -- 172.21.15.31:0/1418260896 wait complete. 2023-12-14T18:30:55.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.295+0000 7fedbecefb80 20 client.16476 trim_cache size 0 max 16384 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.316+0000 7fedbecefb80 1 Processor -- start 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.316+0000 7fedbecefb80 1 -- start start 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.316+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedbecefb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2c2e80 con 0x55e29c2b9d10 2023-12-14T18:30:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedbecefb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2bfe70 con 0x55e29c2c4bd0 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedbecefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2b4630 con 0x55e29c2b9400 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2bc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53576/0 (socket says 172.21.15.31:53576) 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedb08fd700 1 -- 172.21.15.31:0/3062512196 learned_addr learned my addr 172.21.15.31:0/3062512196 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedb10fe700 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 -- 172.21.15.31:0/3062512196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 msgr2=0x55e29c2bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 -- 172.21.15.31:0/3062512196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 msgr2=0x55e29c2b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 -- 172.21.15.31:0/3062512196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29c2c7280 con 0x55e29c2b9d10 2023-12-14T18:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedb08fd700 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.317+0000 7fedabfff700 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb770 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feda4009990 tx=0x7feda4005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bd007d5de5cb96 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.318+0000 7fed937fe700 1 -- 172.21.15.31:0/3062512196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400bd90 con 0x55e29c2b9d10 2023-12-14T18:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.318+0000 7fed937fe700 1 -- 172.21.15.31:0/3062512196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda401e040 con 0x55e29c2b9d10 2023-12-14T18:30:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.318+0000 7fed937fe700 1 -- 172.21.15.31:0/3062512196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda400fce0 con 0x55e29c2b9d10 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.318+0000 7fedbecefb80 1 -- 172.21.15.31:0/3062512196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 msgr2=0x55e29c2cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.318+0000 7fedbecefb80 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb770 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feda4009990 tx=0x7feda4005ce0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 -- 172.21.15.31:0/3062512196 shutdown_connections 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2c4bd0 0x55e29c2b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2b9400 0x55e29c2bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 --2- 172.21.15.31:0/3062512196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb770 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 -- 172.21.15.31:0/3062512196 >> 172.21.15.31:0/3062512196 conn(0x55e29c2afd20 msgr2=0x55e29c2bcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 -- 172.21.15.31:0/3062512196 shutdown_connections 2023-12-14T18:30:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 -- 172.21.15.31:0/3062512196 wait complete. 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 -- start start 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.319+0000 7fedbecefb80 1 Processor -- start 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 0x55e29c2cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2c9fd0 con 0x55e29c2b9d10 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2ca140 con 0x55e29c2b9400 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29c2ca300 con 0x55e29c2c4bd0 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedbecefb80 10 client.0 _ll_register_callbacks cb 0x7feda4005730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 0x55e29c2cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedabfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53592/0 (socket says 172.21.15.31:53592) 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb08fd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 0x55e29c2cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:57530/0 (socket says 172.21.15.31:57530) 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 -- 172.21.15.31:0/2685898422 learned_addr learned my addr 172.21.15.31:0/2685898422 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 -- 172.21.15.31:0/2685898422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 msgr2=0x55e29c2cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 0x55e29c2cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 -- 172.21.15.31:0/2685898422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 msgr2=0x55e29c2cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb10fe700 1 -- 172.21.15.31:0/2685898422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e29c08d430 con 0x55e29c2c4bd0 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.320+0000 7fedb08fd700 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 0x55e29c2cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedb10fe700 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c9930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed9c004780 tx=0x7fed9c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bdd4e9ead3529bb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fed8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fed8bfff700 1 -- 172.21.15.31:0/2685898422 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c01c070 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feda0096a70 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fed8bfff700 1 -- 172.21.15.31:0/2685898422 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fed9c007b00 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fed8bfff700 1 -- 172.21.15.31:0/2685898422 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9c007cc0 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 10 client.26326 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e29c1d21f0 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fed8bfff700 1 -- 172.21.15.31:0/2685898422 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feda0096a70 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fed8bfff700 1 -- 172.21.15.31:0/2685898422 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fed9c017410 con 0x55e29c2c4bd0 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 10 client.26326 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 10 client.26326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 20 client.26326 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c332170 0x55e29c334580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 4 client.26326 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:30:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.321+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e29c204a80 con 0x55e29c332170 2023-12-14T18:30:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.322+0000 7fedb08fd700 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c332170 0x55e29c334580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.322+0000 7fedb08fd700 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c332170 0x55e29c334580 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576152f80f9843ea server_cookie=6bb3e1512e4d4b22 in_seq=0 out_seq=0 2023-12-14T18:30:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.322+0000 7fed8bfff700 10 client.26326 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fed8bfff700 1 -- 172.21.15.31:0/2685898422 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e29c204a80 con 0x55e29c332170 2023-12-14T18:30:55.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fed8bfff700 10 client.26326 handle_command_reply: tid=0 2023-12-14T18:30:55.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 client.26326 shutdown 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 msgr2=0x55e29c2c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c9930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed9c004780 tx=0x7fed9c0047b0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 shutdown_connections 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e29c332170 0x55e29c334580 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e29c2c4bd0 0x55e29c2c9930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e29c2b9400 0x55e29c2cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fed8bfff700 0 client.26326 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 --2- 172.21.15.31:0/2685898422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e29c2b9d10 0x55e29c2cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 >> 172.21.15.31:0/2685898422 conn(0x55e29c2afd20 msgr2=0x55e29c2b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.531+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 shutdown_connections 2023-12-14T18:30:55.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.532+0000 7fedbecefb80 1 -- 172.21.15.31:0/2685898422 wait complete. 2023-12-14T18:30:55.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.532+0000 7fedbecefb80 20 client.26326 trim_cache size 0 max 16384 2023-12-14T18:30:55.536 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:30:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:30:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "e9c9a305-94ae-47d1-85f0-57f435df8e4b", 2023-12-14T18:30:55.536 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:30:55.536 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:30:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.532+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fed940a12c0 msgr2=0x7fed940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.532+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fed940a12c0 0x7fed940a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedac113aa0 tx=0x7feda4005fd0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.532+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 msgr2=0x7fedac1a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.532+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac1a49a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feda000eb30 tx=0x7feda000ee40 comp rx=0 tx=0).stop 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 shutdown_connections 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fed940a12c0 0x7fed940a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fedac1121b0 0x7fedac19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fedac108c00 0x7fedac1a49a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 --2- 172.21.15.31:0/3605640062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedac109510 0x7fedac1a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 >> 172.21.15.31:0/3605640062 conn(0x7fedac06b7e0 msgr2=0x7fedac107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 shutdown_connections 2023-12-14T18:30:55.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.533+0000 7fedb2b61700 1 -- 172.21.15.31:0/3605640062 wait complete. 2023-12-14T18:30:55.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:30:55.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 -- 172.21.15.31:0/2797534300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0100080 msgr2=0x7ff6d0104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 --2- 172.21.15.31:0/2797534300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0100080 0x7ff6d0104640 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff6c400aa00 tx=0x7ff6c400ad10 comp rx=0 tx=0).stop 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 -- 172.21.15.31:0/2797534300 shutdown_connections 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 --2- 172.21.15.31:0/2797534300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d0104d70 0x7ff6d010b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 --2- 172.21.15.31:0/2797534300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d00ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 --2- 172.21.15.31:0/2797534300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0100080 0x7ff6d0104640 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 -- 172.21.15.31:0/2797534300 >> 172.21.15.31:0/2797534300 conn(0x7ff6d0074a60 msgr2=0x7ff6d0074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 -- 172.21.15.31:0/2797534300 shutdown_connections 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.710+0000 7ff6d6161700 1 -- 172.21.15.31:0/2797534300 wait complete. 2023-12-14T18:30:55.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 Processor -- start 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 -- start start 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d0193fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d0100080 0x7ff6d01944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104d70 0x7ff6d01a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0194d20 con 0x7ff6d0100080 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01a4ab0 con 0x7ff6d0104d70 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6d6161700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01a4bf0 con 0x7ff6d00ff6e0 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d0193fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d0100080 0x7ff6d01944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d0193fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53606/0 (socket says 172.21.15.31:53606) 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/3895362444 learned_addr learned my addr 172.21.15.31:0/3895362444 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.711+0000 7ff6cffff700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104d70 0x7ff6d01a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3895362444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 msgr2=0x7ff6d0193fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d0193fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3895362444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104d70 msgr2=0x7ff6d01a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104d70 0x7ff6d01a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3895362444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0009520 con 0x7ff6d0100080 2023-12-14T18:30:55.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d0193fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d0100080 0x7ff6d01944f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0073280 tx=0x7ff6c4004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ccff9700 1 -- 172.21.15.31:0/3895362444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c400ce60 con 0x7ff6d0100080 2023-12-14T18:30:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c400a630 con 0x7ff6d0100080 2023-12-14T18:30:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ccff9700 1 -- 172.21.15.31:0/3895362444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c400caf0 con 0x7ff6d0100080 2023-12-14T18:30:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6ccff9700 1 -- 172.21.15.31:0/3895362444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4016490 con 0x7ff6d0100080 2023-12-14T18:30:55.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.712+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d010e0b0 con 0x7ff6d0100080 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.715+0000 7ff6ccff9700 1 -- 172.21.15.31:0/3895362444 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff6c400e450 con 0x7ff6d0100080 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.715+0000 7ff6ccff9700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff6b80a12c0 0x7ff6b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.715+0000 7ff6ccff9700 1 -- 172.21.15.31:0/3895362444 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff6c4012070 con 0x7ff6d0100080 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.716+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff6b80a12c0 0x7ff6b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.716+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff6b80a12c0 0x7ff6b80a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009fd0 tx=0x7ff6c0009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 Processor -- start 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 -- start start 2023-12-14T18:30:55.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 0x556748a8a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a95610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a95b50 0x556748a97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567489745e0 con 0x556748a8ac90 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567489e3870 con 0x556748a8a380 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567489dcd00 con 0x556748a95b50 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 0x556748a8a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.717+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a95610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 0x556748a8a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:57558/0 (socket says 172.21.15.31:57558) 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/3629956713 learned_addr learned my addr 172.21.15.31:0/3629956713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cffff700 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a95b50 0x556748a97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/3629956713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a95b50 msgr2=0x556748a97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a95b50 0x556748a97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/3629956713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 msgr2=0x556748a95610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a95610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/3629956713 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55674885e3b0 con 0x556748a8a380 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 0x556748a8a750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000f7d0 tx=0x7ff6c0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f35f63162cfbad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6b5ffb700 1 -- 172.21.15.31:0/3629956713 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c001b630 con 0x556748a8a380 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6b5ffb700 1 -- 172.21.15.31:0/3629956713 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c001bc50 con 0x556748a8a380 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.718+0000 7ff6b5ffb700 1 -- 172.21.15.31:0/3629956713 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c001cc60 con 0x556748a8a380 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3629956713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 msgr2=0x556748a8a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 0x556748a8a750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000f7d0 tx=0x7ff6c0014000 comp rx=0 tx=0).stop 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3629956713 shutdown_connections 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a95b50 0x556748a97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8a380 0x556748a8a750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3629956713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a95610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3629956713 >> 172.21.15.31:0/3629956713 conn(0x556748a80ca0 msgr2=0x556748a867e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3629956713 shutdown_connections 2023-12-14T18:30:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.719+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3629956713 wait complete. 2023-12-14T18:30:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 -- start start 2023-12-14T18:30:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 Processor -- start 2023-12-14T18:30:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a95b50 0x556748a92e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748978f90 con 0x556748a95b50 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748978c30 con 0x556748a8ac90 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567488606a0 con 0x556748a8a380 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6e22efb80 10 client.0 _ll_register_callbacks cb 0x55674899c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a95b50 0x556748a92e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a95b50 0x556748a92e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46160/0 (socket says 172.21.15.31:46160) 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a95540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53636/0 (socket says 172.21.15.31:53636) 2023-12-14T18:30:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6cffff700 1 -- 172.21.15.31:0/2510790882 learned_addr learned my addr 172.21.15.31:0/2510790882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.720+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a927f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:57568/0 (socket says 172.21.15.31:57568) 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6ceffd700 1 -- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 msgr2=0x556748a95540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a95540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6ceffd700 1 -- 172.21.15.31:0/2510790882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a95b50 msgr2=0x556748a92e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a95b50 0x556748a92e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6ceffd700 1 -- 172.21.15.31:0/2510790882 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c0017050 con 0x556748a8ac90 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a927f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40cc400 tx=0x7ff6c400af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad8c1ceadd75d37 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6aeffd700 1 -- 172.21.15.31:0/2510790882 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c409fe40 con 0x556748a8ac90 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6aeffd700 1 -- 172.21.15.31:0/2510790882 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff6c40a41d0 con 0x556748a8ac90 2023-12-14T18:30:55.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556748a7e2b0 con 0x556748a8ac90 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6aeffd700 1 -- 172.21.15.31:0/2510790882 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c40ba1a0 con 0x556748a8ac90 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6e22efb80 10 client.26324 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.721+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567489a3170 con 0x556748a8ac90 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6aeffd700 1 -- 172.21.15.31:0/2510790882 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556748a7e2b0 con 0x556748a8ac90 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6aeffd700 1 -- 172.21.15.31:0/2510790882 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff6c409b420 con 0x556748a8ac90 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6e22efb80 10 client.26324 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6e22efb80 10 client.26324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6e22efb80 20 client.26324 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748aa51e0 0x556748aa75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6e22efb80 4 client.26324 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567489d5ce0 con 0x556748aa51e0 2023-12-14T18:30:55.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.722+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748aa51e0 0x556748aa75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.723+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748aa51e0 0x556748aa75f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129e2f5fd3f82680 server_cookie=1af00f6221443742 in_seq=0 out_seq=0 2023-12-14T18:30:55.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.723+0000 7ff6aeffd700 10 client.26324 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.843+0000 7ff6aeffd700 1 -- 172.21.15.31:0/2510790882 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5567489d5ce0 con 0x556748aa51e0 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.843+0000 7ff6aeffd700 10 client.26324 handle_command_reply: tid=0 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.844+0000 7ff6e22efb80 1 client.26324 shutdown 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 msgr2=0x556748a927f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a927f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40cc400 tx=0x7ff6c400af60 comp rx=0 tx=0).stop 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 shutdown_connections 2023-12-14T18:30:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748aa51e0 0x556748aa75f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a95b50 0x556748a92e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a95540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/2510790882 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a927f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.845+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 >> 172.21.15.31:0/2510790882 conn(0x556748a80ca0 msgr2=0x556748a863d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.846+0000 7ff6aeffd700 0 client.26324 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.846+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 shutdown_connections 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.846+0000 7ff6e22efb80 1 -- 172.21.15.31:0/2510790882 wait complete. 2023-12-14T18:30:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.846+0000 7ff6e22efb80 20 client.26324 trim_cache size 0 max 16384 2023-12-14T18:30:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 Processor -- start 2023-12-14T18:30:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 -- start start 2023-12-14T18:30:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a80ca0 0x556748a927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 0x556748a95000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748a8b390 con 0x556748a8a380 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748a8b5b0 con 0x556748a8ac90 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748a933e0 con 0x556748a80ca0 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 0x556748a95000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a80ca0 0x556748a927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 0x556748a95000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46166/0 (socket says 172.21.15.31:46166) 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3965867600 learned_addr learned my addr 172.21.15.31:0/3965867600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.873+0000 7ff6cffff700 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3965867600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a80ca0 msgr2=0x556748a927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a80ca0 0x556748a927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3965867600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 msgr2=0x556748a95540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a95540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6ceffd700 1 -- 172.21.15.31:0/3965867600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567489a2ec0 con 0x556748a8a380 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6cffff700 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a95540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 0x556748a95000 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4098310 tx=0x7ff6c401f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13b62a83ea9d342 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3965867600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c409c270 con 0x556748a8a380 2023-12-14T18:30:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.874+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3965867600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c40ba1a0 con 0x556748a8a380 2023-12-14T18:30:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.875+0000 7ff6b67fc700 1 -- 172.21.15.31:0/3965867600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c409b560 con 0x556748a8a380 2023-12-14T18:30:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3965867600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 msgr2=0x556748a95000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 0x556748a95000 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4098310 tx=0x7ff6c401f490 comp rx=0 tx=0).stop 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3965867600 shutdown_connections 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a8ac90 0x556748a95540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a80ca0 0x556748a927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/3965867600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8a380 0x556748a95000 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3965867600 >> 172.21.15.31:0/3965867600 conn(0x556748a98200 msgr2=0x556748aa1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3965867600 shutdown_connections 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.876+0000 7ff6e22efb80 1 -- 172.21.15.31:0/3965867600 wait complete. 2023-12-14T18:30:55.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 -- start start 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 Processor -- start 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 0x556748a9c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a9a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a9ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748a9b440 con 0x556748a8ac90 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748a9b5b0 con 0x556748a80ca0 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556748a9b770 con 0x556748a8a380 2023-12-14T18:30:55.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6e22efb80 10 client.0 _ll_register_callbacks cb 0x7ff6c409b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a9ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.877+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 0x556748a9c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a9a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 0x556748a9c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:57586/0 (socket says 172.21.15.31:57586) 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6ceffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a9a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53642/0 (socket says 172.21.15.31:53642) 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/267984345 learned_addr learned my addr 172.21.15.31:0/267984345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 msgr2=0x556748a9a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a9a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/267984345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 msgr2=0x556748a9ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a9ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 -- 172.21.15.31:0/267984345 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c80125c0 con 0x556748a80ca0 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a9a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6cf7fe700 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 0x556748a9c600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff6c001c7f0 tx=0x7ff6c001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5844bf48b447b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:55.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.878+0000 7ff6aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6aeffd700 1 -- 172.21.15.31:0/267984345 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0009350 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6aeffd700 1 -- 172.21.15.31:0/267984345 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7ff6c001c410 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6c001bc50 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6aeffd700 1 -- 172.21.15.31:0/267984345 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c001be40 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6e22efb80 10 client.26330 fetch_fsmap learned FSMap version 27 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5567489a3170 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6aeffd700 1 -- 172.21.15.31:0/267984345 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c001bc50 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.879+0000 7ff6aeffd700 1 -- 172.21.15.31:0/267984345 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff6c001c920 con 0x556748a80ca0 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6e22efb80 10 client.26330 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6e22efb80 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6e22efb80 20 client.26330 populate_metadata read hostname 'smithi031' 2023-12-14T18:30:55.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748a95b50 0x556748b04eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6e22efb80 4 client.26330 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:30:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567489d5a00 con 0x556748a95b50 2023-12-14T18:30:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748a95b50 0x556748b04eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6aeffd700 10 client.26330 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.880+0000 7ff6ceffd700 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748a95b50 0x556748b04eb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec9bfcac377c9eb server_cookie=53f34208bf1d0e50 in_seq=0 out_seq=0 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.881+0000 7ff6aeffd700 1 -- 172.21.15.31:0/267984345 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5567489d5a00 con 0x556748a95b50 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.881+0000 7ff6aeffd700 10 client.26330 handle_command_reply: tid=0 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.881+0000 7ff6e22efb80 1 client.26330 shutdown 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.881+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 msgr2=0x556748a9c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.881+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 0x556748a9c600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff6c001c7f0 tx=0x7ff6c001c820 comp rx=0 tx=0).stop 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 shutdown_connections 2023-12-14T18:30:55.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556748a95b50 0x556748b04eb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556748a8ac90 0x556748a9ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556748a80ca0 0x556748a9c600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6aeffd700 0 client.26330 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 --2- 172.21.15.31:0/267984345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556748a8a380 0x556748a9a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 >> 172.21.15.31:0/267984345 conn(0x556748a98200 msgr2=0x556748a92790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 shutdown_connections 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 1 -- 172.21.15.31:0/267984345 wait complete. 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.882+0000 7ff6e22efb80 20 client.26330 trim_cache size 0 max 16384 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (8779 inodes in the stack)", 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "e9c9a305-94ae-47d1-85f0-57f435df8e4b": { 2023-12-14T18:30:55.886 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e9c9a305-94ae-47d1-85f0-57f435df8e4b", 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff6b80a12c0 msgr2=0x7ff6b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff6b80a12c0 0x7ff6b80a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009fd0 tx=0x7ff6c0009260 comp rx=0 tx=0).stop 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d0100080 msgr2=0x7ff6d01944f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d0100080 0x7ff6d01944f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0073280 tx=0x7ff6c4004980 comp rx=0 tx=0).stop 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 shutdown_connections 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff6d0104d70 0x7ff6d01a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff6b80a12c0 0x7ff6b80a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6d00ff6e0 0x7ff6d0193fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 --2- 172.21.15.31:0/3895362444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d0100080 0x7ff6d01944f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 >> 172.21.15.31:0/3895362444 conn(0x7ff6d0074a60 msgr2=0x7ff6d0102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 shutdown_connections 2023-12-14T18:30:55.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:30:55.883+0000 7ff6d6161700 1 -- 172.21.15.31:0/3895362444 wait complete. 2023-12-14T18:30:55.905 INFO:tasks.cephfs.filesystem:scrub status for tag:e9c9a305-94ae-47d1-85f0-57f435df8e4b - {'path': '/', 'tag': 'e9c9a305-94ae-47d1-85f0-57f435df8e4b', 'options': 'recursive,force'} 2023-12-14T18:30:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:55 smithi204 ceph-mon[43866]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 2.53k op/s 2023-12-14T18:30:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:55 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:30:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:55 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:56.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:55 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:30:56.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:55 smithi031 ceph-mon[42085]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 2.53k op/s 2023-12-14T18:30:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:55 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:30:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:55 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:55 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:30:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:55 smithi176 ceph-mon[43506]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.9 MiB/s wr, 2.53k op/s 2023-12-14T18:30:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:55 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:30:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:55 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:30:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:55 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:30:58.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:57 smithi204 ceph-mon[43866]: pgmap v97: 129 pgs: 129 active+clean; 44 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 2.52k op/s 2023-12-14T18:30:58.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:57 smithi031 ceph-mon[42085]: pgmap v97: 129 pgs: 129 active+clean; 44 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 2.52k op/s 2023-12-14T18:30:58.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:57 smithi176 ceph-mon[43506]: pgmap v97: 129 pgs: 129 active+clean; 44 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 2.52k op/s 2023-12-14T18:31:00.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:30:59 smithi204 ceph-mon[43866]: pgmap v98: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 2.84k op/s 2023-12-14T18:31:00.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:30:59 smithi031 ceph-mon[42085]: pgmap v98: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 2.84k op/s 2023-12-14T18:31:00.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:30:59 smithi176 ceph-mon[43506]: pgmap v98: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 6.2 MiB/s wr, 2.84k op/s 2023-12-14T18:31:02.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:01 smithi204 ceph-mon[43866]: pgmap v99: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 2.76k op/s 2023-12-14T18:31:02.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:01 smithi031 ceph-mon[42085]: pgmap v99: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 2.76k op/s 2023-12-14T18:31:02.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:01 smithi176 ceph-mon[43506]: pgmap v99: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 2.76k op/s 2023-12-14T18:31:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:03 smithi204 ceph-mon[43866]: pgmap v100: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 2.76k op/s 2023-12-14T18:31:04.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:03 smithi031 ceph-mon[42085]: pgmap v100: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 2.76k op/s 2023-12-14T18:31:04.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:03 smithi176 ceph-mon[43506]: pgmap v100: 129 pgs: 129 active+clean; 63 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 2.76k op/s 2023-12-14T18:31:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 -- 172.21.15.31:0/3778680546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ffd60 msgr2=0x7f8850103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 --2- 172.21.15.31:0/3778680546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ffd60 0x7f8850103d20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f88440099c0 tx=0x7f8844009cd0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 -- 172.21.15.31:0/3778680546 shutdown_connections 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 --2- 172.21.15.31:0/3778680546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 0x7f885010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 --2- 172.21.15.31:0/3778680546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ff3c0 0x7f88500ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 --2- 172.21.15.31:0/3778680546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ffd60 0x7f8850103d20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 -- 172.21.15.31:0/3778680546 >> 172.21.15.31:0/3778680546 conn(0x7f88500763a0 msgr2=0x7f88500767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 -- 172.21.15.31:0/3778680546 shutdown_connections 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.947+0000 7f88556aa700 1 -- 172.21.15.31:0/3778680546 wait complete. 2023-12-14T18:31:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.948+0000 7f88556aa700 1 Processor -- start 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 -- start start 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 0x7f88501963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 0x7f88501968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 0x7f885019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019b390 con 0x7f88500ff3c0 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019b500 con 0x7f8850104400 2023-12-14T18:31:05.952 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f88556aa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885019b6c0 con 0x7f88500ffd60 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f884effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 0x7f88501963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 0x7f88501968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 0x7f88501968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:34498/0 (socket says 172.21.15.31:34498) 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f884e7fc700 1 -- 172.21.15.31:0/2406847132 learned_addr learned my addr 172.21.15.31:0/2406847132 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f884effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 0x7f88501963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58998/0 (socket says 172.21.15.31:58998) 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.949+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 0x7f885019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884e7fc700 1 -- 172.21.15.31:0/2406847132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 msgr2=0x7f885019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 0x7f885019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884e7fc700 1 -- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 msgr2=0x7f88501963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 0x7f88501963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884e7fc700 1 -- 172.21.15.31:0/2406847132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440095f0 con 0x7f88500ffd60 2023-12-14T18:31:05.953 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884effd700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 0x7f88501963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 0x7f885019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 0x7f88501968f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88440098e0 tx=0x7f884400bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f8837fff700 1 -- 172.21.15.31:0/2406847132 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884401c070 con 0x7f88500ffd60 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885010f1d0 con 0x7f88500ffd60 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f8837fff700 1 -- 172.21.15.31:0/2406847132 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f884400fcc0 con 0x7f88500ffd60 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f8837fff700 1 -- 172.21.15.31:0/2406847132 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844022ca0 con 0x7f88500ffd60 2023-12-14T18:31:05.954 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.950+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885010f040 con 0x7f88500ffd60 2023-12-14T18:31:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.953+0000 7f8837fff700 1 -- 172.21.15.31:0/2406847132 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8844020070 con 0x7f88500ffd60 2023-12-14T18:31:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.953+0000 7f8837fff700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88380a12c0 0x7f88380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.953+0000 7f8837fff700 1 -- 172.21.15.31:0/2406847132 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f88440cdfc0 con 0x7f88500ffd60 2023-12-14T18:31:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.953+0000 7f884effd700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88380a12c0 0x7f88380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.955 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.954+0000 7f884effd700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88380a12c0 0x7f88380a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8840009730 tx=0x7f8840012040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.954+0000 7f8861838b80 1 Processor -- start 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.954+0000 7f8861838b80 1 -- start start 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f8861838b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711af9fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f8861838b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afa0370 0x55711afa07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f8861838b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afab0f0 0x55711afad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f8861838b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711ae8a580 con 0x55711afa0370 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f8861838b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aef9810 con 0x55711adfd430 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f8861838b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711aef2ca0 con 0x55711afab0f0 2023-12-14T18:31:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711af9fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afa0370 0x55711afa07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711af9fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:49184/0 (socket says 172.21.15.31:49184) 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884effd700 1 -- 172.21.15.31:0/2952793223 learned_addr learned my addr 172.21.15.31:0/2952793223 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afab0f0 0x55711afad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 -- 172.21.15.31:0/2952793223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afab0f0 msgr2=0x55711afad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afab0f0 0x55711afad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 -- 172.21.15.31:0/2952793223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 msgr2=0x55711af9fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711af9fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 -- 172.21.15.31:0/2952793223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55711ad74350 con 0x55711afa0370 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884effd700 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711af9fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afab0f0 0x55711afad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.955+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afa0370 0x55711afa07e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f88440cb320 tx=0x7f884409aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7141fc3c81279b68 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.956+0000 7f88357fa700 1 -- 172.21.15.31:0/2952793223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440b2070 con 0x55711afa0370 2023-12-14T18:31:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.956+0000 7f88357fa700 1 -- 172.21.15.31:0/2952793223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f884409ee80 con 0x55711afa0370 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.956+0000 7f88357fa700 1 -- 172.21.15.31:0/2952793223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884409b220 con 0x55711afa0370 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.956+0000 7f8861838b80 1 -- 172.21.15.31:0/2952793223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afa0370 msgr2=0x55711afa07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.956+0000 7f8861838b80 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afa0370 0x55711afa07e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f88440cb320 tx=0x7f884409aa80 comp rx=0 tx=0).stop 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- 172.21.15.31:0/2952793223 shutdown_connections 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afab0f0 0x55711afad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711af9fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 --2- 172.21.15.31:0/2952793223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afa0370 0x55711afa07e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- 172.21.15.31:0/2952793223 >> 172.21.15.31:0/2952793223 conn(0x55711af96890 msgr2=0x55711af9c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- 172.21.15.31:0/2952793223 shutdown_connections 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- 172.21.15.31:0/2952793223 wait complete. 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- start start 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 Processor -- start 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711afaa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711afa7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afab0f0 0x55711afa8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711ae8ef30 con 0x55711afab0f0 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711ae8ebd0 con 0x55711adfd430 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.957+0000 7f8861838b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711ad76640 con 0x55711afa0370 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f8861838b80 10 client.0 _ll_register_callbacks cb 0x55711aeb28b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711afa7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884effd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711afaa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711afa7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:34510/0 (socket says 172.21.15.31:34510) 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 -- 172.21.15.31:0/2309783542 learned_addr learned my addr 172.21.15.31:0/2309783542 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afab0f0 0x55711afa8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 -- 172.21.15.31:0/2309783542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 msgr2=0x55711afaa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711afaa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 -- 172.21.15.31:0/2309783542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afab0f0 msgr2=0x55711afa8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afab0f0 0x55711afa8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 -- 172.21.15.31:0/2309783542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885010f040 con 0x55711afa0370 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884effd700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711afaa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afab0f0 0x55711afa8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711afa7cf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f884409ab50 tx=0x7f8844003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a1370c61c8b7a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f8832ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.958+0000 7f8832ff5700 1 -- 172.21.15.31:0/2309783542 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440b2070 con 0x55711afa0370 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8832ff5700 1 -- 172.21.15.31:0/2309783542 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f884409b0b0 con 0x55711afa0370 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55711af93ea0 con 0x55711afa0370 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8832ff5700 1 -- 172.21.15.31:0/2309783542 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f884409a610 con 0x55711afa0370 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 10 client.26356 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55711aeb9110 con 0x55711afa0370 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8832ff5700 1 -- 172.21.15.31:0/2309783542 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55711af93ea0 con 0x55711afa0370 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8832ff5700 1 -- 172.21.15.31:0/2309783542 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f884409d070 con 0x55711afa0370 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 10 client.26356 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 20 client.26356 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711afba360 0x55711afbc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 4 client.26356 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55711aeebc80 con 0x55711afba360 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.959+0000 7f884effd700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711afba360 0x55711afbc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.960+0000 7f884effd700 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711afba360 0x55711afbc740 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a42acd2adb5ec35e server_cookie=6ede47fdcb198364 in_seq=0 out_seq=0 2023-12-14T18:31:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.960+0000 7f8832ff5700 10 client.26356 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8832ff5700 1 -- 172.21.15.31:0/2309783542 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55711aeebc80 con 0x55711afba360 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8832ff5700 10 client.26356 handle_command_reply: tid=0 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 client.26356 shutdown 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 msgr2=0x55711afa7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711afa7cf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f884409ab50 tx=0x7f8844003680 comp rx=0 tx=0).stop 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 shutdown_connections 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711afba360 0x55711afbc740 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711afab0f0 0x55711afa8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711adfd430 0x55711afaa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 --2- 172.21.15.31:0/2309783542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711afa7cf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8832ff5700 0 client.26356 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 >> 172.21.15.31:0/2309783542 conn(0x55711af96890 msgr2=0x55711af9bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 shutdown_connections 2023-12-14T18:31:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 1 -- 172.21.15.31:0/2309783542 wait complete. 2023-12-14T18:31:05.969 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.962+0000 7f8861838b80 20 client.26356 trim_cache size 0 max 16384 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.979+0000 7f8861838b80 1 Processor -- start 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.979+0000 7f8861838b80 1 -- start start 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.979+0000 7f8861838b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711afb1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.979+0000 7f8861838b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711af9bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f8861838b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711af9c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f8861838b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711afa19d0 con 0x55711adfd430 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f8861838b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711afad4d0 con 0x55711afab0f0 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f8861838b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711afa10a0 con 0x55711afa0370 2023-12-14T18:31:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711afb1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711afb1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59038/0 (socket says 172.21.15.31:59038) 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 -- 172.21.15.31:0/2404355399 learned_addr learned my addr 172.21.15.31:0/2404355399 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884e7fc700 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711af9bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711af9c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 -- 172.21.15.31:0/2404355399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 msgr2=0x55711af9bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711af9bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 -- 172.21.15.31:0/2404355399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 msgr2=0x55711af9c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711af9c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884effd700 1 -- 172.21.15.31:0/2404355399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55711af980d0 con 0x55711adfd430 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.980+0000 7f884f7fe700 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711af9c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.981+0000 7f884effd700 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711afb1aa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55711afbd050 tx=0x7f8840018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd330cc9d050e63f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.981+0000 7f8835ffb700 1 -- 172.21.15.31:0/2404355399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840025070 con 0x55711adfd430 2023-12-14T18:31:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.981+0000 7f8835ffb700 1 -- 172.21.15.31:0/2404355399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8840026040 con 0x55711adfd430 2023-12-14T18:31:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.981+0000 7f8835ffb700 1 -- 172.21.15.31:0/2404355399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884002c7b0 con 0x55711adfd430 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.981+0000 7f8861838b80 1 -- 172.21.15.31:0/2404355399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 msgr2=0x55711afb1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711afb1aa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55711afbd050 tx=0x7f8840018000 comp rx=0 tx=0).stop 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 -- 172.21.15.31:0/2404355399 shutdown_connections 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711af9c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711afb1aa0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- 172.21.15.31:0/2404355399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711af9bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 -- 172.21.15.31:0/2404355399 >> 172.21.15.31:0/2404355399 conn(0x55711af96890 msgr2=0x55711afa3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 -- 172.21.15.31:0/2404355399 shutdown_connections 2023-12-14T18:31:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 -- 172.21.15.31:0/2404355399 wait complete. 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 -- start start 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 Processor -- start 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711b006540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711b006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.982+0000 7f8861838b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711b007260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f8861838b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711b007970 con 0x55711adfd430 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f8861838b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711b007ae0 con 0x55711afab0f0 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f8861838b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55711b007ca0 con 0x55711afa0370 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f8861838b80 10 client.0 _ll_register_callbacks cb 0x7f8840004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711b006540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884e7fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711b006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711b006540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59044/0 (socket says 172.21.15.31:59044) 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 -- 172.21.15.31:0/1430109672 learned_addr learned my addr 172.21.15.31:0/1430109672 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884f7fe700 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711b007260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 -- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 msgr2=0x55711b006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711b006a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 -- 172.21.15.31:0/1430109672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 msgr2=0x55711b007260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711b007260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 -- 172.21.15.31:0/1430109672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400041b0 con 0x55711adfd430 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f884effd700 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711b006540 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f884002df30 tx=0x7f884002dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff9f64a674b1c8d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f8832ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.983+0000 7f8832ff5700 1 -- 172.21.15.31:0/1430109672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840025040 con 0x55711adfd430 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8832ff5700 1 -- 172.21.15.31:0/1430109672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f884002d450 con 0x55711adfd430 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f884409ee80 con 0x55711adfd430 2023-12-14T18:31:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8832ff5700 1 -- 172.21.15.31:0/1430109672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f884002d610 con 0x55711adfd430 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 10 client.16518 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55711ad20b00 con 0x55711adfd430 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8832ff5700 1 -- 172.21.15.31:0/1430109672 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f884409ee80 con 0x55711adfd430 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8832ff5700 1 -- 172.21.15.31:0/1430109672 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f884001c3f0 con 0x55711adfd430 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 10 client.16518 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 20 client.16518 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711b00f350 0x55711b011760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 4 client.16518 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55711aeeb9a0 con 0x55711b00f350 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.984+0000 7f884e7fc700 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711b00f350 0x55711b011760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.985+0000 7f8832ff5700 10 client.16518 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:05.985+0000 7f884e7fc700 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711b00f350 0x55711b011760 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795f7846f026de44 server_cookie=eecabee6c650f67 in_seq=0 out_seq=0 2023-12-14T18:31:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:05 smithi204 ceph-mon[43866]: pgmap v101: 129 pgs: 129 active+clean; 98 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.5 MiB/s wr, 3.20k op/s 2023-12-14T18:31:06.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:05 smithi031 ceph-mon[42085]: pgmap v101: 129 pgs: 129 active+clean; 98 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.5 MiB/s wr, 3.20k op/s 2023-12-14T18:31:06.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:05 smithi176 ceph-mon[43506]: pgmap v101: 129 pgs: 129 active+clean; 98 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.5 MiB/s wr, 3.20k op/s 2023-12-14T18:31:08.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:07 smithi204 ceph-mon[43866]: pgmap v102: 129 pgs: 129 active+clean; 98 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 769 op/s 2023-12-14T18:31:08.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:07 smithi031 ceph-mon[42085]: pgmap v102: 129 pgs: 129 active+clean; 98 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 769 op/s 2023-12-14T18:31:08.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:07 smithi176 ceph-mon[43506]: pgmap v102: 129 pgs: 129 active+clean; 98 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.4 MiB/s wr, 769 op/s 2023-12-14T18:31:10.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:09 smithi031 ceph-mon[42085]: pgmap v103: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 923 op/s 2023-12-14T18:31:10.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:09 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:31:10.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:09 smithi176 ceph-mon[43506]: pgmap v103: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 923 op/s 2023-12-14T18:31:10.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:09 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:31:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:09 smithi204 ceph-mon[43866]: pgmap v103: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 923 op/s 2023-12-14T18:31:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:09 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:31:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:10 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:10 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:10 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:10 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:10 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:10 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:10 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:10 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:10 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:10 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:10 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:10 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8832ff5700 1 -- 172.21.15.31:0/1430109672 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55711aeeb9a0 con 0x55711b00f350 2023-12-14T18:31:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8832ff5700 10 client.16518 handle_command_reply: tid=0 2023-12-14T18:31:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 client.16518 shutdown 2023-12-14T18:31:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 msgr2=0x55711b006540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711b006540 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f884002df30 tx=0x7f884002dad0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 shutdown_connections 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55711b00f350 0x55711b011760 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55711afab0f0 0x55711b007260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55711adfd430 0x55711b006540 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8832ff5700 0 client.16518 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.664+0000 7f8861838b80 1 --2- 172.21.15.31:0/1430109672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55711afa0370 0x55711b006a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.665+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 >> 172.21.15.31:0/1430109672 conn(0x55711af96890 msgr2=0x55711afa9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.665+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 shutdown_connections 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.665+0000 7f8861838b80 1 -- 172.21.15.31:0/1430109672 wait complete. 2023-12-14T18:31:11.667 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.665+0000 7f8861838b80 20 client.16518 trim_cache size 0 max 16384 2023-12-14T18:31:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:31:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:31:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:31:11.668 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88380a12c0 msgr2=0x7f88380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88380a12c0 0x7f88380a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8840009730 tx=0x7f8840012040 comp rx=0 tx=0).stop 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 msgr2=0x7f88501968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 0x7f88501968f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f88440098e0 tx=0x7f884400bfd0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 shutdown_connections 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8850104400 0x7f885019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.669 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88500ff3c0 0x7f88501963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88380a12c0 0x7f88380a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 --2- 172.21.15.31:0/2406847132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88500ffd60 0x7f88501968f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 >> 172.21.15.31:0/2406847132 conn(0x7f88500763a0 msgr2=0x7f8850068060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 shutdown_connections 2023-12-14T18:31:11.670 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:11.666+0000 7f88556aa700 1 -- 172.21.15.31:0/2406847132 wait complete. 2023-12-14T18:31:12.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:11 smithi031 ceph-mon[42085]: pgmap v104: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 598 op/s 2023-12-14T18:31:12.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:11 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:31:12.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:11 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:31:12.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:11 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:12.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:11 smithi176 ceph-mon[43506]: pgmap v104: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 598 op/s 2023-12-14T18:31:12.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:11 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:31:12.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:11 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:31:12.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:11 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:12.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:11 smithi204 ceph-mon[43866]: pgmap v104: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 598 op/s 2023-12-14T18:31:12.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:11 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:31:12.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:11 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:31:12.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:11 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:31:13.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:12 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:31:13.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:12 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:31:13.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:12 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:31:13.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:12 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:31:13.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:12 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:31:13.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:12 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:31:13.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:12 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:31:13.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:12 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:31:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:12 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:31:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:12 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:31:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:12 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:31:13.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:12 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:31:14.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:13 smithi031 ceph-mon[42085]: pgmap v105: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 598 op/s 2023-12-14T18:31:14.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:13 smithi176 ceph-mon[43506]: pgmap v105: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 598 op/s 2023-12-14T18:31:14.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:13 smithi204 ceph-mon[43866]: pgmap v105: 129 pgs: 129 active+clean; 160 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 598 op/s 2023-12-14T18:31:16.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:15 smithi031 ceph-mon[42085]: pgmap v106: 129 pgs: 129 active+clean; 169 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 1.75k op/s 2023-12-14T18:31:16.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:15 smithi176 ceph-mon[43506]: pgmap v106: 129 pgs: 129 active+clean; 169 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 1.75k op/s 2023-12-14T18:31:16.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:15 smithi204 ceph-mon[43866]: pgmap v106: 129 pgs: 129 active+clean; 169 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 1.75k op/s 2023-12-14T18:31:18.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:17 smithi031 ceph-mon[42085]: pgmap v107: 129 pgs: 129 active+clean; 169 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 1.31k op/s 2023-12-14T18:31:18.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:17 smithi176 ceph-mon[43506]: pgmap v107: 129 pgs: 129 active+clean; 169 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 1.31k op/s 2023-12-14T18:31:18.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:17 smithi204 ceph-mon[43866]: pgmap v107: 129 pgs: 129 active+clean; 169 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.0 MiB/s wr, 1.31k op/s 2023-12-14T18:31:20.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:19 smithi031 ceph-mon[42085]: pgmap v108: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.41k op/s 2023-12-14T18:31:20.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:19 smithi176 ceph-mon[43506]: pgmap v108: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.41k op/s 2023-12-14T18:31:20.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:19 smithi204 ceph-mon[43866]: pgmap v108: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.41k op/s 2023-12-14T18:31:22.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:21 smithi031 ceph-mon[42085]: pgmap v109: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 1.26k op/s 2023-12-14T18:31:22.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:21 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:31:22.241 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:21 smithi176 ceph-mon[43506]: pgmap v109: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 1.26k op/s 2023-12-14T18:31:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:21 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:31:22.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:21 smithi204 ceph-mon[43866]: pgmap v109: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 1.26k op/s 2023-12-14T18:31:22.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:21 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:31:24.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:23 smithi031 ceph-mon[42085]: pgmap v110: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 1.26k op/s 2023-12-14T18:31:24.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:23 smithi176 ceph-mon[43506]: pgmap v110: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 1.26k op/s 2023-12-14T18:31:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:23 smithi204 ceph-mon[43866]: pgmap v110: 129 pgs: 129 active+clean; 213 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.5 MiB/s wr, 1.26k op/s 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 -- 172.21.15.31:0/384843407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9484101c10 msgr2=0x7f948410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 --2- 172.21.15.31:0/384843407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9484101c10 0x7f948410a2e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9478009990 tx=0x7f9478009ca0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 -- 172.21.15.31:0/384843407 shutdown_connections 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 --2- 172.21.15.31:0/384843407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f948410a820 0x7f948410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 --2- 172.21.15.31:0/384843407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9484101300 0x7f94841016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 --2- 172.21.15.31:0/384843407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9484101c10 0x7f948410a2e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.795 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 -- 172.21.15.31:0/384843407 >> 172.21.15.31:0/384843407 conn(0x7f94840763a0 msgr2=0x7f94840767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 -- 172.21.15.31:0/384843407 shutdown_connections 2023-12-14T18:31:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.793+0000 7f948c7fa700 1 -- 172.21.15.31:0/384843407 wait complete. 2023-12-14T18:31:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 Processor -- start 2023-12-14T18:31:24.796 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 -- start start 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 0x7f9484196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9484101c10 0x7f94841968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f948410a820 0x7f948419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419b340 con 0x7f948410a820 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419b4b0 con 0x7f9484101300 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.794+0000 7f948c7fa700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948419b670 con 0x7f9484101c10 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 0x7f9484196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948ad97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f948410a820 0x7f948419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 0x7f9484196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:59206/0 (socket says 172.21.15.31:59206) 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 -- 172.21.15.31:0/1559858371 learned_addr learned my addr 172.21.15.31:0/1559858371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f9489d95700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9484101c10 0x7f94841968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.798 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 -- 172.21.15.31:0/1559858371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9484101c10 msgr2=0x7f94841968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9484101c10 0x7f94841968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 -- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f948410a820 msgr2=0x7f948419ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f948410a820 0x7f948419ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 -- 172.21.15.31:0/1559858371 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780095f0 con 0x7f9484101300 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948ad97700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f948410a820 0x7f948419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948a596700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 0x7f9484196360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f948000e9a0 tx=0x7f948000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f94777fe700 1 -- 172.21.15.31:0/1559858371 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480009790 con 0x7f9484101300 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f94777fe700 1 -- 172.21.15.31:0/1559858371 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9480010430 con 0x7f9484101300 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f94777fe700 1 -- 172.21.15.31:0/1559858371 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480018680 con 0x7f9484101300 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94841102b0 con 0x7f9484101300 2023-12-14T18:31:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.795+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9484110120 con 0x7f9484101300 2023-12-14T18:31:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.798+0000 7f94777fe700 1 -- 172.21.15.31:0/1559858371 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f94800105a0 con 0x7f9484101300 2023-12-14T18:31:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.799+0000 7f94777fe700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f94700a1340 0x7f94700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.799+0000 7f94777fe700 1 -- 172.21.15.31:0/1559858371 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9480014070 con 0x7f9484101300 2023-12-14T18:31:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.799+0000 7f9489d95700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f94700a1340 0x7f94700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.799+0000 7f9489d95700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f94700a1340 0x7f94700a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9484197970 tx=0x7f9478005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 Processor -- start 2023-12-14T18:31:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 -- start start 2023-12-14T18:31:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa53e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa547e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa63500 0x556d2aa658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2a93e580 con 0x556d2a8b1430 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2a9ad810 con 0x556d2aa54370 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9498988b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2a9a6ca0 con 0x556d2aa63500 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f948a596700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa53e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f948a596700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa53e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48512/0 (socket says 172.21.15.31:48512) 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f948a596700 1 -- 172.21.15.31:0/2530339442 learned_addr learned my addr 172.21.15.31:0/2530339442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f948ad97700 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa63500 0x556d2aa658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.800+0000 7f9489d95700 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f948a596700 1 -- 172.21.15.31:0/2530339442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa63500 msgr2=0x556d2aa658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f948a596700 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa63500 0x556d2aa658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f948a596700 1 -- 172.21.15.31:0/2530339442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 msgr2=0x556d2aa547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.804 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f948a596700 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f948a596700 1 -- 172.21.15.31:0/2530339442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d2a828350 con 0x556d2a8b1430 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f948a596700 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa53e30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9480096440 tx=0x7f9480099b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34001608934d7098 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f9474ff9700 1 -- 172.21.15.31:0/2530339442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948009df70 con 0x556d2a8b1430 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f9474ff9700 1 -- 172.21.15.31:0/2530339442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94800a2390 con 0x556d2a8b1430 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.801+0000 7f9474ff9700 1 -- 172.21.15.31:0/2530339442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9480099670 con 0x556d2a8b1430 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 -- 172.21.15.31:0/2530339442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 msgr2=0x556d2aa53e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa53e30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9480096440 tx=0x7f9480099b70 comp rx=0 tx=0).stop 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 -- 172.21.15.31:0/2530339442 shutdown_connections 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa63500 0x556d2aa658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa53e30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 --2- 172.21.15.31:0/2530339442 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 -- 172.21.15.31:0/2530339442 >> 172.21.15.31:0/2530339442 conn(0x556d2aa4a890 msgr2=0x556d2aa50200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 -- 172.21.15.31:0/2530339442 shutdown_connections 2023-12-14T18:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 -- 172.21.15.31:0/2530339442 wait complete. 2023-12-14T18:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 -- start start 2023-12-14T18:31:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.802+0000 7f9498988b80 1 Processor -- start 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa62ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa54370 0x556d2aa60260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2a942f30 con 0x556d2a8b1430 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2a942bd0 con 0x556d2aa63500 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2a82a640 con 0x556d2aa54370 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9498988b80 10 client.0 _ll_register_callbacks cb 0x556d2a9668b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa62ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.809 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa62ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48522/0 (socket says 172.21.15.31:48522) 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 -- 172.21.15.31:0/3057536116 learned_addr learned my addr 172.21.15.31:0/3057536116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f9489d95700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa54370 0x556d2aa60260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948ad97700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 -- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa54370 msgr2=0x556d2aa60260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa54370 0x556d2aa60260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 -- 172.21.15.31:0/3057536116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 msgr2=0x556d2aa608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 -- 172.21.15.31:0/3057536116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9484110120 con 0x556d2a8b1430 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f948a596700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa62ed0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94800cd320 tx=0x7f9480010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c711c1dfa16cfae server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f946dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.803+0000 7f946dffb700 1 -- 172.21.15.31:0/3057536116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948009a3b0 con 0x556d2a8b1430 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f946dffb700 1 -- 172.21.15.31:0/3057536116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f94800a1f00 con 0x556d2a8b1430 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d2aa47ea0 con 0x556d2a8b1430 2023-12-14T18:31:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f946dffb700 1 -- 172.21.15.31:0/3057536116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94800b82a0 con 0x556d2a8b1430 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 10 client.16530 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556d2a96d110 con 0x556d2a8b1430 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f946dffb700 1 -- 172.21.15.31:0/3057536116 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d2aa47ea0 con 0x556d2a8b1430 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f946dffb700 1 -- 172.21.15.31:0/3057536116 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9480094c70 con 0x556d2a8b1430 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 20 client.16530 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aa72ad0 0x556d2aa74eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 4 client.16530 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d2a99fc80 con 0x556d2aa72ad0 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.804+0000 7f9489d95700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aa72ad0 0x556d2aa74eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.805+0000 7f946dffb700 10 client.16530 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.805+0000 7f9489d95700 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aa72ad0 0x556d2aa74eb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240b24189eb1a0da server_cookie=7d423d0b9a649233 in_seq=0 out_seq=0 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f946dffb700 1 -- 172.21.15.31:0/3057536116 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556d2a99fc80 con 0x556d2aa72ad0 2023-12-14T18:31:24.811 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f946dffb700 10 client.16530 handle_command_reply: tid=0 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 client.16530 shutdown 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 msgr2=0x556d2aa62ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa62ed0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94800cd320 tx=0x7f9480010c80 comp rx=0 tx=0).stop 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 shutdown_connections 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aa72ad0 0x556d2aa74eb0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2a8b1430 0x556d2aa62ed0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 --2- 172.21.15.31:0/3057536116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2aa54370 0x556d2aa60260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f946dffb700 0 client.16530 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 >> 172.21.15.31:0/3057536116 conn(0x556d2aa4a890 msgr2=0x556d2aa4fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 shutdown_connections 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 1 -- 172.21.15.31:0/3057536116 wait complete. 2023-12-14T18:31:24.812 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.806+0000 7f9498988b80 20 client.16530 trim_cache size 0 max 16384 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 Processor -- start 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 -- start start 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 0x556d2aa5fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2aa51800 con 0x556d2aa63500 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2aa611c0 con 0x556d2aa54370 2023-12-14T18:31:24.824 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9498988b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2aa5beb0 con 0x556d2a8b1430 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f948ad97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 0x556d2aa5fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa62a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:59236/0 (socket says 172.21.15.31:59236) 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 -- 172.21.15.31:0/3166799604 learned_addr learned my addr 172.21.15.31:0/3166799604 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f948a596700 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f948ad97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 0x556d2aa5fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48534/0 (socket says 172.21.15.31:48534) 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 -- 172.21.15.31:0/3166799604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 msgr2=0x556d2aa6f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 -- 172.21.15.31:0/3166799604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 msgr2=0x556d2aa5fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 0x556d2aa5fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f9489d95700 1 -- 172.21.15.31:0/3166799604 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d2a96ce60 con 0x556d2aa54370 2023-12-14T18:31:24.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.822+0000 7f948ad97700 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 0x556d2aa5fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9489d95700 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa62a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94780098e0 tx=0x7f9478023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9a31c6620e68a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f94757fa700 1 -- 172.21.15.31:0/3166799604 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947802d050 con 0x556d2aa54370 2023-12-14T18:31:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f94757fa700 1 -- 172.21.15.31:0/3166799604 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f947802e040 con 0x556d2aa54370 2023-12-14T18:31:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f94757fa700 1 -- 172.21.15.31:0/3166799604 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478004b40 con 0x556d2aa54370 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 -- 172.21.15.31:0/3166799604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 msgr2=0x556d2aa62a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa62a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94780098e0 tx=0x7f9478023000 comp rx=0 tx=0).stop 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 -- 172.21.15.31:0/3166799604 shutdown_connections 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa63500 0x556d2aa5fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 --2- 172.21.15.31:0/3166799604 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa54370 0x556d2aa62a00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.823+0000 7f9498988b80 1 -- 172.21.15.31:0/3166799604 >> 172.21.15.31:0/3166799604 conn(0x556d2aa4a890 msgr2=0x556d2aa5b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 -- 172.21.15.31:0/3166799604 shutdown_connections 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 -- 172.21.15.31:0/3166799604 wait complete. 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 -- start start 2023-12-14T18:31:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 Processor -- start 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 0x556d2aa684b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2aa69100 con 0x556d2aa54370 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2aa69270 con 0x556d2aa63500 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d2aac6940 con 0x556d2a8b1430 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9498988b80 10 client.0 _ll_register_callbacks cb 0x7f9478005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f948a596700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9489d95700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 0x556d2aa684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9489d95700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 0x556d2aa684b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48538/0 (socket says 172.21.15.31:48538) 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f948a596700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:50266/0 (socket says 172.21.15.31:50266) 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.824+0000 7f9489d95700 1 -- 172.21.15.31:0/2282090207 learned_addr learned my addr 172.21.15.31:0/2282090207 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f948ad97700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9489d95700 1 -- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 msgr2=0x556d2aa6a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9489d95700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9489d95700 1 -- 172.21.15.31:0/2282090207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 msgr2=0x556d2aa689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9489d95700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9489d95700 1 -- 172.21.15.31:0/2282090207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c003a00 con 0x556d2aa54370 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f948ad97700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9489d95700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 0x556d2aa684b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9478005f50 tx=0x7f94780056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb8af73452b9272 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f946dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f946dffb700 1 -- 172.21.15.31:0/2282090207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478029040 con 0x556d2aa54370 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f946dffb700 1 -- 172.21.15.31:0/2282090207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f94780345c0 con 0x556d2aa54370 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94800a2390 con 0x556d2aa54370 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f946dffb700 1 -- 172.21.15.31:0/2282090207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f947802d030 con 0x556d2aa54370 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9498988b80 10 client.16536 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.825+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f948410d900 con 0x556d2aa54370 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f946dffb700 1 -- 172.21.15.31:0/2282090207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9478047080 con 0x556d2aa54370 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f946dffb700 1 -- 172.21.15.31:0/2282090207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f947803b030 con 0x556d2aa54370 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f9498988b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f9498988b80 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f9498988b80 20 client.16536 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f9498988b80 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aac7cc0 0x556d2aaca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f9498988b80 4 client.16536 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556d2a99f9a0 con 0x556d2aac7cc0 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f948a596700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aac7cc0 0x556d2aaca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f946dffb700 10 client.16536 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:24.826+0000 7f948a596700 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aac7cc0 0x556d2aaca0d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ec7e4f0702bc2d server_cookie=e22e6ae7a223178f in_seq=0 out_seq=0 2023-12-14T18:31:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.360+0000 7f946dffb700 1 -- 172.21.15.31:0/2282090207 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556d2a99f9a0 con 0x556d2aac7cc0 2023-12-14T18:31:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.360+0000 7f946dffb700 10 client.16536 handle_command_reply: tid=0 2023-12-14T18:31:25.362 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.360+0000 7f9498988b80 1 client.16536 shutdown 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 msgr2=0x556d2aa684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 0x556d2aa684b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9478005f50 tx=0x7f94780056c0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 shutdown_connections 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556d2aac7cc0 0x556d2aaca0d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556d2aa63500 0x556d2aa689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556d2a8b1430 0x556d2aa6a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 --2- 172.21.15.31:0/2282090207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556d2aa54370 0x556d2aa684b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 >> 172.21.15.31:0/2282090207 conn(0x556d2aa4a890 msgr2=0x556d2aa6f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f946dffb700 0 client.16536 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 shutdown_connections 2023-12-14T18:31:25.363 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 1 -- 172.21.15.31:0/2282090207 wait complete. 2023-12-14T18:31:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.361+0000 7f9498988b80 20 client.16536 trim_cache size 0 max 16384 2023-12-14T18:31:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:31:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:31:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:31:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f94700a1340 msgr2=0x7f94700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f94700a1340 0x7f94700a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9484197970 tx=0x7f9478005fd0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 msgr2=0x7f9484196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 0x7f9484196360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f948000e9a0 tx=0x7f948000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 shutdown_connections 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f948410a820 0x7f948419ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9484101300 0x7f9484196360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f94700a1340 0x7f94700a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 --2- 172.21.15.31:0/1559858371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9484101c10 0x7f94841968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 >> 172.21.15.31:0/1559858371 conn(0x7f94840763a0 msgr2=0x7f94840fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 shutdown_connections 2023-12-14T18:31:25.366 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:25.362+0000 7f948c7fa700 1 -- 172.21.15.31:0/1559858371 wait complete. 2023-12-14T18:31:25.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:31:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/1804705560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a740680c0 msgr2=0x7f9a74068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/1804705560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a740680c0 0x7f9a74068490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009990 tx=0x7f9a70009ca0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/1804705560 shutdown_connections 2023-12-14T18:31:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/1804705560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a741054d0 0x7f9a7410d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/1804705560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a740680c0 0x7f9a74068490 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/1804705560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a74068a60 0x7f9a74104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/1804705560 >> 172.21.15.31:0/1804705560 conn(0x7f9a74075fd0 msgr2=0x7f9a740763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:26.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/1804705560 shutdown_connections 2023-12-14T18:31:26.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.058+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/1804705560 wait complete. 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 Processor -- start 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 -- start start 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 0x7f9a74196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 0x7f9a74196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a741054d0 0x7f9a7419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419b400 con 0x7f9a74068a60 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419b570 con 0x7f9a741054d0 2023-12-14T18:31:26.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7c5cd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7419b730 con 0x7f9a740680c0 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a79b68700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 0x7f9a74196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 0x7f9a74196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a79b68700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 0x7f9a74196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48558/0 (socket says 172.21.15.31:48558) 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 0x7f9a74196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:50296/0 (socket says 172.21.15.31:50296) 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.059+0000 7f9a79b68700 1 -- 172.21.15.31:0/2953568583 learned_addr learned my addr 172.21.15.31:0/2953568583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a7ab6a700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a741054d0 0x7f9a7419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a79b68700 1 -- 172.21.15.31:0/2953568583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 msgr2=0x7f9a74196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 0x7f9a74196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a79b68700 1 -- 172.21.15.31:0/2953568583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a741054d0 msgr2=0x7f9a7419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a741054d0 0x7f9a7419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a79b68700 1 -- 172.21.15.31:0/2953568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700095f0 con 0x7f9a74068a60 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 0x7f9a74196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:26.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 0x7f9a74196960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400e940 tx=0x7f9a6400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a6b7fe700 1 -- 172.21.15.31:0/2953568583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64009730 con 0x7f9a74068a60 2023-12-14T18:31:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a6b7fe700 1 -- 172.21.15.31:0/2953568583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a64010430 con 0x7f9a74068a60 2023-12-14T18:31:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a741102e0 con 0x7f9a74068a60 2023-12-14T18:31:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a6b7fe700 1 -- 172.21.15.31:0/2953568583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400cea0 con 0x7f9a74068a60 2023-12-14T18:31:26.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.060+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a74110150 con 0x7f9a74068a60 2023-12-14T18:31:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.063+0000 7f9a6b7fe700 1 -- 172.21.15.31:0/2953568583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9a64010810 con 0x7f9a74068a60 2023-12-14T18:31:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.063+0000 7f9a6b7fe700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a600a12c0 0x7f9a600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.063+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a600a12c0 0x7f9a600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.063+0000 7f9a6b7fe700 1 -- 172.21.15.31:0/2953568583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9a64014070 con 0x7f9a74068a60 2023-12-14T18:31:26.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.064+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a600a12c0 0x7f9a600a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b2e0 tx=0x7f9a70005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 Processor -- start 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 -- start start 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b47750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b52610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b54f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8a315e0 con 0x55b2f8b47380 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8aa0870 con 0x55b2f8b47c90 2023-12-14T18:31:26.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8a99d00 con 0x55b2f8b52b50 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b47750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b47750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48574/0 (socket says 172.21.15.31:48574) 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a7a369700 1 -- 172.21.15.31:0/1861619637 learned_addr learned my addr 172.21.15.31:0/1861619637 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a79b68700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b52610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.065+0000 7f9a7ab6a700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b54f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7a369700 1 -- 172.21.15.31:0/1861619637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 msgr2=0x55b2f8b54f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7a369700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b54f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7a369700 1 -- 172.21.15.31:0/1861619637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 msgr2=0x55b2f8b52610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7a369700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b52610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7a369700 1 -- 172.21.15.31:0/1861619637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f891b3b0 con 0x55b2f8b47380 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a79b68700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7ab6a700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b54f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:26.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a7a369700 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b47750 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9a70015000 tx=0x7f9a70020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af9573043b31df8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a68ff9700 1 -- 172.21.15.31:0/1861619637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7002b070 con 0x55b2f8b47380 2023-12-14T18:31:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a68ff9700 1 -- 172.21.15.31:0/1861619637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7001e040 con 0x55b2f8b47380 2023-12-14T18:31:26.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.066+0000 7f9a68ff9700 1 -- 172.21.15.31:0/1861619637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70024e10 con 0x55b2f8b47380 2023-12-14T18:31:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1861619637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 msgr2=0x55b2f8b47750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b47750 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9a70015000 tx=0x7f9a70020000 comp rx=0 tx=0).stop 2023-12-14T18:31:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1861619637 shutdown_connections 2023-12-14T18:31:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b54f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b47750 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1861619637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b52610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1861619637 >> 172.21.15.31:0/1861619637 conn(0x55b2f8b3dca0 msgr2=0x55b2f8b437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1861619637 shutdown_connections 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.067+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1861619637 wait complete. 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 -- start start 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 Processor -- start 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b4f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8a35f90 con 0x55b2f8b47380 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8a35c30 con 0x55b2f8b47c90 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f891d6a0 con 0x55b2f8b52b50 2023-12-14T18:31:26.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a8875bb80 10 client.0 _ll_register_callbacks cb 0x55b2f8a59910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48590/0 (socket says 172.21.15.31:48590) 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a7a369700 1 -- 172.21.15.31:0/2284777819 learned_addr learned my addr 172.21.15.31:0/2284777819 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.068+0000 7f9a7ab6a700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b4f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7a369700 1 -- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 msgr2=0x55b2f8b4fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b4fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7a369700 1 -- 172.21.15.31:0/2284777819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 msgr2=0x55b2f8b4f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b4f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7a369700 1 -- 172.21.15.31:0/2284777819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7410d8e0 con 0x55b2f8b47380 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7ab6a700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b4fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b524f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b6f0 tx=0x7f9a7000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13df081edfa08bff server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:26.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a5dffb700 1 -- 172.21.15.31:0/2284777819 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7002b040 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a5dffb700 1 -- 172.21.15.31:0/2284777819 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9a7000be80 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2f8b3b2b0 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a5dffb700 1 -- 172.21.15.31:0/2284777819 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a7002f4a0 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a8875bb80 10 client.16548 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.069+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2f8a60170 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a5dffb700 1 -- 172.21.15.31:0/2284777819 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b2f8b3b2b0 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a5dffb700 1 -- 172.21.15.31:0/2284777819 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9a70040070 con 0x55b2f8b47380 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a8875bb80 10 client.16548 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a8875bb80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a8875bb80 20 client.16548 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8b61f60 0x55b2f8b64340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a8875bb80 4 client.16548 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:26.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2f8a92ce0 con 0x55b2f8b61f60 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.070+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8b61f60 0x55b2f8b64340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.071+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8b61f60 0x55b2f8b64340 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abae6abf610d9726 server_cookie=23823ee1e9d43967 in_seq=0 out_seq=0 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.071+0000 7f9a5dffb700 10 client.16548 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.071+0000 7f9a5dffb700 1 -- 172.21.15.31:0/2284777819 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b2f8a92ce0 con 0x55b2f8b61f60 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.071+0000 7f9a5dffb700 10 client.16548 handle_command_reply: tid=0 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.071+0000 7f9a8875bb80 1 client.16548 shutdown 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 msgr2=0x55b2f8b524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b524f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b6f0 tx=0x7f9a7000bd10 comp rx=0 tx=0).stop 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 shutdown_connections 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8b61f60 0x55b2f8b64340 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b52b50 0x55b2f8b4fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b524f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a5dffb700 0 client.16548 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2284777819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b4f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 >> 172.21.15.31:0/2284777819 conn(0x55b2f8b3dca0 msgr2=0x55b2f8b433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 shutdown_connections 2023-12-14T18:31:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2284777819 wait complete. 2023-12-14T18:31:26.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.072+0000 7f9a8875bb80 20 client.16548 trim_cache size 0 max 16384 2023-12-14T18:31:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 Processor -- start 2023-12-14T18:31:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 -- start start 2023-12-14T18:31:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 0x55b2f8b4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b52b50 0x55b2f8b5e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8b59b10 con 0x55b2f8b52b50 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8b5ec80 con 0x55b2f8b47c90 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8b5ee40 con 0x55b2f8b47380 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a7ab6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b52b50 0x55b2f8b5e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 0x55b2f8b4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a79b68700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 0x55b2f8b4a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:50332/0 (socket says 172.21.15.31:50332) 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a7a369700 1 -- 172.21.15.31:0/2793690121 learned_addr learned my addr 172.21.15.31:0/2793690121 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.090+0000 7f9a7ab6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b52b50 0x55b2f8b5e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48606/0 (socket says 172.21.15.31:48606) 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a7a369700 1 -- 172.21.15.31:0/2793690121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 msgr2=0x55b2f8b595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a7a369700 1 -- 172.21.15.31:0/2793690121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b52b50 msgr2=0x55b2f8b5e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b52b50 0x55b2f8b5e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a7a369700 1 -- 172.21.15.31:0/2793690121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2f8a5fec0 con 0x55b2f8b47380 2023-12-14T18:31:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a79b68700 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a7a369700 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 0x55b2f8b4a270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005f50 tx=0x7f9a7000ff80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53e12c2a78dbad2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a697fa700 1 -- 172.21.15.31:0/2793690121 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7002b040 con 0x55b2f8b47380 2023-12-14T18:31:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a697fa700 1 -- 172.21.15.31:0/2793690121 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a70038960 con 0x55b2f8b47380 2023-12-14T18:31:26.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.091+0000 7f9a697fa700 1 -- 172.21.15.31:0/2793690121 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70022390 con 0x55b2f8b47380 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2793690121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 msgr2=0x55b2f8b4a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 0x55b2f8b4a270 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a70005f50 tx=0x7f9a7000ff80 comp rx=0 tx=0).stop 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2793690121 shutdown_connections 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b52b50 0x55b2f8b5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47380 0x55b2f8b4a270 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/2793690121 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b47c90 0x55b2f8b595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2793690121 >> 172.21.15.31:0/2793690121 conn(0x55b2f8b3dca0 msgr2=0x55b2f8b41f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:26.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2793690121 shutdown_connections 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- 172.21.15.31:0/2793690121 wait complete. 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- start start 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 Processor -- start 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b56d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47c90 0x55b2f8b57270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b52b50 0x55b2f8bb91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8b57a10 con 0x55b2f8b47380 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8b57b80 con 0x55b2f8b52b50 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2f8bb9700 con 0x55b2f8b47c90 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a8875bb80 10 client.0 _ll_register_callbacks cb 0x7f9a64010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.092+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b56d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b56d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48616/0 (socket says 172.21.15.31:48616) 2023-12-14T18:31:26.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 -- 172.21.15.31:0/1459489631 learned_addr learned my addr 172.21.15.31:0/1459489631 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a79b68700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47c90 0x55b2f8b57270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7ab6a700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b52b50 0x55b2f8bb91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 -- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47c90 msgr2=0x55b2f8b57270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47c90 0x55b2f8b57270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 -- 172.21.15.31:0/1459489631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b52b50 msgr2=0x55b2f8bb91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b52b50 0x55b2f8bb91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 -- 172.21.15.31:0/1459489631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a741102e0 con 0x55b2f8b47380 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a7a369700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b56d30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9a700386f0 tx=0x7f9a7000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d1fcdd451a809d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a5dffb700 1 -- 172.21.15.31:0/1459489631 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7002b040 con 0x55b2f8b47380 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a5dffb700 1 -- 172.21.15.31:0/1459489631 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9a70004a30 con 0x55b2f8b47380 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.093+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a7001e040 con 0x55b2f8b47380 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a5dffb700 1 -- 172.21.15.31:0/1459489631 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a70004bf0 con 0x55b2f8b47380 2023-12-14T18:31:26.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 10 client.16554 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b2f8a60170 con 0x55b2f8b47380 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a5dffb700 1 -- 172.21.15.31:0/1459489631 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7001e040 con 0x55b2f8b47380 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a5dffb700 1 -- 172.21.15.31:0/1459489631 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9a7004f070 con 0x55b2f8b47380 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 10 client.16554 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 20 client.16554 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8bba9b0 0x55b2f8bbcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 4 client.16554 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2f8a92a00 con 0x55b2f8bba9b0 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.094+0000 7f9a79b68700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8bba9b0 0x55b2f8bbcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a5dffb700 10 client.16554 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a79b68700 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8bba9b0 0x55b2f8bbcdc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad71b1e3d0e7d702 server_cookie=da659f4aa1e6dd31 in_seq=0 out_seq=0 2023-12-14T18:31:26.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a5dffb700 1 -- 172.21.15.31:0/1459489631 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55b2f8a92a00 con 0x55b2f8bba9b0 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a5dffb700 10 client.16554 handle_command_reply: tid=0 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 client.16554 shutdown 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 msgr2=0x55b2f8b56d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b56d30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9a700386f0 tx=0x7f9a7000bc00 comp rx=0 tx=0).stop 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 shutdown_connections 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b2f8bba9b0 0x55b2f8bbcdc0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b2f8b52b50 0x55b2f8bb91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a5dffb700 0 client.16554 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b2f8b47380 0x55b2f8b56d30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 --2- 172.21.15.31:0/1459489631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2f8b47c90 0x55b2f8b57270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 >> 172.21.15.31:0/1459489631 conn(0x55b2f8b3dca0 msgr2=0x55b2f8b4aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 shutdown_connections 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 1 -- 172.21.15.31:0/1459489631 wait complete. 2023-12-14T18:31:26.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.095+0000 7f9a8875bb80 20 client.16554 trim_cache size 0 max 16384 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (409 inodes in the stack)", 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: "e9c9a305-94ae-47d1-85f0-57f435df8e4b": { 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e9c9a305-94ae-47d1-85f0-57f435df8e4b", 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:31:26.103 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:31:26.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a600a12c0 msgr2=0x7f9a600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a600a12c0 0x7f9a600a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000b2e0 tx=0x7f9a70005fb0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 msgr2=0x7f9a74196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:26.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 0x7f9a74196960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400e940 tx=0x7f9a6400ec50 comp rx=0 tx=0).stop 2023-12-14T18:31:26.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 shutdown_connections 2023-12-14T18:31:26.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a741054d0 0x7f9a7419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a600a12c0 0x7f9a600a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a740680c0 0x7f9a74196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 --2- 172.21.15.31:0/2953568583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a74068a60 0x7f9a74196960 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:26.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 >> 172.21.15.31:0/2953568583 conn(0x7f9a74075fd0 msgr2=0x7f9a740fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:26.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 shutdown_connections 2023-12-14T18:31:26.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:26.096+0000 7f9a7c5cd700 1 -- 172.21.15.31:0/2953568583 wait complete. 2023-12-14T18:31:26.111 INFO:tasks.cephfs.filesystem:scrub status for tag:e9c9a305-94ae-47d1-85f0-57f435df8e4b - {'path': '/', 'tag': 'e9c9a305-94ae-47d1-85f0-57f435df8e4b', 'options': 'recursive,force'} 2023-12-14T18:31:26.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:25 smithi031 ceph-mon[42085]: pgmap v111: 129 pgs: 129 active+clean; 253 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2023-12-14T18:31:26.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:25 smithi176 ceph-mon[43506]: pgmap v111: 129 pgs: 129 active+clean; 253 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2023-12-14T18:31:26.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:25 smithi204 ceph-mon[43866]: pgmap v111: 129 pgs: 129 active+clean; 253 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 1.34k op/s 2023-12-14T18:31:27.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:26 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:31:27.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:26 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:31:27.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:26 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:31:28.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:27 smithi031 ceph-mon[42085]: pgmap v112: 129 pgs: 129 active+clean; 253 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 183 op/s 2023-12-14T18:31:28.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:27 smithi176 ceph-mon[43506]: pgmap v112: 129 pgs: 129 active+clean; 253 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 183 op/s 2023-12-14T18:31:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:27 smithi204 ceph-mon[43866]: pgmap v112: 129 pgs: 129 active+clean; 253 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 183 op/s 2023-12-14T18:31:30.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:29 smithi031 ceph-mon[42085]: pgmap v113: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 553 op/s 2023-12-14T18:31:30.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:29 smithi176 ceph-mon[43506]: pgmap v113: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 553 op/s 2023-12-14T18:31:30.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:29 smithi204 ceph-mon[43866]: pgmap v113: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 553 op/s 2023-12-14T18:31:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:31 smithi031 ceph-mon[42085]: pgmap v114: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 452 op/s 2023-12-14T18:31:32.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:31 smithi176 ceph-mon[43506]: pgmap v114: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 452 op/s 2023-12-14T18:31:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:31 smithi204 ceph-mon[43866]: pgmap v114: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 452 op/s 2023-12-14T18:31:32.429 INFO:tasks.workunit.client.0.smithi031.stdout:created hierarchy, now cleaning up 2023-12-14T18:31:34.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:33 smithi031 ceph-mon[42085]: pgmap v115: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 452 op/s 2023-12-14T18:31:34.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:33 smithi176 ceph-mon[43506]: pgmap v115: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 452 op/s 2023-12-14T18:31:34.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:33 smithi204 ceph-mon[43866]: pgmap v115: 129 pgs: 129 active+clean; 235 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.3 MiB/s wr, 452 op/s 2023-12-14T18:31:36.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:35 smithi031 ceph-mon[42085]: pgmap v116: 129 pgs: 129 active+clean; 255 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 511 op/s 2023-12-14T18:31:36.241 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:35 smithi176 ceph-mon[43506]: pgmap v116: 129 pgs: 129 active+clean; 255 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 511 op/s 2023-12-14T18:31:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:35 smithi204 ceph-mon[43866]: pgmap v116: 129 pgs: 129 active+clean; 255 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 511 op/s 2023-12-14T18:31:38.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:37 smithi031 ceph-mon[42085]: pgmap v117: 129 pgs: 129 active+clean; 255 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2023-12-14T18:31:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:37 smithi176 ceph-mon[43506]: pgmap v117: 129 pgs: 129 active+clean; 255 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2023-12-14T18:31:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:37 smithi204 ceph-mon[43866]: pgmap v117: 129 pgs: 129 active+clean; 255 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2023-12-14T18:31:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 -- 172.21.15.31:0/93970251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ff3c0 msgr2=0x7f05140ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 --2- 172.21.15.31:0/93970251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ff3c0 0x7f05140ff790 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f05040099c0 tx=0x7f0504009cd0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 -- 172.21.15.31:0/93970251 shutdown_connections 2023-12-14T18:31:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 --2- 172.21.15.31:0/93970251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0514104400 0x7f051410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 --2- 172.21.15.31:0/93970251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ff3c0 0x7f05140ff790 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 --2- 172.21.15.31:0/93970251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05140ffd60 0x7f0514103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 -- 172.21.15.31:0/93970251 >> 172.21.15.31:0/93970251 conn(0x7f05140763a0 msgr2=0x7f05140767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 -- 172.21.15.31:0/93970251 shutdown_connections 2023-12-14T18:31:38.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.497+0000 7f0519215700 1 -- 172.21.15.31:0/93970251 wait complete. 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 Processor -- start 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 -- start start 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05140ff3c0 0x7f051410c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 0x7f051410ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514104400 0x7f05141a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051410d650 con 0x7f05140ffd60 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05141a4ad0 con 0x7f05140ff3c0 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.498+0000 7f0519215700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05141a4c10 con 0x7f0514104400 2023-12-14T18:31:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 0x7f051410ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514104400 0x7f05141a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 0x7f051410ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57392/0 (socket says 172.21.15.31:57392) 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051259c700 1 -- 172.21.15.31:0/584958440 learned_addr learned my addr 172.21.15.31:0/584958440 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f0512d9d700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05140ff3c0 0x7f051410c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 -- 172.21.15.31:0/584958440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05140ff3c0 msgr2=0x7f051410c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05140ff3c0 0x7f051410c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 -- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 msgr2=0x7f051410ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 0x7f051410ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 -- 172.21.15.31:0/584958440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05040095f0 con 0x7f0514104400 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051259c700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 0x7f051410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f0512d9d700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05140ff3c0 0x7f051410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f051359e700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514104400 0x7f05141a4590 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f050c00efc0 tx=0x7f050c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f04fbfff700 1 -- 172.21.15.31:0/584958440 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c018070 con 0x7f0514104400 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0514110390 con 0x7f0514104400 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f04fbfff700 1 -- 172.21.15.31:0/584958440 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f050c00f040 con 0x7f0514104400 2023-12-14T18:31:38.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f04fbfff700 1 -- 172.21.15.31:0/584958440 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c00cd40 con 0x7f0514104400 2023-12-14T18:31:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.499+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0514110200 con 0x7f0514104400 2023-12-14T18:31:38.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.502+0000 7f04fbfff700 1 -- 172.21.15.31:0/584958440 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f050c003a00 con 0x7f0514104400 2023-12-14T18:31:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.503+0000 7f04fbfff700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04fc0a12c0 0x7f04fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.503+0000 7f0512d9d700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04fc0a12c0 0x7f04fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.503+0000 7f04fbfff700 1 -- 172.21.15.31:0/584958440 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f050c0ccbb0 con 0x7f0514104400 2023-12-14T18:31:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.503+0000 7f0512d9d700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04fc0a12c0 0x7f04fc0a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0504009990 tx=0x7f050401a040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.503+0000 7f05253a3b80 1 Processor -- start 2023-12-14T18:31:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 -- start start 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c4a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0b34580 con 0x5555b0c4a370 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0ba3810 con 0x5555b0c550f0 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f05253a3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0b9cca0 con 0x5555b0aa7430 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c4a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c49e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:57368/0 (socket says 172.21.15.31:57368) 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f0512d9d700 1 -- 172.21.15.31:0/3569382327 learned_addr learned my addr 172.21.15.31:0/3569382327 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051359e700 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051259c700 1 -- 172.21.15.31:0/3569382327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 msgr2=0x5555b0c49e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051259c700 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c49e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051259c700 1 -- 172.21.15.31:0/3569382327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 msgr2=0x5555b0c574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051259c700 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.504+0000 7f051259c700 1 -- 172.21.15.31:0/3569382327 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b0a1e350 con 0x5555b0c4a370 2023-12-14T18:31:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.505+0000 7f051259c700 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c4a7e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0508009fd0 tx=0x7f05080117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eb7144d1077245 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.505+0000 7f04f97fa700 1 -- 172.21.15.31:0/3569382327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508004230 con 0x5555b0c4a370 2023-12-14T18:31:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.505+0000 7f04f97fa700 1 -- 172.21.15.31:0/3569382327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0508004390 con 0x5555b0c4a370 2023-12-14T18:31:38.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.505+0000 7f04f97fa700 1 -- 172.21.15.31:0/3569382327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05080198a0 con 0x5555b0c4a370 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 -- 172.21.15.31:0/3569382327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 msgr2=0x5555b0c4a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c4a7e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0508009fd0 tx=0x7f05080117f0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 -- 172.21.15.31:0/3569382327 shutdown_connections 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3569382327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c4a7e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 -- 172.21.15.31:0/3569382327 >> 172.21.15.31:0/3569382327 conn(0x5555b0c40890 msgr2=0x5555b0c46200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 -- 172.21.15.31:0/3569382327 shutdown_connections 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 -- 172.21.15.31:0/3569382327 wait complete. 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 -- start start 2023-12-14T18:31:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.506+0000 7f05253a3b80 1 Processor -- start 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c54a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c4a370 0x5555b0c51cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c550f0 0x5555b0c521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0b38f30 con 0x5555b0c550f0 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0b38bd0 con 0x5555b0c4a370 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0a20640 con 0x5555b0aa7430 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f05253a3b80 10 client.0 _ll_register_callbacks cb 0x5555b0b5c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c54a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c550f0 0x5555b0c521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051259c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c4a370 0x5555b0c51cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c54a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:57376/0 (socket says 172.21.15.31:57376) 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c550f0 0x5555b0c521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57400/0 (socket says 172.21.15.31:57400) 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f0512d9d700 1 -- 172.21.15.31:0/349671727 learned_addr learned my addr 172.21.15.31:0/349671727 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051259c700 1 -- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 msgr2=0x5555b0c54a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051259c700 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c54a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051259c700 1 -- 172.21.15.31:0/349671727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c550f0 msgr2=0x5555b0c521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051259c700 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c550f0 0x5555b0c521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.507+0000 7f051259c700 1 -- 172.21.15.31:0/349671727 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0508009d00 con 0x5555b0c4a370 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f0512d9d700 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c54a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f051259c700 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c4a370 0x5555b0c51cb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0508000c00 tx=0x7f0508003910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4eec5adbc309934 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f04f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f04f6ff5700 1 -- 172.21.15.31:0/349671727 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508005240 con 0x5555b0c4a370 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f04f6ff5700 1 -- 172.21.15.31:0/349671727 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f05080053a0 con 0x5555b0c4a370 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5555b0c3dea0 con 0x5555b0c4a370 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f04f6ff5700 1 -- 172.21.15.31:0/349671727 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0508005d50 con 0x5555b0c4a370 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f05253a3b80 10 client.26393 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.508+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5555b0b63110 con 0x5555b0c4a370 2023-12-14T18:31:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f04f6ff5700 1 -- 172.21.15.31:0/349671727 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5555b0c3dea0 con 0x5555b0c4a370 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f04f6ff5700 1 -- 172.21.15.31:0/349671727 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0508021040 con 0x5555b0c4a370 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f05253a3b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f05253a3b80 10 client.26393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f05253a3b80 20 client.26393 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f05253a3b80 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0c64b20 0x5555b0c66f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f05253a3b80 4 client.26393 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555b0b95c80 con 0x5555b0c64b20 2023-12-14T18:31:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f0512d9d700 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0c64b20 0x5555b0c66f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f0512d9d700 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0c64b20 0x5555b0c66f30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cad5ddba13c7c server_cookie=ee63667ac6301592 in_seq=0 out_seq=0 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.509+0000 7f04f6ff5700 10 client.26393 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f04f6ff5700 1 -- 172.21.15.31:0/349671727 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5555b0b95c80 con 0x5555b0c64b20 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f04f6ff5700 10 client.26393 handle_command_reply: tid=0 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 client.26393 shutdown 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c4a370 msgr2=0x5555b0c51cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c4a370 0x5555b0c51cb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0508000c00 tx=0x7f0508003910 comp rx=0 tx=0).stop 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 shutdown_connections 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0c64b20 0x5555b0c66f30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c550f0 0x5555b0c521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f04f6ff5700 0 client.26393 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c54a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 --2- 172.21.15.31:0/349671727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c4a370 0x5555b0c51cb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 >> 172.21.15.31:0/349671727 conn(0x5555b0c40890 msgr2=0x5555b0c45d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 shutdown_connections 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 1 -- 172.21.15.31:0/349671727 wait complete. 2023-12-14T18:31:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.510+0000 7f05253a3b80 20 client.26393 trim_cache size 0 max 16384 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 Processor -- start 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 -- start start 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c54a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0c4b0e0 con 0x5555b0c4a370 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0c4bd00 con 0x5555b0c550f0 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f05253a3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0c57a70 con 0x5555b0aa7430 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f051259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c54a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:57384/0 (socket says 172.21.15.31:57384) 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f0512d9d700 1 -- 172.21.15.31:0/829520281 learned_addr learned my addr 172.21.15.31:0/829520281 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f051259c700 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c54a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57408/0 (socket says 172.21.15.31:57408) 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.527+0000 7f051359e700 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c4cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f0512d9d700 1 -- 172.21.15.31:0/829520281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 msgr2=0x5555b0c4cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f0512d9d700 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c4cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f0512d9d700 1 -- 172.21.15.31:0/829520281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 msgr2=0x5555b0c54a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f0512d9d700 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c54a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f0512d9d700 1 -- 172.21.15.31:0/829520281 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555b0c51820 con 0x5555b0aa7430 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f051259c700 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c54a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f0512d9d700 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c544c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f05040099c0 tx=0x7f050401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59642f7c1b6b915 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f04f9ffb700 1 -- 172.21.15.31:0/829520281 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050402a070 con 0x5555b0aa7430 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f04f9ffb700 1 -- 172.21.15.31:0/829520281 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f050402b040 con 0x5555b0aa7430 2023-12-14T18:31:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.528+0000 7f04f9ffb700 1 -- 172.21.15.31:0/829520281 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504005520 con 0x5555b0aa7430 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 -- 172.21.15.31:0/829520281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 msgr2=0x5555b0c544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c544c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f05040099c0 tx=0x7f050401f000 comp rx=0 tx=0).stop 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 -- 172.21.15.31:0/829520281 shutdown_connections 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0c4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0aa7430 0x5555b0c544c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 --2- 172.21.15.31:0/829520281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0c4a370 0x5555b0c54a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.534 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 -- 172.21.15.31:0/829520281 >> 172.21.15.31:0/829520281 conn(0x5555b0c40890 msgr2=0x5555b0c50cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 -- 172.21.15.31:0/829520281 shutdown_connections 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 -- 172.21.15.31:0/829520281 wait complete. 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 -- start start 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.529+0000 7f05253a3b80 1 Processor -- start 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 0x5555b0c596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0c4a370 0x5555b0c59c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0cb79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0c5a420 con 0x5555b0aa7430 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0c5a590 con 0x5555b0c550f0 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555b0cb7fa0 con 0x5555b0c4a370 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f05253a3b80 10 client.0 _ll_register_callbacks cb 0x5555b0b5c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 0x5555b0c596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 0x5555b0c596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57422/0 (socket says 172.21.15.31:57422) 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 -- 172.21.15.31:0/3513855077 learned_addr learned my addr 172.21.15.31:0/3513855077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f051259c700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0c4a370 0x5555b0c59c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.535 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f051359e700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0cb79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 -- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0c4a370 msgr2=0x5555b0c59c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0c4a370 0x5555b0c59c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 -- 172.21.15.31:0/3513855077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 msgr2=0x5555b0cb79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0cb79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.530+0000 7f0512d9d700 1 -- 172.21.15.31:0/3513855077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555b0b63110 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f0512d9d700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 0x5555b0c596e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f05040095c0 tx=0x7f05040092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd40dcaae5b4c4a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f04f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f04f6ff5700 1 -- 172.21.15.31:0/3513855077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050402a050 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f04f6ff5700 1 -- 172.21.15.31:0/3513855077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0504034d90 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0508004390 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f04f6ff5700 1 -- 172.21.15.31:0/3513855077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05040340f0 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f05253a3b80 10 client.16572 fetch_fsmap learned FSMap version 27 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f0514068060 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f04f6ff5700 1 -- 172.21.15.31:0/3513855077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0508004390 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.531+0000 7f04f6ff5700 1 -- 172.21.15.31:0/3513855077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f05040245c0 con 0x5555b0aa7430 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f05253a3b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:31:38.536 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f05253a3b80 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f05253a3b80 20 client.16572 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0cb9280 0x5555b0cbb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f05253a3b80 4 client.16572 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5555b0b959a0 con 0x5555b0cb9280 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f051259c700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0cb9280 0x5555b0cbb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f04f6ff5700 10 client.16572 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:38.537 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.532+0000 7f051259c700 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0cb9280 0x5555b0cbb690 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c532a8d2ed0c20c2 server_cookie=fa40f33f1343600b in_seq=0 out_seq=0 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.995+0000 7f04f6ff5700 1 -- 172.21.15.31:0/3513855077 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5555b0b959a0 con 0x5555b0cb9280 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.995+0000 7f04f6ff5700 10 client.16572 handle_command_reply: tid=0 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 client.16572 shutdown 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 msgr2=0x5555b0c596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 0x5555b0c596e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f05040095c0 tx=0x7f05040092f0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 shutdown_connections 2023-12-14T18:31:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5555b0cb9280 0x5555b0cbb690 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5555b0c550f0 0x5555b0cb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5555b0aa7430 0x5555b0c596e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 --2- 172.21.15.31:0/3513855077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555b0c4a370 0x5555b0c59c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 >> 172.21.15.31:0/3513855077 conn(0x5555b0c40890 msgr2=0x5555b0c60490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f04f6ff5700 0 client.16572 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 shutdown_connections 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 1 -- 172.21.15.31:0/3513855077 wait complete. 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.996+0000 7f05253a3b80 20 client.16572 trim_cache size 0 max 16384 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:31:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04fc0a12c0 msgr2=0x7f04fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04fc0a12c0 0x7f04fc0a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0504009990 tx=0x7f050401a040 comp rx=0 tx=0).stop 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514104400 msgr2=0x7f05141a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514104400 0x7f05141a4590 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f050c00efc0 tx=0x7f050c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 shutdown_connections 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f04fc0a12c0 0x7f04fc0a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:39.000 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514104400 0x7f05141a4590 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05140ff3c0 0x7f051410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 --2- 172.21.15.31:0/584958440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05140ffd60 0x7f051410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.997+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 >> 172.21.15.31:0/584958440 conn(0x7f05140763a0 msgr2=0x7f05140fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.998+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 shutdown_connections 2023-12-14T18:31:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:38.998+0000 7f0519215700 1 -- 172.21.15.31:0/584958440 wait complete. 2023-12-14T18:31:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:39 smithi031 ceph-mon[42085]: pgmap v118: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 453 op/s 2023-12-14T18:31:40.241 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:39 smithi176 ceph-mon[43506]: pgmap v118: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 453 op/s 2023-12-14T18:31:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:39 smithi204 ceph-mon[43866]: pgmap v118: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 453 op/s 2023-12-14T18:31:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:41 smithi176 ceph-mon[43506]: pgmap v119: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 83 op/s 2023-12-14T18:31:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:41 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:31:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:41 smithi204 ceph-mon[43866]: pgmap v119: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 83 op/s 2023-12-14T18:31:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:41 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:31:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:41 smithi031 ceph-mon[42085]: pgmap v119: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 83 op/s 2023-12-14T18:31:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:41 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:31:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:43 smithi176 ceph-mon[43506]: pgmap v120: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 83 op/s 2023-12-14T18:31:44.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:43 smithi204 ceph-mon[43866]: pgmap v120: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 83 op/s 2023-12-14T18:31:44.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:43 smithi031 ceph-mon[42085]: pgmap v120: 129 pgs: 129 active+clean; 273 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 83 op/s 2023-12-14T18:31:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:45 smithi176 ceph-mon[43506]: pgmap v121: 129 pgs: 129 active+clean; 277 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 287 op/s 2023-12-14T18:31:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:45 smithi204 ceph-mon[43866]: pgmap v121: 129 pgs: 129 active+clean; 277 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 287 op/s 2023-12-14T18:31:46.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:45 smithi031 ceph-mon[42085]: pgmap v121: 129 pgs: 129 active+clean; 277 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 287 op/s 2023-12-14T18:31:48.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:47 smithi176 ceph-mon[43506]: pgmap v122: 129 pgs: 129 active+clean; 277 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-12-14T18:31:48.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:47 smithi204 ceph-mon[43866]: pgmap v122: 129 pgs: 129 active+clean; 277 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-12-14T18:31:48.463 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:47 smithi031 ceph-mon[42085]: pgmap v122: 129 pgs: 129 active+clean; 277 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2023-12-14T18:31:50.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:49 smithi204 ceph-mon[43866]: pgmap v123: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2023-12-14T18:31:50.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:49 smithi031 ceph-mon[42085]: pgmap v123: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2023-12-14T18:31:50.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:49 smithi176 ceph-mon[43506]: pgmap v123: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 396 op/s 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 -- 172.21.15.31:0/579120597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 msgr2=0x7f0a50107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 --2- 172.21.15.31:0/579120597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a50107ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009990 tx=0x7f0a44009ca0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 -- 172.21.15.31:0/579120597 shutdown_connections 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 --2- 172.21.15.31:0/579120597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a501133b0 0x7f0a50115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 --2- 172.21.15.31:0/579120597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a50107ec0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 --2- 172.21.15.31:0/579120597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a50108400 0x7f0a50112e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.156 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.153+0000 7f0a567c6700 1 -- 172.21.15.31:0/579120597 >> 172.21.15.31:0/579120597 conn(0x7f0a5006c510 msgr2=0x7f0a500716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:52.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.155+0000 7f0a567c6700 1 -- 172.21.15.31:0/579120597 shutdown_connections 2023-12-14T18:31:52.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.157+0000 7f0a567c6700 1 -- 172.21.15.31:0/579120597 wait complete. 2023-12-14T18:31:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.158+0000 7f0a567c6700 1 Processor -- start 2023-12-14T18:31:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.158+0000 7f0a567c6700 1 -- start start 2023-12-14T18:31:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a567c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a501a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a567c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a50108400 0x7f0a501a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a567c6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a501133b0 0x7f0a501a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a567c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a501a5e30 con 0x7f0a50108400 2023-12-14T18:31:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a567c6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a501a5fa0 con 0x7f0a501133b0 2023-12-14T18:31:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a567c6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a501a6160 con 0x7f0a50107af0 2023-12-14T18:31:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.159+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a501a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.160+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a501a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53334/0 (socket says 172.21.15.31:53334) 2023-12-14T18:31:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.160+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3286422375 learned_addr learned my addr 172.21.15.31:0/3286422375 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.161+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3286422375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a501133b0 msgr2=0x7f0a501a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.161+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a50108400 0x7f0a501a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.160+0000 7f0a54d63700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a501133b0 0x7f0a501a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a501133b0 0x7f0a501a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a50108400 msgr2=0x7f0a501a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a50108400 0x7f0a501a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3286422375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440095f0 con 0x7f0a50107af0 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a501a0e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a44000c00 tx=0x7f0a44003d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a50108400 0x7f0a501a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:31:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a54d63700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a501133b0 0x7f0a501a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:31:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.162+0000 7f0a4d7fa700 1 -- 172.21.15.31:0/3286422375 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4401d070 con 0x7f0a50107af0 2023-12-14T18:31:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.163+0000 7f0a4d7fa700 1 -- 172.21.15.31:0/3286422375 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4400f460 con 0x7f0a50107af0 2023-12-14T18:31:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.163+0000 7f0a4d7fa700 1 -- 172.21.15.31:0/3286422375 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44017650 con 0x7f0a50107af0 2023-12-14T18:31:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.164+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5011af40 con 0x7f0a50107af0 2023-12-14T18:31:52.166 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.164+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5011adb0 con 0x7f0a50107af0 2023-12-14T18:31:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.171+0000 7f0a4d7fa700 1 -- 172.21.15.31:0/3286422375 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0a4400f5d0 con 0x7f0a50107af0 2023-12-14T18:31:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.172+0000 7f0a4d7fa700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0a380a1450 0x7f0a380a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.172+0000 7f0a4d7fa700 1 -- 172.21.15.31:0/3286422375 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0a440ce1b0 con 0x7f0a50107af0 2023-12-14T18:31:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.173+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0a380a1450 0x7f0a380a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.173+0000 7f0a62954b80 1 Processor -- start 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.174+0000 7f0a62954b80 1 -- start start 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d720ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d720b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d72160f0 0x5642d72184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a62954b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d70f5580 con 0x5642d7068430 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a62954b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d7164810 con 0x5642d720b370 2023-12-14T18:31:52.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a62954b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d715dca0 con 0x5642d72160f0 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d720ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d720ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58194/0 (socket says 172.21.15.31:58194) 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a4ffff700 1 -- 172.21.15.31:0/2239541368 learned_addr learned my addr 172.21.15.31:0/2239541368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d720b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.175+0000 7f0a54d63700 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d72160f0 0x5642d72184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.176+0000 7f0a4ffff700 1 -- 172.21.15.31:0/2239541368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d72160f0 msgr2=0x5642d72184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.176+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d72160f0 0x5642d72184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.176+0000 7f0a4ffff700 1 -- 172.21.15.31:0/2239541368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 msgr2=0x5642d720b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.176+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d720b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.176+0000 7f0a4ffff700 1 -- 172.21.15.31:0/2239541368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642d6fdf350 con 0x5642d7068430 2023-12-14T18:31:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.177+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0a380a1450 0x7f0a380a3900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a501a2460 tx=0x7f0a480073c0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.177+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d720ae30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0a44011040 tx=0x7f0a44009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f83388a82011de server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.177+0000 7f0a367fc700 1 -- 172.21.15.31:0/2239541368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4409af80 con 0x5642d7068430 2023-12-14T18:31:52.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.177+0000 7f0a367fc700 1 -- 172.21.15.31:0/2239541368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4409efb0 con 0x5642d7068430 2023-12-14T18:31:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.177+0000 7f0a367fc700 1 -- 172.21.15.31:0/2239541368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4409a470 con 0x5642d7068430 2023-12-14T18:31:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 -- 172.21.15.31:0/2239541368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 msgr2=0x5642d720ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d720ae30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0a44011040 tx=0x7f0a44009f60 comp rx=0 tx=0).stop 2023-12-14T18:31:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 -- 172.21.15.31:0/2239541368 shutdown_connections 2023-12-14T18:31:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d72160f0 0x5642d72184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d720ae30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2239541368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d720b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.181 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.179+0000 7f0a62954b80 1 -- 172.21.15.31:0/2239541368 >> 172.21.15.31:0/2239541368 conn(0x5642d7201890 msgr2=0x5642d7207200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.182+0000 7f0a62954b80 1 -- 172.21.15.31:0/2239541368 shutdown_connections 2023-12-14T18:31:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.182+0000 7f0a62954b80 1 -- 172.21.15.31:0/2239541368 wait complete. 2023-12-14T18:31:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.182+0000 7f0a62954b80 1 -- start start 2023-12-14T18:31:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 Processor -- start 2023-12-14T18:31:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d7215a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7213370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d70f9f30 con 0x5642d7068430 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d70f9bd0 con 0x5642d72160f0 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d6fe1640 con 0x5642d720b370 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.183+0000 7f0a62954b80 10 client.0 _ll_register_callbacks cb 0x5642d711d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.184+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d7215a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.184+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d7215a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58198/0 (socket says 172.21.15.31:58198) 2023-12-14T18:31:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.184+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3061312746 learned_addr learned my addr 172.21.15.31:0/3061312746 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a54d63700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7213370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 msgr2=0x5642d7212db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3061312746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 msgr2=0x5642d7213370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7213370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a4ffff700 1 -- 172.21.15.31:0/3061312746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5011adb0 con 0x5642d7068430 2023-12-14T18:31:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.185+0000 7f0a54d63700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7213370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:31:52.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.186+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d7215a20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0a44097480 tx=0x7f0a4400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fad9d94bfd9f9f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.186+0000 7f0a2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.186+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/3061312746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4409a680 con 0x5642d7068430 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.186+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/3061312746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0a440a2eb0 con 0x5642d7068430 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.186+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642d71feea0 con 0x5642d7068430 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/3061312746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a440a3070 con 0x5642d7068430 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 10 client.16578 fetch_fsmap learned FSMap version 28 2023-12-14T18:31:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5642d7124110 con 0x5642d7068430 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/3061312746 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642d71feea0 con 0x5642d7068430 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/3061312746 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0a440b32a0 con 0x5642d7068430 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 10 client.16578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 20 client.16578 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d7225370 0x5642d7227750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 4 client.16578 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.187+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642d7156c80 con 0x5642d7225370 2023-12-14T18:31:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.188+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d7225370 0x5642d7227750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.188+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d7225370 0x5642d7227750 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=923390bf25fbf5b8 server_cookie=578f8c4dc44bd8f6 in_seq=0 out_seq=0 2023-12-14T18:31:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.188+0000 7f0a2f7fe700 10 client.16578 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/3061312746 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5642d7156c80 con 0x5642d7225370 2023-12-14T18:31:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a2f7fe700 10 client.16578 handle_command_reply: tid=0 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 client.16578 shutdown 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 msgr2=0x5642d7215a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d7215a20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0a44097480 tx=0x7f0a4400fac0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 shutdown_connections 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d7225370 0x5642d7227750 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7213370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a2f7fe700 0 client.16578 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d7215a20 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 --2- 172.21.15.31:0/3061312746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 >> 172.21.15.31:0/3061312746 conn(0x5642d7201890 msgr2=0x5642d7206d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 shutdown_connections 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.189+0000 7f0a62954b80 1 -- 172.21.15.31:0/3061312746 wait complete. 2023-12-14T18:31:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.190+0000 7f0a62954b80 20 client.16578 trim_cache size 0 max 16384 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 Processor -- start 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 -- start start 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d721caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7221b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d72184d0 con 0x5642d7068430 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d721cfe0 con 0x5642d72160f0 2023-12-14T18:31:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a62954b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d7212ab0 con 0x5642d720b370 2023-12-14T18:31:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a4f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a4f7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53354/0 (socket says 172.21.15.31:53354) 2023-12-14T18:31:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/2678899928 learned_addr learned my addr 172.21.15.31:0/2678899928 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.208+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d721caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a54d63700 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7221b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/2678899928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 msgr2=0x5642d7221b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7221b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/2678899928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 msgr2=0x5642d721caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d721caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/2678899928 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642d7123e60 con 0x5642d720b370 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5642d72142f0 tx=0x7f0a48012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c20d4ec8e73426d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a36ffd700 1 -- 172.21.15.31:0/2678899928 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48022070 con 0x5642d720b370 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a36ffd700 1 -- 172.21.15.31:0/2678899928 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a48023e30 con 0x5642d720b370 2023-12-14T18:31:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.209+0000 7f0a36ffd700 1 -- 172.21.15.31:0/2678899928 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800f440 con 0x5642d720b370 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 -- 172.21.15.31:0/2678899928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 msgr2=0x5642d7212570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x5642d72142f0 tx=0x7f0a48012000 comp rx=0 tx=0).stop 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 -- 172.21.15.31:0/2678899928 shutdown_connections 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d72160f0 0x5642d7221b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d7068430 0x5642d721caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 --2- 172.21.15.31:0/2678899928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d720b370 0x5642d7212570 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 -- 172.21.15.31:0/2678899928 >> 172.21.15.31:0/2678899928 conn(0x5642d7201890 msgr2=0x5642d722bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 -- 172.21.15.31:0/2678899928 shutdown_connections 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.210+0000 7f0a62954b80 1 -- 172.21.15.31:0/2678899928 wait complete. 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 -- start start 2023-12-14T18:31:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 Processor -- start 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d7068430 0x5642d721c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d721aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d72160f0 0x5642d721afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d721b530 con 0x5642d72160f0 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d721b6a0 con 0x5642d720b370 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642d721b860 con 0x5642d7068430 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a62954b80 10 client.0 _ll_register_callbacks cb 0x5642d711d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.211+0000 7f0a4ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d7068430 0x5642d721c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a54d63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d72160f0 0x5642d721afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d721aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a54d63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d72160f0 0x5642d721afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58212/0 (socket says 172.21.15.31:58212) 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d721aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35612/0 (socket says 172.21.15.31:35612) 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a54d63700 1 -- 172.21.15.31:0/1387740025 learned_addr learned my addr 172.21.15.31:0/1387740025 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d7068430 msgr2=0x5642d721c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d7068430 0x5642d721c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/1387740025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d72160f0 msgr2=0x5642d721afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d72160f0 0x5642d721afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 -- 172.21.15.31:0/1387740025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4800cb80 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a4f7fe700 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d721aa60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5642d7156ce0 tx=0x7f0a4801d8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=703324139a402378 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/1387740025 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48022040 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/1387740025 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0a4801dd20 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.212+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a4409efb0 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/1387740025 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a48003aa0 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 10 client.26408 fetch_fsmap learned FSMap version 28 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5642d710a940 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/1387740025 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a4409efb0 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/1387740025 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0a480043c0 con 0x5642d720b370 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 10 client.26408 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:31:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 10 client.26408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 20 client.26408 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d727a4f0 0x5642d727c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 4 client.26408 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5642d71569a0 con 0x5642d727a4f0 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.213+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d727a4f0 0x5642d727c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.214+0000 7f0a4ffff700 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d727a4f0 0x5642d727c8d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ade237f906130a0 server_cookie=3a4cfe82c07f59f8 in_seq=0 out_seq=0 2023-12-14T18:31:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:52.214+0000 7f0a2f7fe700 10 client.26408 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:52 smithi204 ceph-mon[43866]: pgmap v124: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2023-12-14T18:31:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:51 smithi031 ceph-mon[42085]: pgmap v124: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2023-12-14T18:31:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:52 smithi176 ceph-mon[43506]: pgmap v124: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2023-12-14T18:31:54.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:54 smithi204 ceph-mon[43866]: pgmap v125: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2023-12-14T18:31:54.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:54 smithi031 ceph-mon[42085]: pgmap v125: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2023-12-14T18:31:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:54 smithi176 ceph-mon[43506]: pgmap v125: 129 pgs: 129 active+clean; 295 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2023-12-14T18:31:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.691+0000 7f0a2f7fe700 1 -- 172.21.15.31:0/1387740025 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5642d71569a0 con 0x5642d727a4f0 2023-12-14T18:31:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.691+0000 7f0a2f7fe700 10 client.26408 handle_command_reply: tid=0 2023-12-14T18:31:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.691+0000 7f0a62954b80 1 client.26408 shutdown 2023-12-14T18:31:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.694+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 msgr2=0x5642d721aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.694+0000 7f0a62954b80 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d721aa60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5642d7156ce0 tx=0x7f0a4801d8e0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 shutdown_connections 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5642d727a4f0 0x5642d727c8d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5642d72160f0 0x5642d721afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5642d7068430 0x5642d721c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 --2- 172.21.15.31:0/1387740025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5642d720b370 0x5642d721aa60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 >> 172.21.15.31:0/1387740025 conn(0x5642d7201890 msgr2=0x5642d7205930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a2f7fe700 0 client.26408 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 shutdown_connections 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 1 -- 172.21.15.31:0/1387740025 wait complete. 2023-12-14T18:31:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.696+0000 7f0a62954b80 20 client.26408 trim_cache size 0 max 16384 2023-12-14T18:31:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:31:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:31:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:31:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:31:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.700+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0a380a1450 msgr2=0x7f0a380a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.700+0000 7f0a567c6700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0a380a1450 0x7f0a380a3900 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a501a2460 tx=0x7f0a480073c0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.700+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 msgr2=0x7f0a501a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.700+0000 7f0a567c6700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a501a0e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a44000c00 tx=0x7f0a44003d10 comp rx=0 tx=0).stop 2023-12-14T18:31:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.704+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 shutdown_connections 2023-12-14T18:31:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.704+0000 7f0a567c6700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a501133b0 0x7f0a501a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.704+0000 7f0a567c6700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0a380a1450 0x7f0a380a3900 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.704+0000 7f0a567c6700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a50107af0 0x7f0a501a0e50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.704+0000 7f0a567c6700 1 --2- 172.21.15.31:0/3286422375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a50108400 0x7f0a501a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.704+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 >> 172.21.15.31:0/3286422375 conn(0x7f0a5006c510 msgr2=0x7f0a5010ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.707+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 shutdown_connections 2023-12-14T18:31:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:31:55.707+0000 7f0a567c6700 1 -- 172.21.15.31:0/3286422375 wait complete. 2023-12-14T18:31:56.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:31:56.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 -- 172.21.15.31:0/3317509122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 msgr2=0x7fe53410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 --2- 172.21.15.31:0/3317509122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 0x7fe53410d0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00b0d0 tx=0x7fe52c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 -- 172.21.15.31:0/3317509122 shutdown_connections 2023-12-14T18:31:56.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 --2- 172.21.15.31:0/3317509122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 0x7fe53410d0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 --2- 172.21.15.31:0/3317509122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5340680c0 0x7fe534068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 --2- 172.21.15.31:0/3317509122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe534068a60 0x7fe534104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 -- 172.21.15.31:0/3317509122 >> 172.21.15.31:0/3317509122 conn(0x7fe534075fd0 msgr2=0x7fe5340763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:56.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.270+0000 7fe53b70b700 1 -- 172.21.15.31:0/3317509122 shutdown_connections 2023-12-14T18:31:56.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.271+0000 7fe53b70b700 1 -- 172.21.15.31:0/3317509122 wait complete. 2023-12-14T18:31:56.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.271+0000 7fe53b70b700 1 Processor -- start 2023-12-14T18:31:56.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.271+0000 7fe53b70b700 1 -- start start 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.271+0000 7fe53b70b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 0x7fe534196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.271+0000 7fe53b70b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe534068a60 0x7fe534196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.271+0000 7fe53b70b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 0x7fe53419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe53b70b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419b4e0 con 0x7fe534068a60 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe53b70b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419b650 con 0x7fe5340680c0 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe53b70b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53419b810 con 0x7fe5341054d0 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 0x7fe534196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe538ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe534068a60 0x7fe534196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe539ca8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 0x7fe53419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 0x7fe534196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:42122/0 (socket says 172.21.15.31:42122) 2023-12-14T18:31:56.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe538ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe534068a60 0x7fe534196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55750/0 (socket says 172.21.15.31:55750) 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 -- 172.21.15.31:0/4051318345 learned_addr learned my addr 172.21.15.31:0/4051318345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 -- 172.21.15.31:0/4051318345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 msgr2=0x7fe53419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 0x7fe53419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 -- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe534068a60 msgr2=0x7fe534196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe534068a60 0x7fe534196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 -- 172.21.15.31:0/4051318345 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524009520 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.272+0000 7fe5394a7700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 0x7fe534196500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe53000e9a0 tx=0x7fe53000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.273+0000 7fe52a7fc700 1 -- 172.21.15.31:0/4051318345 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530009790 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.273+0000 7fe52a7fc700 1 -- 172.21.15.31:0/4051318345 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe530010430 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.273+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe52c009d00 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.273+0000 7fe52a7fc700 1 -- 172.21.15.31:0/4051318345 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530018680 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.273+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5341102a0 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.275+0000 7fe52a7fc700 1 -- 172.21.15.31:0/4051318345 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe5300105a0 con 0x7fe5340680c0 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.276+0000 7fe52a7fc700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5200a12c0 0x7fe5200a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.276+0000 7fe52a7fc700 1 -- 172.21.15.31:0/4051318345 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe530014070 con 0x7fe5340680c0 2023-12-14T18:31:56.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.276+0000 7fe538ca6700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5200a12c0 0x7fe5200a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.276+0000 7fe538ca6700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5200a12c0 0x7fe5200a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe534197b10 tx=0x7fe524009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.277+0000 7fe547899b80 1 Processor -- start 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.277+0000 7fe547899b80 1 -- start start 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe547899b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545cd4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe547899b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4c90 0x558545ce3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe547899b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545ce6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe547899b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545bbe5e0 con 0x558545ce3f60 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe547899b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545c2d870 con 0x558545cd4380 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe547899b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545c26d00 con 0x558545cd4c90 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545cd4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545cd4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:42134/0 (socket says 172.21.15.31:42134) 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 -- 172.21.15.31:0/3716491717 learned_addr learned my addr 172.21.15.31:0/3716491717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe539ca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545ce6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe538ca6700 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4c90 0x558545ce3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 -- 172.21.15.31:0/3716491717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4c90 msgr2=0x558545ce3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4c90 0x558545ce3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 -- 172.21.15.31:0/3716491717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 msgr2=0x558545ce6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545ce6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 -- 172.21.15.31:0/3716491717 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558545aa83b0 con 0x558545cd4380 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe538ca6700 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4c90 0x558545ce3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.278+0000 7fe5394a7700 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545cd4750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe5300a1ec0 tx=0x7fe530099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d428063dceea559 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe51f7fe700 1 -- 172.21.15.31:0/3716491717 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300a23f0 con 0x558545cd4380 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe51f7fe700 1 -- 172.21.15.31:0/3716491717 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe530010880 con 0x558545cd4380 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe51f7fe700 1 -- 172.21.15.31:0/3716491717 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300a1f60 con 0x558545cd4380 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 -- 172.21.15.31:0/3716491717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 msgr2=0x558545cd4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545cd4750 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe5300a1ec0 tx=0x7fe530099fa0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 -- 172.21.15.31:0/3716491717 shutdown_connections 2023-12-14T18:31:56.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545ce6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545cd4750 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 --2- 172.21.15.31:0/3716491717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4c90 0x558545ce3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 -- 172.21.15.31:0/3716491717 >> 172.21.15.31:0/3716491717 conn(0x558545ccaca0 msgr2=0x558545cd07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 -- 172.21.15.31:0/3716491717 shutdown_connections 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.279+0000 7fe547899b80 1 -- 172.21.15.31:0/3716491717 wait complete. 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 -- start start 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 Processor -- start 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545ce13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545bc2f90 con 0x558545cd4c90 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545bc2c30 con 0x558545cd4380 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545aaa6a0 con 0x558545ce3f60 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.280+0000 7fe547899b80 10 client.0 _ll_register_callbacks cb 0x558545be6910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:56.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe538ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe538ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55764/0 (socket says 172.21.15.31:55764) 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe538ca6700 1 -- 172.21.15.31:0/2655440761 learned_addr learned my addr 172.21.15.31:0/2655440761 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545ce13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe5394a7700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 -- 172.21.15.31:0/2655440761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 msgr2=0x558545ce3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 -- 172.21.15.31:0/2655440761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 msgr2=0x558545ce0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 -- 172.21.15.31:0/2655440761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5341102a0 con 0x558545ce3f60 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe538ca6700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe539ca8700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545ce13f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00b770 tx=0x7fe52c008f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ceba5dbb1fe4762 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe51cff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.281+0000 7fe51cff9700 1 -- 172.21.15.31:0/2655440761 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00e040 con 0x558545ce3f60 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558545cc82b0 con 0x558545ce3f60 2023-12-14T18:31:56.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe51cff9700 1 -- 172.21.15.31:0/2655440761 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe52c0074e0 con 0x558545ce3f60 2023-12-14T18:31:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe51cff9700 1 -- 172.21.15.31:0/2655440761 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe52c0076a0 con 0x558545ce3f60 2023-12-14T18:31:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 10 client.26443 fetch_fsmap learned FSMap version 28 2023-12-14T18:31:56.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558545bed170 con 0x558545ce3f60 2023-12-14T18:31:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe51cff9700 1 -- 172.21.15.31:0/2655440761 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558545cc82b0 con 0x558545ce3f60 2023-12-14T18:31:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe51cff9700 1 -- 172.21.15.31:0/2655440761 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe52c00b100 con 0x558545ce3f60 2023-12-14T18:31:56.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 10 client.26443 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 20 client.26443 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545cf3a60 0x558545cf5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 4 client.26443 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.282+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558545c1fce0 con 0x558545cf3a60 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.283+0000 7fe5394a7700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545cf3a60 0x558545cf5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.283+0000 7fe51cff9700 10 client.26443 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.283+0000 7fe5394a7700 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545cf3a60 0x558545cf5e70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a7907d4f62bf2c server_cookie=82add5913a8b4080 in_seq=0 out_seq=0 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe51cff9700 1 -- 172.21.15.31:0/2655440761 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558545c1fce0 con 0x558545cf3a60 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe51cff9700 10 client.26443 handle_command_reply: tid=0 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 client.26443 shutdown 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 msgr2=0x558545ce13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545ce13f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00b770 tx=0x7fe52c008f80 comp rx=0 tx=0).stop 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 shutdown_connections 2023-12-14T18:31:56.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545cf3a60 0x558545cf5e70 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545ce13f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe51cff9700 0 client.26443 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 --2- 172.21.15.31:0/2655440761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 >> 172.21.15.31:0/2655440761 conn(0x558545ccaca0 msgr2=0x558545cd0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 shutdown_connections 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.284+0000 7fe547899b80 1 -- 172.21.15.31:0/2655440761 wait complete. 2023-12-14T18:31:56.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.285+0000 7fe547899b80 20 client.26443 trim_cache size 0 max 16384 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 Processor -- start 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 -- start start 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545cdbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545cd5390 con 0x558545cd4c90 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545cd55b0 con 0x558545cd4380 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe547899b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545cd5060 con 0x558545ce3f60 2023-12-14T18:31:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe538ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe538ca6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55778/0 (socket says 172.21.15.31:55778) 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe538ca6700 1 -- 172.21.15.31:0/1050546201 learned_addr learned my addr 172.21.15.31:0/1050546201 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe539ca8700 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545cdbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.303+0000 7fe5394a7700 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe5394a7700 1 -- 172.21.15.31:0/1050546201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 msgr2=0x558545cdbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe5394a7700 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545cdbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe5394a7700 1 -- 172.21.15.31:0/1050546201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 msgr2=0x558545ce3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe5394a7700 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe5394a7700 1 -- 172.21.15.31:0/1050546201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558545becec0 con 0x558545cd4380 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe538ca6700 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe5394a7700 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce34c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe5300a1ec0 tx=0x7fe530010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a71a1ca2aa362711 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe51ffff700 1 -- 172.21.15.31:0/1050546201 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300109f0 con 0x558545cd4380 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe51ffff700 1 -- 172.21.15.31:0/1050546201 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5300a2470 con 0x558545cd4380 2023-12-14T18:31:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.304+0000 7fe51ffff700 1 -- 172.21.15.31:0/1050546201 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300a1f60 con 0x558545cd4380 2023-12-14T18:31:56.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 -- 172.21.15.31:0/1050546201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 msgr2=0x558545ce34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce34c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe5300a1ec0 tx=0x7fe530010c80 comp rx=0 tx=0).stop 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 -- 172.21.15.31:0/1050546201 shutdown_connections 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545ce3f60 0x558545cdbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4380 0x558545ce34c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 --2- 172.21.15.31:0/1050546201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545cd4c90 0x558545ce3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 -- 172.21.15.31:0/1050546201 >> 172.21.15.31:0/1050546201 conn(0x558545ccaca0 msgr2=0x558545cefd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 -- 172.21.15.31:0/1050546201 shutdown_connections 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 -- 172.21.15.31:0/1050546201 wait complete. 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 -- start start 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.305+0000 7fe547899b80 1 Processor -- start 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4380 0x558545ce8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4c90 0x558545ce8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545d46920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545ce94e0 con 0x558545ce3f60 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545d46e90 con 0x558545cd4c90 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558545d47050 con 0x558545cd4380 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe547899b80 10 client.0 _ll_register_callbacks cb 0x558545be6910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545d46920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe5394a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4380 0x558545ce8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545d46920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55784/0 (socket says 172.21.15.31:55784) 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 -- 172.21.15.31:0/4100290446 learned_addr learned my addr 172.21.15.31:0/4100290446 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe5394a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4380 0x558545ce8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:58148/0 (socket says 172.21.15.31:58148) 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe538ca6700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4c90 0x558545ce8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 -- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4380 msgr2=0x558545ce8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4380 0x558545ce8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 -- 172.21.15.31:0/4100290446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4c90 msgr2=0x558545ce8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4c90 0x558545ce8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 -- 172.21.15.31:0/4100290446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe524017070 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe538ca6700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4c90 0x558545ce8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe539ca8700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545d46920 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00bfd0 tx=0x7fe52c006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9870b03cbac4908b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.306+0000 7fe51cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe51cff9700 1 -- 172.21.15.31:0/4100290446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c00e040 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe51cff9700 1 -- 172.21.15.31:0/4100290446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe52c00b0b0 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe530010880 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe51cff9700 1 -- 172.21.15.31:0/4100290446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe52c00b270 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 10 client.16602 fetch_fsmap learned FSMap version 28 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558545aa83b0 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe51cff9700 1 -- 172.21.15.31:0/4100290446 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe530010880 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe51cff9700 1 -- 172.21.15.31:0/4100290446 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe52c032430 con 0x558545ce3f60 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 10 client.16602 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 20 client.16602 populate_metadata read hostname 'smithi031' 2023-12-14T18:31:56.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545d48660 0x558545d4aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:56.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 4 client.16602 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:31:56.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.307+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558545c1fa00 con 0x558545d48660 2023-12-14T18:31:56.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.308+0000 7fe5394a7700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545d48660 0x558545d4aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:56.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.308+0000 7fe51cff9700 10 client.16602 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:56.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.308+0000 7fe5394a7700 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545d48660 0x558545d4aa70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f49d42d32ebb3ef server_cookie=10bf360eafef5e85 in_seq=0 out_seq=0 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe51cff9700 1 -- 172.21.15.31:0/4100290446 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558545c1fa00 con 0x558545d48660 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe51cff9700 10 client.16602 handle_command_reply: tid=0 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 client.16602 shutdown 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 msgr2=0x558545d46920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545d46920 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00bfd0 tx=0x7fe52c006210 comp rx=0 tx=0).stop 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 shutdown_connections 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558545d48660 0x558545d4aa70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558545ce3f60 0x558545d46920 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe51cff9700 0 client.16602 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558545cd4380 0x558545ce8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 --2- 172.21.15.31:0/4100290446 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558545cd4c90 0x558545ce8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 >> 172.21.15.31:0/4100290446 conn(0x558545ccaca0 msgr2=0x558545cdbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 shutdown_connections 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 1 -- 172.21.15.31:0/4100290446 wait complete. 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe547899b80 20 client.16602 trim_cache size 0 max 16384 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "e9c9a305-94ae-47d1-85f0-57f435df8e4b": { 2023-12-14T18:31:56.330 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e9c9a305-94ae-47d1-85f0-57f435df8e4b", 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5200a12c0 msgr2=0x7fe5200a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe53b70b700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5200a12c0 0x7fe5200a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe534197b10 tx=0x7fe524009260 comp rx=0 tx=0).stop 2023-12-14T18:31:56.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 msgr2=0x7fe534196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.327+0000 7fe53b70b700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 0x7fe534196500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe53000e9a0 tx=0x7fe53000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 shutdown_connections 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5200a12c0 0x7fe5200a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5341054d0 0x7fe53419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5340680c0 0x7fe534196500 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 --2- 172.21.15.31:0/4051318345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe534068a60 0x7fe534196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 >> 172.21.15.31:0/4051318345 conn(0x7fe534075fd0 msgr2=0x7fe5340fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 shutdown_connections 2023-12-14T18:31:56.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:31:56.328+0000 7fe53b70b700 1 -- 172.21.15.31:0/4051318345 wait complete. 2023-12-14T18:31:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:56 smithi204 ceph-mon[43866]: pgmap v126: 129 pgs: 129 active+clean; 313 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 534 op/s 2023-12-14T18:31:56.344 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:56 smithi031 ceph-mon[42085]: pgmap v126: 129 pgs: 129 active+clean; 313 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 534 op/s 2023-12-14T18:31:56.345 INFO:tasks.cephfs.filesystem:scrub status for tag:e9c9a305-94ae-47d1-85f0-57f435df8e4b - {'path': '/', 'tag': 'e9c9a305-94ae-47d1-85f0-57f435df8e4b', 'options': 'recursive,force'} 2023-12-14T18:31:56.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:56 smithi176 ceph-mon[43506]: pgmap v126: 129 pgs: 129 active+clean; 313 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.8 MiB/s wr, 534 op/s 2023-12-14T18:31:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:31:58 smithi204 ceph-mon[43866]: pgmap v127: 129 pgs: 129 active+clean; 313 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2023-12-14T18:31:58.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:31:58 smithi031 ceph-mon[42085]: pgmap v127: 129 pgs: 129 active+clean; 313 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2023-12-14T18:31:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:31:58 smithi176 ceph-mon[43506]: pgmap v127: 129 pgs: 129 active+clean; 313 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2023-12-14T18:32:00.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:00 smithi204 ceph-mon[43866]: pgmap v128: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 1.13k op/s 2023-12-14T18:32:00.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:00 smithi031 ceph-mon[42085]: pgmap v128: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 1.13k op/s 2023-12-14T18:32:00.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:00 smithi176 ceph-mon[43506]: pgmap v128: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 1.13k op/s 2023-12-14T18:32:02.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:02 smithi204 ceph-mon[43866]: pgmap v129: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 960 op/s 2023-12-14T18:32:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:02 smithi031 ceph-mon[42085]: pgmap v129: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 960 op/s 2023-12-14T18:32:02.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:02 smithi176 ceph-mon[43506]: pgmap v129: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 960 op/s 2023-12-14T18:32:04.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:04 smithi204 ceph-mon[43866]: pgmap v130: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 960 op/s 2023-12-14T18:32:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:04 smithi031 ceph-mon[42085]: pgmap v130: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 960 op/s 2023-12-14T18:32:04.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:04 smithi176 ceph-mon[43506]: pgmap v130: 129 pgs: 129 active+clean; 314 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 960 op/s 2023-12-14T18:32:06.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:06 smithi204 ceph-mon[43866]: pgmap v131: 129 pgs: 129 active+clean; 330 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 1.14k op/s 2023-12-14T18:32:06.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:06 smithi031 ceph-mon[42085]: pgmap v131: 129 pgs: 129 active+clean; 330 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 1.14k op/s 2023-12-14T18:32:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:06 smithi176 ceph-mon[43506]: pgmap v131: 129 pgs: 129 active+clean; 330 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 1.14k op/s 2023-12-14T18:32:08.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:08 smithi204 ceph-mon[43866]: pgmap v132: 129 pgs: 129 active+clean; 330 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 976 op/s 2023-12-14T18:32:08.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:08 smithi031 ceph-mon[42085]: pgmap v132: 129 pgs: 129 active+clean; 330 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 976 op/s 2023-12-14T18:32:08.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:08 smithi176 ceph-mon[43506]: pgmap v132: 129 pgs: 129 active+clean; 330 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 976 op/s 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 -- 172.21.15.31:0/1344254721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ff230 msgr2=0x7eff380ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 --2- 172.21.15.31:0/1344254721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ff230 0x7eff380ff600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7eff340098f0 tx=0x7eff34009c00 comp rx=0 tx=0).stop 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 -- 172.21.15.31:0/1344254721 shutdown_connections 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 --2- 172.21.15.31:0/1344254721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff38104630 0x7eff38104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 --2- 172.21.15.31:0/1344254721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ff230 0x7eff380ff600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 --2- 172.21.15.31:0/1344254721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff380ffb40 0x7eff38104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 -- 172.21.15.31:0/1344254721 >> 172.21.15.31:0/1344254721 conn(0x7eff380763a0 msgr2=0x7eff380767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 -- 172.21.15.31:0/1344254721 shutdown_connections 2023-12-14T18:32:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.851+0000 7eff3f538700 1 -- 172.21.15.31:0/1344254721 wait complete. 2023-12-14T18:32:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 Processor -- start 2023-12-14T18:32:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 -- start start 2023-12-14T18:32:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff380ff230 0x7eff3819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ffb40 0x7eff3819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 0x7eff381a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381a1950 con 0x7eff38104630 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381a1ac0 con 0x7eff380ffb40 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3f538700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff381a1c80 con 0x7eff380ff230 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.852+0000 7eff3dad5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 0x7eff381a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 0x7eff381a1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55528/0 (socket says 172.21.15.31:55528) 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 -- 172.21.15.31:0/3903683278 learned_addr learned my addr 172.21.15.31:0/3903683278 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3cad3700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ffb40 0x7eff3819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 -- 172.21.15.31:0/3903683278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff380ff230 msgr2=0x7eff3819c9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff380ff230 0x7eff3819c9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 -- 172.21.15.31:0/3903683278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ffb40 msgr2=0x7eff3819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ffb40 0x7eff3819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 -- 172.21.15.31:0/3903683278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280095f0 con 0x7eff38104630 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff3dad5700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 0x7eff381a1380 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7eff3000ef30 tx=0x7eff3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.853+0000 7eff2e7fc700 1 -- 172.21.15.31:0/3903683278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30018070 con 0x7eff38104630 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.854+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff34009520 con 0x7eff38104630 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.854+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff38116870 con 0x7eff38104630 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.854+0000 7eff2e7fc700 1 -- 172.21.15.31:0/3903683278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff3000f040 con 0x7eff38104630 2023-12-14T18:32:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.854+0000 7eff2e7fc700 1 -- 172.21.15.31:0/3903683278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3000ccd0 con 0x7eff38104630 2023-12-14T18:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.857+0000 7eff2e7fc700 1 -- 172.21.15.31:0/3903683278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7eff30004ad0 con 0x7eff38104630 2023-12-14T18:32:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.857+0000 7eff2e7fc700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff240a12c0 0x7eff240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff2e7fc700 1 -- 172.21.15.31:0/3903683278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7eff300ccba0 con 0x7eff38104630 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff240a12c0 0x7eff240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 Processor -- start 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 -- start start 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff240a12c0 0x7eff240a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff34009230 tx=0x7eff34009f70 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab71f8e430 0x55ab72130e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab721317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab7213e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7201b580 con 0x55ab72131370 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7208a810 con 0x55ab7213c0f0 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab72083ca0 con 0x55ab71f8e430 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff3d2d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab71f8e430 0x55ab72130e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff3d2d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab71f8e430 0x55ab72130e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45972/0 (socket says 172.21.15.31:45972) 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab721317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3dad5700 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab7213e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.858+0000 7eff3d2d4700 1 -- 172.21.15.31:0/2809414110 learned_addr learned my addr 172.21.15.31:0/2809414110 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 -- 172.21.15.31:0/2809414110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab71f8e430 msgr2=0x55ab72130e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab71f8e430 0x55ab72130e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 -- 172.21.15.31:0/2809414110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 msgr2=0x55ab7213e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab7213e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 -- 172.21.15.31:0/2809414110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab71f05350 con 0x55ab72131370 2023-12-14T18:32:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3dad5700 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab7213e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff3cad3700 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab721317e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7eff3819d910 tx=0x7eff28011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5a8cbc74338ddc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff237fe700 1 -- 172.21.15.31:0/2809414110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28004310 con 0x55ab72131370 2023-12-14T18:32:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff237fe700 1 -- 172.21.15.31:0/2809414110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff28004470 con 0x55ab72131370 2023-12-14T18:32:08.863 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.859+0000 7eff237fe700 1 -- 172.21.15.31:0/2809414110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280199d0 con 0x55ab72131370 2023-12-14T18:32:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/2809414110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 msgr2=0x55ab721317e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab721317e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7eff3819d910 tx=0x7eff28011720 comp rx=0 tx=0).stop 2023-12-14T18:32:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/2809414110 shutdown_connections 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab7213e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab71f8e430 0x55ab72130e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/2809414110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab721317e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/2809414110 >> 172.21.15.31:0/2809414110 conn(0x55ab72127890 msgr2=0x55ab7212d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/2809414110 shutdown_connections 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.860+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/2809414110 wait complete. 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 -- start start 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 Processor -- start 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 0x55ab7213ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab72138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab72139230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7201ff30 con 0x55ab71f8e430 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7201fbd0 con 0x55ab7213c0f0 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab71f07640 con 0x55ab72131370 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.861+0000 7eff4b6c6b80 10 client.0 _ll_register_callbacks cb 0x55ab720438b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3d2d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 0x55ab7213ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3cad3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab72138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab72139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3d2d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 0x55ab7213ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55552/0 (socket says 172.21.15.31:55552) 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3cad3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab72138cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45974/0 (socket says 172.21.15.31:45974) 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3d2d4700 1 -- 172.21.15.31:0/3695776288 learned_addr learned my addr 172.21.15.31:0/3695776288 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 -- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 msgr2=0x55ab72138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab72138cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 -- 172.21.15.31:0/3695776288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 msgr2=0x55ab7213ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 0x55ab7213ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 -- 172.21.15.31:0/3695776288 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff28009b80 con 0x55ab7213c0f0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3cad3700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab72138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 0x55ab7213ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff3dad5700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab72139230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff300c9f60 tx=0x7eff300993c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41084e620c8417a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff20ff9700 1 -- 172.21.15.31:0/3695776288 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30099060 con 0x55ab7213c0f0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff20ff9700 1 -- 172.21.15.31:0/3695776288 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7eff300a1d30 con 0x55ab7213c0f0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab72124ea0 con 0x55ab7213c0f0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff20ff9700 1 -- 172.21.15.31:0/3695776288 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff300a18a0 con 0x55ab7213c0f0 2023-12-14T18:32:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff4b6c6b80 10 client.26438 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.862+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ab7204a110 con 0x55ab7213c0f0 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff20ff9700 1 -- 172.21.15.31:0/3695776288 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab72124ea0 con 0x55ab7213c0f0 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff20ff9700 1 -- 172.21.15.31:0/3695776288 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7eff300b90c0 con 0x55ab7213c0f0 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff4b6c6b80 10 client.26438 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff4b6c6b80 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff4b6c6b80 20 client.26438 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab7214ba50 0x55ab7214de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff4b6c6b80 4 client.26438 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab7207cc80 con 0x55ab7214ba50 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.863+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab7214ba50 0x55ab7214de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.864+0000 7eff20ff9700 10 client.26438 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:08.872 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.864+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab7214ba50 0x55ab7214de60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15037768591b54ea server_cookie=8e89428670ba6c21 in_seq=0 out_seq=0 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff20ff9700 1 -- 172.21.15.31:0/3695776288 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ab7207cc80 con 0x55ab7214ba50 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff20ff9700 10 client.26438 handle_command_reply: tid=0 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 client.26438 shutdown 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 msgr2=0x55ab72139230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab72139230 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7eff300c9f60 tx=0x7eff300993c0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 shutdown_connections 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab7214ba50 0x55ab7214de60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab7213c0f0 0x55ab72139230 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab71f8e430 0x55ab7213ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/3695776288 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab72138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff20ff9700 0 client.26438 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:08.873 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 >> 172.21.15.31:0/3695776288 conn(0x55ab72127890 msgr2=0x55ab7212cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 shutdown_connections 2023-12-14T18:32:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/3695776288 wait complete. 2023-12-14T18:32:08.874 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.867+0000 7eff4b6c6b80 20 client.26438 trim_cache size 0 max 16384 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 Processor -- start 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 -- start start 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72138490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab7213b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 0x55ab7213ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab72127290 con 0x55ab7213c0f0 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab721320a0 con 0x55ab71f8e430 2023-12-14T18:32:08.885 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab72132930 con 0x55ab72131370 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3dad5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 0x55ab7213ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3cad3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab7213b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3dad5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 0x55ab7213ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55568/0 (socket says 172.21.15.31:55568) 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3dad5700 1 -- 172.21.15.31:0/1975883682 learned_addr learned my addr 172.21.15.31:0/1975883682 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72138490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3cad3700 1 -- 172.21.15.31:0/1975883682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 msgr2=0x55ab72138490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3cad3700 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72138490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3cad3700 1 -- 172.21.15.31:0/1975883682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 msgr2=0x55ab7213ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3cad3700 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 0x55ab7213ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3cad3700 1 -- 172.21.15.31:0/1975883682 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab72049e60 con 0x55ab72131370 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3dad5700 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 0x55ab7213ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.886 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.883+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:32:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.884+0000 7eff3cad3700 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab7213b4e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff28005b40 tx=0x7eff28005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a42db282d9e4cd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.884+0000 7eff23fff700 1 -- 172.21.15.31:0/1975883682 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28005050 con 0x55ab72131370 2023-12-14T18:32:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.884+0000 7eff23fff700 1 -- 172.21.15.31:0/1975883682 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff28005670 con 0x55ab72131370 2023-12-14T18:32:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.884+0000 7eff23fff700 1 -- 172.21.15.31:0/1975883682 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280045e0 con 0x55ab72131370 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1975883682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 msgr2=0x55ab7213b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab7213b4e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7eff28005b40 tx=0x7eff28005e80 comp rx=0 tx=0).stop 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1975883682 shutdown_connections 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab7213c0f0 0x55ab7213ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72138490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1975883682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab72131370 0x55ab7213b4e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.891 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1975883682 >> 172.21.15.31:0/1975883682 conn(0x55ab72127890 msgr2=0x55ab72147e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1975883682 shutdown_connections 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1975883682 wait complete. 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 -- start start 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 Processor -- start 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.885+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72142670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab72142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff4b6c6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7213c0f0 0x55ab72140c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab721411a0 con 0x55ab72131370 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab72141310 con 0x55ab71f8e430 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff4b6c6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab721414d0 con 0x55ab7213c0f0 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff4b6c6b80 10 client.0 _ll_register_callbacks cb 0x7eff300997f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3cad3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab72142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3cad3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab72142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55578/0 (socket says 172.21.15.31:55578) 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3cad3700 1 -- 172.21.15.31:0/1387935713 learned_addr learned my addr 172.21.15.31:0/1387935713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7213c0f0 0x55ab72140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 -- 172.21.15.31:0/1387935713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 msgr2=0x55ab72142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 -- 172.21.15.31:0/1387935713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 msgr2=0x55ab72142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab72142bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 -- 172.21.15.31:0/1387935713 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab7204a110 con 0x55ab7213c0f0 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3cad3700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab72142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff3dad5700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7213c0f0 0x55ab72140c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff30095e70 tx=0x7eff300994b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85029452c4a76b53 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.886+0000 7eff20ff9700 1 -- 172.21.15.31:0/1387935713 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff300a1d30 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff28004470 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff20ff9700 1 -- 172.21.15.31:0/1387935713 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7eff300a18a0 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff20ff9700 1 -- 172.21.15.31:0/1387935713 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff3009a540 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 10 client.26464 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7eff2801e040 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff20ff9700 1 -- 172.21.15.31:0/1387935713 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff3007c080 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff20ff9700 1 -- 172.21.15.31:0/1387935713 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7eff300988d0 con 0x55ab7213c0f0 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 10 client.26464 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 10 client.26464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 20 client.26464 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:08.894 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab721a0750 0x55ab721a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 4 client.26464 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ab7207c9a0 con 0x55ab721a0750 2023-12-14T18:32:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.887+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab721a0750 0x55ab721a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.888+0000 7eff20ff9700 10 client.26464 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:08.888+0000 7eff3d2d4700 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab721a0750 0x55ab721a2b60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b069b33095425faa server_cookie=87679995278bfa81 in_seq=0 out_seq=0 2023-12-14T18:32:10.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:10 smithi204 ceph-mon[43866]: pgmap v133: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 1.15k op/s 2023-12-14T18:32:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:10 smithi031 ceph-mon[42085]: pgmap v133: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 1.15k op/s 2023-12-14T18:32:10.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:10 smithi176 ceph-mon[43506]: pgmap v133: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.4 MiB/s wr, 1.15k op/s 2023-12-14T18:32:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.500+0000 7eff20ff9700 1 -- 172.21.15.31:0/1387935713 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ab7207c9a0 con 0x55ab721a0750 2023-12-14T18:32:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.500+0000 7eff20ff9700 10 client.26464 handle_command_reply: tid=0 2023-12-14T18:32:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.500+0000 7eff4b6c6b80 1 client.26464 shutdown 2023-12-14T18:32:10.502 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.500+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7213c0f0 msgr2=0x55ab72140c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:10.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.500+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7213c0f0 0x55ab72140c10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7eff30095e70 tx=0x7eff300994b0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 shutdown_connections 2023-12-14T18:32:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ab721a0750 0x55ab721a2b60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab7213c0f0 0x55ab72140c10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ab71f8e430 0x55ab72142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 --2- 172.21.15.31:0/1387935713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ab72131370 0x55ab72142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 >> 172.21.15.31:0/1387935713 conn(0x55ab72127890 msgr2=0x55ab72132f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff20ff9700 0 client.26464 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 shutdown_connections 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.501+0000 7eff4b6c6b80 1 -- 172.21.15.31:0/1387935713 wait complete. 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.502+0000 7eff4b6c6b80 20 client.26464 trim_cache size 0 max 16384 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:32:10.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:32:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:32:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:32:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.502+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff240a12c0 msgr2=0x7eff240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.502+0000 7eff3f538700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff240a12c0 0x7eff240a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7eff34009230 tx=0x7eff34009f70 comp rx=0 tx=0).stop 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.502+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 msgr2=0x7eff381a1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.502+0000 7eff3f538700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 0x7eff381a1380 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7eff3000ef30 tx=0x7eff3000c3a0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 shutdown_connections 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff38104630 0x7eff381a1380 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7eff240a12c0 0x7eff240a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff380ff230 0x7eff3819c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 --2- 172.21.15.31:0/3903683278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff380ffb40 0x7eff3819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 >> 172.21.15.31:0/3903683278 conn(0x7eff380763a0 msgr2=0x7eff381020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:10.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 shutdown_connections 2023-12-14T18:32:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:10.503+0000 7eff3f538700 1 -- 172.21.15.31:0/3903683278 wait complete. 2023-12-14T18:32:12.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:12 smithi031 ceph-mon[42085]: pgmap v134: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2023-12-14T18:32:12.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:12 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:32:12.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:12 smithi176 ceph-mon[43506]: pgmap v134: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2023-12-14T18:32:12.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:12 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:32:12.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:12 smithi204 ceph-mon[43866]: pgmap v134: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2023-12-14T18:32:12.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:12 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:32:13.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:13 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:13 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:13 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:32:13.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:13 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:32:13.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:13 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:13 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:13 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:13 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:32:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:13 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:32:13.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:13 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:13 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:13 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:13 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:32:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:13 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:32:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:13 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:32:14.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:14 smithi204 ceph-mon[43866]: pgmap v135: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2023-12-14T18:32:14.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:14 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:32:14.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:14 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:32:14.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:14 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:32:14.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:14 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:32:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:14 smithi031 ceph-mon[42085]: pgmap v135: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2023-12-14T18:32:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:14 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:32:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:14 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:32:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:14 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:32:14.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:14 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:32:14.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:14 smithi176 ceph-mon[43506]: pgmap v135: 129 pgs: 129 active+clean; 356 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 355 op/s 2023-12-14T18:32:14.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:14 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:32:14.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:14 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:32:14.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:14 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:32:14.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:14 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:32:16.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:16 smithi204 ceph-mon[43866]: pgmap v136: 129 pgs: 129 active+clean; 359 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 769 op/s 2023-12-14T18:32:16.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:16 smithi031 ceph-mon[42085]: pgmap v136: 129 pgs: 129 active+clean; 359 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 769 op/s 2023-12-14T18:32:16.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:16 smithi176 ceph-mon[43506]: pgmap v136: 129 pgs: 129 active+clean; 359 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 769 op/s 2023-12-14T18:32:18.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:18 smithi204 ceph-mon[43866]: pgmap v137: 129 pgs: 129 active+clean; 359 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 590 op/s 2023-12-14T18:32:18.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:18 smithi031 ceph-mon[42085]: pgmap v137: 129 pgs: 129 active+clean; 359 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 590 op/s 2023-12-14T18:32:18.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:18 smithi176 ceph-mon[43506]: pgmap v137: 129 pgs: 129 active+clean; 359 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 590 op/s 2023-12-14T18:32:20.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:20 smithi204 ceph-mon[43866]: pgmap v138: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 821 op/s 2023-12-14T18:32:20.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:20 smithi031 ceph-mon[42085]: pgmap v138: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 821 op/s 2023-12-14T18:32:20.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:20 smithi176 ceph-mon[43506]: pgmap v138: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.7 MiB/s wr, 821 op/s 2023-12-14T18:32:22.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:22 smithi204 ceph-mon[43866]: pgmap v139: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 645 op/s 2023-12-14T18:32:22.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:22 smithi031 ceph-mon[42085]: pgmap v139: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 645 op/s 2023-12-14T18:32:22.743 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:22 smithi176 ceph-mon[43506]: pgmap v139: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 645 op/s 2023-12-14T18:32:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 -- 172.21.15.31:0/3425938922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ff230 msgr2=0x7f84980ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3425938922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ff230 0x7f84980ff600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8494009990 tx=0x7f8494009ca0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 -- 172.21.15.31:0/3425938922 shutdown_connections 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3425938922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8498104630 0x7f8498104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3425938922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ff230 0x7f84980ff600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3425938922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980ffb40 0x7f8498104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 -- 172.21.15.31:0/3425938922 >> 172.21.15.31:0/3425938922 conn(0x7f84980763a0 msgr2=0x7f84980767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 -- 172.21.15.31:0/3425938922 shutdown_connections 2023-12-14T18:32:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.677+0000 7f849f6d3700 1 -- 172.21.15.31:0/3425938922 wait complete. 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 Processor -- start 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 -- start start 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 0x7f84981a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ffb40 0x7f84981a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8498104630 0x7f849819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849819ced0 con 0x7f8498104630 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849819d040 con 0x7f84980ffb40 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849f6d3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849819d200 con 0x7f84980ff230 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849dc70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8498104630 0x7f849819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849d46f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 0x7f84981a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849dc70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8498104630 0x7f849819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33146/0 (socket says 172.21.15.31:33146) 2023-12-14T18:32:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849dc70700 1 -- 172.21.15.31:0/3627266438 learned_addr learned my addr 172.21.15.31:0/3627266438 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849d46f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 0x7f84981a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46594/0 (socket says 172.21.15.31:46594) 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.678+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ffb40 0x7f84981a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849d46f700 1 -- 172.21.15.31:0/3627266438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ffb40 msgr2=0x7f84981a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849d46f700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ffb40 0x7f84981a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849d46f700 1 -- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8498104630 msgr2=0x7f849819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849d46f700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8498104630 0x7f849819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849d46f700 1 -- 172.21.15.31:0/3627266438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490009c40 con 0x7f84980ff230 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849d46f700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 0x7f84981a27f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84940059d0 tx=0x7f8494005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f848e7fc700 1 -- 172.21.15.31:0/3627266438 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849401d070 con 0x7f84980ff230 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84940095f0 con 0x7f84980ff230 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f848e7fc700 1 -- 172.21.15.31:0/3627266438 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8494022470 con 0x7f84980ff230 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f848e7fc700 1 -- 172.21.15.31:0/3627266438 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494017650 con 0x7f84980ff230 2023-12-14T18:32:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.679+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8498116870 con 0x7f84980ff230 2023-12-14T18:32:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.682+0000 7f848e7fc700 1 -- 172.21.15.31:0/3627266438 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8494022890 con 0x7f84980ff230 2023-12-14T18:32:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.682+0000 7f848e7fc700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f84840a12c0 0x7f84840a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.683+0000 7f848e7fc700 1 -- 172.21.15.31:0/3627266438 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f84940ce350 con 0x7f84980ff230 2023-12-14T18:32:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.683+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f84840a12c0 0x7f84840a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.683+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f84840a12c0 0x7f84840a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8498068b60 tx=0x7f8488009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 Processor -- start 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 -- start start 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef7ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef7b370 0x55650ef7b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef860f0 0x55650ef884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ee65580 con 0x55650edd8430 2023-12-14T18:32:23.686 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650eed4810 con 0x55650ef860f0 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f84ab861b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650eecdca0 con 0x55650ef7b370 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f849d46f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef7ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f849cc6e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef7b370 0x55650ef7b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f849dc70700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef860f0 0x55650ef884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f849d46f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef7ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33162/0 (socket says 172.21.15.31:33162) 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.684+0000 7f849d46f700 1 -- 172.21.15.31:0/2979544361 learned_addr learned my addr 172.21.15.31:0/2979544361 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849cc6e700 1 -- 172.21.15.31:0/2979544361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef860f0 msgr2=0x55650ef884d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849cc6e700 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef860f0 0x55650ef884d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849cc6e700 1 -- 172.21.15.31:0/2979544361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 msgr2=0x55650ef7ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849cc6e700 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef7ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849cc6e700 1 -- 172.21.15.31:0/2979544361 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650ed4f350 con 0x55650ef7b370 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849d46f700 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef7ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:23.687 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849dc70700 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef860f0 0x55650ef884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f849cc6e700 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef7b370 0x55650ef7b7e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8488011c90 tx=0x7f8488014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=225f69dbc2c09281 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f84837fe700 1 -- 172.21.15.31:0/2979544361 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488004230 con 0x55650ef7b370 2023-12-14T18:32:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f84837fe700 1 -- 172.21.15.31:0/2979544361 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8488021070 con 0x55650ef7b370 2023-12-14T18:32:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.685+0000 7f84837fe700 1 -- 172.21.15.31:0/2979544361 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488003c40 con 0x55650ef7b370 2023-12-14T18:32:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 -- 172.21.15.31:0/2979544361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef7b370 msgr2=0x55650ef7b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef7b370 0x55650ef7b7e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8488011c90 tx=0x7f8488014000 comp rx=0 tx=0).stop 2023-12-14T18:32:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 -- 172.21.15.31:0/2979544361 shutdown_connections 2023-12-14T18:32:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef860f0 0x55650ef884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef7ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2979544361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef7b370 0x55650ef7b7e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 -- 172.21.15.31:0/2979544361 >> 172.21.15.31:0/2979544361 conn(0x55650ef71890 msgr2=0x55650ef77200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 -- 172.21.15.31:0/2979544361 shutdown_connections 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.686+0000 7f84ab861b80 1 -- 172.21.15.31:0/2979544361 wait complete. 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 -- start start 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 Processor -- start 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 0x55650ef85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650ef7b370 0x55650ef82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef83540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ee69f30 con 0x55650ef7b370 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ee69bd0 con 0x55650edd8430 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.687+0000 7f84ab861b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ed51640 con 0x55650ef860f0 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f84ab861b80 10 client.0 _ll_register_callbacks cb 0x55650ee8d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 0x55650ef85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 0x55650ef85b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:39028/0 (socket says 172.21.15.31:39028) 2023-12-14T18:32:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 -- 172.21.15.31:0/3481035414 learned_addr learned my addr 172.21.15.31:0/3481035414 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849dc70700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef83540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650ef7b370 0x55650ef82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 -- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 msgr2=0x55650ef83540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef83540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 -- 172.21.15.31:0/3481035414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650ef7b370 msgr2=0x55650ef82e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650ef7b370 0x55650ef82e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 -- 172.21.15.31:0/3481035414 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8488017070 con 0x55650edd8430 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650ef7b370 0x55650ef82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f849d46f700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 0x55650ef85b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8494097b90 tx=0x7f849409ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58fad51a139156a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f8480ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f8480ff9700 1 -- 172.21.15.31:0/3481035414 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940b2070 con 0x55650edd8430 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f8480ff9700 1 -- 172.21.15.31:0/3481035414 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f849409f150 con 0x55650edd8430 2023-12-14T18:32:23.693 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.688+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55650ef6eea0 con 0x55650edd8430 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f8480ff9700 1 -- 172.21.15.31:0/3481035414 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84940a34e0 con 0x55650edd8430 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 10 client.26456 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55650ee94110 con 0x55650edd8430 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f8480ff9700 1 -- 172.21.15.31:0/3481035414 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84940c9080 con 0x55650edd8430 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f8480ff9700 1 -- 172.21.15.31:0/3481035414 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f84940a3050 con 0x55650edd8430 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 10 client.26456 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 10 client.26456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 20 client.26456 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650ef95860 0x55650ef97c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 4 client.26456 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55650eec6c80 con 0x55650ef95860 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.689+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650ef95860 0x55650ef97c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.690+0000 7f849cc6e700 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650ef95860 0x55650ef97c70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34994420a22b0c25 server_cookie=c4572867dae1fe4d in_seq=0 out_seq=0 2023-12-14T18:32:23.694 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.690+0000 7f8480ff9700 10 client.26456 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.692+0000 7f8480ff9700 1 -- 172.21.15.31:0/3481035414 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55650eec6c80 con 0x55650ef95860 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.692+0000 7f8480ff9700 10 client.26456 handle_command_reply: tid=0 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.692+0000 7f84ab861b80 1 client.26456 shutdown 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 msgr2=0x55650ef85b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 0x55650ef85b50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8494097b90 tx=0x7f849409ad90 comp rx=0 tx=0).stop 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 shutdown_connections 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650ef95860 0x55650ef97c70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650edd8430 0x55650ef85b50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 --2- 172.21.15.31:0/3481035414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650ef7b370 0x55650ef82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 >> 172.21.15.31:0/3481035414 conn(0x55650ef71890 msgr2=0x55650ef76df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f8480ff9700 0 client.26456 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 shutdown_connections 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 1 -- 172.21.15.31:0/3481035414 wait complete. 2023-12-14T18:32:23.696 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.693+0000 7f84ab861b80 20 client.26456 trim_cache size 0 max 16384 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.710+0000 7f84ab861b80 1 Processor -- start 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 -- start start 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef91460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef97870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef85610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ef843e0 con 0x55650edd8430 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ef6ea70 con 0x55650ef7b370 2023-12-14T18:32:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f84ab861b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ef77e40 con 0x55650ef860f0 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef91460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef91460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33184/0 (socket says 172.21.15.31:33184) 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 -- 172.21.15.31:0/550040015 learned_addr learned my addr 172.21.15.31:0/550040015 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849cc6e700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef97870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849dc70700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef85610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 -- 172.21.15.31:0/550040015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 msgr2=0x55650ef85610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef85610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 -- 172.21.15.31:0/550040015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 msgr2=0x55650ef97870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef97870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849d46f700 1 -- 172.21.15.31:0/550040015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55650ee93e60 con 0x55650edd8430 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849dc70700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef85610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.711+0000 7f849cc6e700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef97870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.712+0000 7f849d46f700 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef91460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8494099470 tx=0x7f849409b2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc6554324f9b259 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.712+0000 7f8483fff700 1 -- 172.21.15.31:0/550040015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849409f150 con 0x55650edd8430 2023-12-14T18:32:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.712+0000 7f8483fff700 1 -- 172.21.15.31:0/550040015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55650ef6eea0 con 0x55650edd8430 2023-12-14T18:32:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.712+0000 7f8483fff700 1 -- 172.21.15.31:0/550040015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940a3050 con 0x55650edd8430 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 -- 172.21.15.31:0/550040015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 msgr2=0x55650ef91460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef91460 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8494099470 tx=0x7f849409b2a0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 -- 172.21.15.31:0/550040015 shutdown_connections 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650ef85610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef91460 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 --2- 172.21.15.31:0/550040015 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef97870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 -- 172.21.15.31:0/550040015 >> 172.21.15.31:0/550040015 conn(0x55650ef71890 msgr2=0x55650ef7e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 -- 172.21.15.31:0/550040015 shutdown_connections 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 -- 172.21.15.31:0/550040015 wait complete. 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 -- start start 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.713+0000 7f84ab861b80 1 Processor -- start 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef8aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef8af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650efe8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650ef8b6e0 con 0x55650edd8430 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650efe8c90 con 0x55650ef7b370 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55650efe8e50 con 0x55650ef860f0 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f84ab861b80 10 client.0 _ll_register_callbacks cb 0x55650ee8d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef8af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef8af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:39056/0 (socket says 172.21.15.31:39056) 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849d46f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef8aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 -- 172.21.15.31:0/2719121378 learned_addr learned my addr 172.21.15.31:0/2719121378 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849dc70700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650efe8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 -- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 msgr2=0x55650efe8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650efe8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 -- 172.21.15.31:0/2719121378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 msgr2=0x55650ef8aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef8aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849cc6e700 1 -- 172.21.15.31:0/2719121378 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55650ed4f350 con 0x55650ef7b370 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.714+0000 7f849d46f700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef8aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f849cc6e700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef8af40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f848800e690 tx=0x7f8488005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8266366dffa4173 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:23.719 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f8480ff9700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f8480ff9700 1 -- 172.21.15.31:0/2719121378 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848802b040 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f8480ff9700 1 -- 172.21.15.31:0/2719121378 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f848800ed80 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8488021070 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f8480ff9700 1 -- 172.21.15.31:0/2719121378 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84880303a0 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f84ab861b80 10 client.26462 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55650ee94110 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f8480ff9700 1 -- 172.21.15.31:0/2719121378 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8488021070 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f8480ff9700 1 -- 172.21.15.31:0/2719121378 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8488005440 con 0x55650ef7b370 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f84ab861b80 10 client.26462 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f84ab861b80 10 client.26462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.715+0000 7f84ab861b80 20 client.26462 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.716+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650efea5e0 0x55650efec9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.716+0000 7f84ab861b80 4 client.26462 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.716+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55650eec69a0 con 0x55650efea5e0 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.716+0000 7f849d46f700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650efea5e0 0x55650efec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.716+0000 7f849d46f700 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650efea5e0 0x55650efec9f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e56092d2bae7bf server_cookie=26f4c6078417bfed in_seq=0 out_seq=0 2023-12-14T18:32:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:23.716+0000 7f8480ff9700 10 client.26462 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:24 smithi204 ceph-mon[43866]: pgmap v140: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 645 op/s 2023-12-14T18:32:24.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:24 smithi031 ceph-mon[42085]: pgmap v140: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 645 op/s 2023-12-14T18:32:24.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:24 smithi176 ceph-mon[43506]: pgmap v140: 129 pgs: 129 active+clean; 380 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 645 op/s 2023-12-14T18:32:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.341+0000 7f8480ff9700 1 -- 172.21.15.31:0/2719121378 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55650eec69a0 con 0x55650efea5e0 2023-12-14T18:32:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.341+0000 7f8480ff9700 10 client.26462 handle_command_reply: tid=0 2023-12-14T18:32:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.341+0000 7f84ab861b80 1 client.26462 shutdown 2023-12-14T18:32:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.342+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 msgr2=0x55650ef8af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.342+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef8af40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f848800e690 tx=0x7f8488005d30 comp rx=0 tx=0).stop 2023-12-14T18:32:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 shutdown_connections 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55650efea5e0 0x55650efec9f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55650ef860f0 0x55650efe8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55650edd8430 0x55650ef8aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 --2- 172.21.15.31:0/2719121378 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55650ef7b370 0x55650ef8af40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 >> 172.21.15.31:0/2719121378 conn(0x55650ef71890 msgr2=0x55650ef91460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f8480ff9700 0 client.26462 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 shutdown_connections 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.343+0000 7f84ab861b80 1 -- 172.21.15.31:0/2719121378 wait complete. 2023-12-14T18:32:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.344+0000 7f84ab861b80 20 client.26462 trim_cache size 0 max 16384 2023-12-14T18:32:26.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:32:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:32:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:32:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:32:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:32:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.345+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f84840a12c0 msgr2=0x7f84840a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.345+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f84840a12c0 0x7f84840a3770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8498068b60 tx=0x7f8488009260 comp rx=0 tx=0).stop 2023-12-14T18:32:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.345+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 msgr2=0x7f84981a27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.345+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 0x7f84981a27f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84940059d0 tx=0x7f8494005720 comp rx=0 tx=0).stop 2023-12-14T18:32:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.346+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 shutdown_connections 2023-12-14T18:32:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.346+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8498104630 0x7f849819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.346+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f84840a12c0 0x7f84840a3770 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8498068b60 tx=0x7f8488009260 comp rx=0 tx=0).stop 2023-12-14T18:32:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.346+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84980ff230 0x7f84981a27f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.346+0000 7f849f6d3700 1 --2- 172.21.15.31:0/3627266438 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f84980ffb40 0x7f84981a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.346+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 >> 172.21.15.31:0/3627266438 conn(0x7f84980763a0 msgr2=0x7f84981020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.348+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 shutdown_connections 2023-12-14T18:32:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:26.348+0000 7f849f6d3700 1 -- 172.21.15.31:0/3627266438 wait complete. 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 -- 172.21.15.31:0/4094610621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 msgr2=0x7f9334101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 --2- 172.21.15.31:0/4094610621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334101020 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9324009990 tx=0x7f9324009ca0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 -- 172.21.15.31:0/4094610621 shutdown_connections 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 --2- 172.21.15.31:0/4094610621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9334105550 0x7f933410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 --2- 172.21.15.31:0/4094610621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334101020 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 --2- 172.21.15.31:0/4094610621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93341015f0 0x7f9334104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.547+0000 7f9339aa4700 1 -- 172.21.15.31:0/4094610621 >> 172.21.15.31:0/4094610621 conn(0x7f9334077570 msgr2=0x7f9334077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.548+0000 7f9339aa4700 1 -- 172.21.15.31:0/4094610621 shutdown_connections 2023-12-14T18:32:26.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.548+0000 7f9339aa4700 1 -- 172.21.15.31:0/4094610621 wait complete. 2023-12-14T18:32:26.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.548+0000 7f9339aa4700 1 Processor -- start 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 -- start start 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93341015f0 0x7f9334199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9334105550 0x7f933419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933419c600 con 0x7f9334105550 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933419c770 con 0x7f9334100c50 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.549+0000 7f9339aa4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933419c930 con 0x7f93341015f0 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.550+0000 7f93337fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.550+0000 7f93337fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:47210/0 (socket says 172.21.15.31:47210) 2023-12-14T18:32:26.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.550+0000 7f93337fe700 1 -- 172.21.15.31:0/594801667 learned_addr learned my addr 172.21.15.31:0/594801667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:26.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.550+0000 7f9332ffd700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93341015f0 0x7f9334199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.552+0000 7f9333fff700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9334105550 0x7f933419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.552+0000 7f93337fe700 1 -- 172.21.15.31:0/594801667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93341015f0 msgr2=0x7f9334199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.552+0000 7f93337fe700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93341015f0 0x7f9334199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.552+0000 7f93337fe700 1 -- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9334105550 msgr2=0x7f933419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.552+0000 7f93337fe700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9334105550 0x7f933419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.552+0000 7f93337fe700 1 -- 172.21.15.31:0/594801667 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240095f0 con 0x7f9334100c50 2023-12-14T18:32:26.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.553+0000 7f93337fe700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334198de0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f932400c010 tx=0x7f932400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.554+0000 7f9330ff9700 1 -- 172.21.15.31:0/594801667 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932401d070 con 0x7f9334100c50 2023-12-14T18:32:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.554+0000 7f9330ff9700 1 -- 172.21.15.31:0/594801667 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9324003680 con 0x7f9334100c50 2023-12-14T18:32:26.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.554+0000 7f9330ff9700 1 -- 172.21.15.31:0/594801667 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324004bf0 con 0x7f9334100c50 2023-12-14T18:32:26.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.554+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9334073630 con 0x7f9334100c50 2023-12-14T18:32:26.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.554+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340734a0 con 0x7f9334100c50 2023-12-14T18:32:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.560+0000 7f9330ff9700 1 -- 172.21.15.31:0/594801667 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9324004d50 con 0x7f9334100c50 2023-12-14T18:32:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.561+0000 7f9330ff9700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f931c0a1410 0x7f931c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.561+0000 7f9330ff9700 1 -- 172.21.15.31:0/594801667 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f93240ce030 con 0x7f9334100c50 2023-12-14T18:32:26.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.566+0000 7f9345c32b80 1 Processor -- start 2023-12-14T18:32:26.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.568+0000 7f9332ffd700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f931c0a1410 0x7f931c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 -- start start 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c4b90 0x5594b56c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 0x5594b56d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b55a65e0 con 0x5594b56d0360 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b5615870 con 0x5594b56c4b90 2023-12-14T18:32:26.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.570+0000 7f9345c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b560ed00 con 0x5594b56c54a0 2023-12-14T18:32:26.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.571+0000 7f9332ffd700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f931c0a1410 0x7f931c0a38c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f933410d500 tx=0x7f93280091c0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:26.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.572+0000 7f9333fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 0x5594b56d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.572+0000 7f9333fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 0x5594b56d2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33822/0 (socket says 172.21.15.31:33822) 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.572+0000 7f9333fff700 1 -- 172.21.15.31:0/3218798570 learned_addr learned my addr 172.21.15.31:0/3218798570 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.573+0000 7f9333fff700 1 -- 172.21.15.31:0/3218798570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 msgr2=0x5594b56cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.573+0000 7f9333fff700 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.573+0000 7f9333fff700 1 -- 172.21.15.31:0/3218798570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c4b90 msgr2=0x5594b56c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.573+0000 7f9333fff700 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c4b90 0x5594b56c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.573+0000 7f9333fff700 1 -- 172.21.15.31:0/3218798570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594b54903b0 con 0x5594b56d0360 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.575+0000 7f9333fff700 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 0x5594b56d2740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b500 tx=0x7f932c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2eecbf54b680823 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:26.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.576+0000 7f9319ffb700 1 -- 172.21.15.31:0/3218798570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c010040 con 0x5594b56d0360 2023-12-14T18:32:26.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.576+0000 7f9319ffb700 1 -- 172.21.15.31:0/3218798570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c00acb0 con 0x5594b56d0360 2023-12-14T18:32:26.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.576+0000 7f9319ffb700 1 -- 172.21.15.31:0/3218798570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c014d10 con 0x5594b56d0360 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 -- 172.21.15.31:0/3218798570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 msgr2=0x5594b56d2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 0x5594b56d2740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f932c00b500 tx=0x7f932c009630 comp rx=0 tx=0).stop 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 -- 172.21.15.31:0/3218798570 shutdown_connections 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56d0360 0x5594b56d2740 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c4b90 0x5594b56c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 --2- 172.21.15.31:0/3218798570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.582+0000 7f9345c32b80 1 -- 172.21.15.31:0/3218798570 >> 172.21.15.31:0/3218798570 conn(0x5594b56b2ca0 msgr2=0x5594b56b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.583+0000 7f9345c32b80 1 -- 172.21.15.31:0/3218798570 shutdown_connections 2023-12-14T18:32:26.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.583+0000 7f9345c32b80 1 -- 172.21.15.31:0/3218798570 wait complete. 2023-12-14T18:32:26.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.583+0000 7f9345c32b80 1 -- start start 2023-12-14T18:32:26.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 Processor -- start 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b56cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b55aaf90 con 0x5594b56c4b90 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b55aac30 con 0x5594b56d0360 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b54926a0 con 0x5594b56c54a0 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.584+0000 7f9345c32b80 10 client.0 _ll_register_callbacks cb 0x5594b55ce910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.585+0000 7f93337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b56cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.585+0000 7f9333fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.585+0000 7f9333fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:47230/0 (socket says 172.21.15.31:47230) 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.585+0000 7f93337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b56cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33826/0 (socket says 172.21.15.31:33826) 2023-12-14T18:32:26.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.585+0000 7f9333fff700 1 -- 172.21.15.31:0/2741316482 learned_addr learned my addr 172.21.15.31:0/2741316482 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.586+0000 7f9332ffd700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.586+0000 7f93337fe700 1 -- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 msgr2=0x5594b56cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.586+0000 7f93337fe700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.586+0000 7f93337fe700 1 -- 172.21.15.31:0/2741316482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 msgr2=0x5594b56cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.586+0000 7f93337fe700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.587+0000 7f93337fe700 1 -- 172.21.15.31:0/2741316482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c009ce0 con 0x5594b56c4b90 2023-12-14T18:32:26.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.587+0000 7f9333fff700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:26.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:26 smithi204 ceph-mon[43866]: pgmap v141: 129 pgs: 129 active+clean; 404 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 879 op/s 2023-12-14T18:32:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.594+0000 7f93337fe700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b56cfd20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f93240cb3f0 tx=0x7f932409a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a6c08338b9527c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.594+0000 7f9312ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:32:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.596+0000 7f9312ffd700 1 -- 172.21.15.31:0/2741316482 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932409ee30 con 0x5594b56c4b90 2023-12-14T18:32:26.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.596+0000 7f9312ffd700 1 -- 172.21.15.31:0/2741316482 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f93240a31c0 con 0x5594b56c4b90 2023-12-14T18:32:26.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.598+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594b56b02b0 con 0x5594b56c4b90 2023-12-14T18:32:26.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.602+0000 7f9312ffd700 1 -- 172.21.15.31:0/2741316482 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f932409a2f0 con 0x5594b56c4b90 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.604+0000 7f9345c32b80 10 client.16647 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.604+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5594b55d5170 con 0x5594b56c4b90 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9312ffd700 1 -- 172.21.15.31:0/2741316482 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594b56b02b0 con 0x5594b56c4b90 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9312ffd700 1 -- 172.21.15.31:0/2741316482 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9324005240 con 0x5594b56c4b90 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9345c32b80 10 client.16647 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9345c32b80 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9345c32b80 20 client.16647 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9345c32b80 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b56f9110 0x5594b56fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9345c32b80 4 client.16647 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:26.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.605+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594b5607ce0 con 0x5594b56f9110 2023-12-14T18:32:26.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.606+0000 7f9332ffd700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b56f9110 0x5594b56fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.610+0000 7f9332ffd700 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b56f9110 0x5594b56fb4f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aceddb84b419005f server_cookie=803e4e8ef437efc7 in_seq=0 out_seq=0 2023-12-14T18:32:26.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.610+0000 7f9312ffd700 10 client.16647 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:26.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.614+0000 7f9312ffd700 1 -- 172.21.15.31:0/2741316482 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5594b5607ce0 con 0x5594b56f9110 2023-12-14T18:32:26.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.614+0000 7f9312ffd700 10 client.16647 handle_command_reply: tid=0 2023-12-14T18:32:26.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.614+0000 7f9345c32b80 1 client.16647 shutdown 2023-12-14T18:32:26.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.615+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 msgr2=0x5594b56cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.615+0000 7f9345c32b80 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b56cfd20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f93240cb3f0 tx=0x7f932409a7e0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 shutdown_connections 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9345c32b80 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b56f9110 0x5594b56fb4f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9345c32b80 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9345c32b80 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b56cfd20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9345c32b80 1 --2- 172.21.15.31:0/2741316482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c54a0 0x5594b56cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 >> 172.21.15.31:0/2741316482 conn(0x5594b56b2ca0 msgr2=0x5594b56b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.616+0000 7f9312ffd700 0 client.16647 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:26.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.617+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 shutdown_connections 2023-12-14T18:32:26.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.619+0000 7f9345c32b80 1 -- 172.21.15.31:0/2741316482 wait complete. 2023-12-14T18:32:26.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.620+0000 7f9345c32b80 20 client.16647 trim_cache size 0 max 16384 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 Processor -- start 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 -- start start 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 0x5594b56c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c54a0 0x5594b56be2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b56c1b70 con 0x5594b56c54a0 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b56c5de0 con 0x5594b56d0360 2023-12-14T18:32:26.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.646+0000 7f9345c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b56b2820 con 0x5594b56c4b90 2023-12-14T18:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f93337fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 0x5594b56c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f93337fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 0x5594b56c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:35020/0 (socket says 172.21.15.31:35020) 2023-12-14T18:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f93337fe700 1 -- 172.21.15.31:0/662512587 learned_addr learned my addr 172.21.15.31:0/662512587 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f9332ffd700 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c54a0 0x5594b56be2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f9333fff700 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f93337fe700 1 -- 172.21.15.31:0/662512587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 msgr2=0x5594b56c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.647+0000 7f93337fe700 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.648+0000 7f93337fe700 1 -- 172.21.15.31:0/662512587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c54a0 msgr2=0x5594b56be2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.648+0000 7f93337fe700 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c54a0 0x5594b56be2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.648+0000 7f93337fe700 1 -- 172.21.15.31:0/662512587 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594b55d4ec0 con 0x5594b56c4b90 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.648+0000 7f9332ffd700 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c54a0 0x5594b56be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.648+0000 7f93337fe700 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 0x5594b56c7ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93240cb9f0 tx=0x7f93240993b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f97fb408780e3604 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.649+0000 7f931a7fc700 1 -- 172.21.15.31:0/662512587 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240092b0 con 0x5594b56c4b90 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.649+0000 7f931a7fc700 1 -- 172.21.15.31:0/662512587 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93240a31c0 con 0x5594b56c4b90 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.649+0000 7f931a7fc700 1 -- 172.21.15.31:0/662512587 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240a2d30 con 0x5594b56c4b90 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.649+0000 7f9345c32b80 1 -- 172.21.15.31:0/662512587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 msgr2=0x5594b56c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.649+0000 7f9345c32b80 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 0x5594b56c7ac0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93240cb9f0 tx=0x7f93240993b0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 -- 172.21.15.31:0/662512587 shutdown_connections 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56d0360 0x5594b56c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56c4b90 0x5594b56c7ac0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 --2- 172.21.15.31:0/662512587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c54a0 0x5594b56be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 -- 172.21.15.31:0/662512587 >> 172.21.15.31:0/662512587 conn(0x5594b56b2ca0 msgr2=0x5594b56c1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 -- 172.21.15.31:0/662512587 shutdown_connections 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 -- 172.21.15.31:0/662512587 wait complete. 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.650+0000 7f9345c32b80 1 -- start start 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 Processor -- start 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b5740ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c54a0 0x5594b5741430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56d0360 0x5594b574bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b5741c80 con 0x5594b56c4b90 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b574c4b0 con 0x5594b56c54a0 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594b574c5f0 con 0x5594b56d0360 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.651+0000 7f9345c32b80 10 client.0 _ll_register_callbacks cb 0x7f932409abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.652+0000 7f93337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b5740ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.652+0000 7f93337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b5740ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33850/0 (socket says 172.21.15.31:33850) 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.652+0000 7f93337fe700 1 -- 172.21.15.31:0/4204670094 learned_addr learned my addr 172.21.15.31:0/4204670094 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.652+0000 7f9332ffd700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c54a0 0x5594b5741430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.652+0000 7f9333fff700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56d0360 0x5594b574bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f93337fe700 1 -- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56d0360 msgr2=0x5594b574bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f93337fe700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56d0360 0x5594b574bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f93337fe700 1 -- 172.21.15.31:0/4204670094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c54a0 msgr2=0x5594b5741430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f93337fe700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c54a0 0x5594b5741430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f93337fe700 1 -- 172.21.15.31:0/4204670094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594b55d5170 con 0x5594b56c4b90 2023-12-14T18:32:26.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f93337fe700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b5740ef0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f93240b92f0 tx=0x7f93240ce1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20de2bc730d785a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.653+0000 7f9312ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:32:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.654+0000 7f9312ffd700 1 -- 172.21.15.31:0/4204670094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240092b0 con 0x5594b56c4b90 2023-12-14T18:32:26.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.654+0000 7f9312ffd700 1 -- 172.21.15.31:0/4204670094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f932409ee30 con 0x5594b56c4b90 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.654+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f932c00acb0 con 0x5594b56c4b90 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.654+0000 7f9312ffd700 1 -- 172.21.15.31:0/4204670094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93240cb300 con 0x5594b56c4b90 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.654+0000 7f9345c32b80 10 client.16653 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.654+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5594b55bb9a0 con 0x5594b56c4b90 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9312ffd700 1 -- 172.21.15.31:0/4204670094 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f932c00acb0 con 0x5594b56c4b90 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9312ffd700 1 -- 172.21.15.31:0/4204670094 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f93240c5020 con 0x5594b56c4b90 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9345c32b80 10 client.16653 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9345c32b80 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9345c32b80 20 client.16653 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9345c32b80 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b574d7d0 0x5594b574fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9345c32b80 4 client.16653 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:32:26.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.655+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5594b5607a00 con 0x5594b574d7d0 2023-12-14T18:32:26.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.659+0000 7f9332ffd700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b574d7d0 0x5594b574fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:26.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.660+0000 7f9332ffd700 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b574d7d0 0x5594b574fbb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465cd6ee58d33991 server_cookie=40132d9555dfb464 in_seq=0 out_seq=0 2023-12-14T18:32:26.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.660+0000 7f9312ffd700 10 client.16653 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.675+0000 7f9312ffd700 1 -- 172.21.15.31:0/4204670094 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5594b5607a00 con 0x5594b574d7d0 2023-12-14T18:32:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.675+0000 7f9312ffd700 10 client.16653 handle_command_reply: tid=0 2023-12-14T18:32:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.675+0000 7f9345c32b80 1 client.16653 shutdown 2023-12-14T18:32:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.675+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 msgr2=0x5594b5740ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.675+0000 7f9345c32b80 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b5740ef0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f93240b92f0 tx=0x7f93240ce1f0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 shutdown_connections 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9345c32b80 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5594b574d7d0 0x5594b574fbb0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9345c32b80 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594b56d0360 0x5594b574bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9345c32b80 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5594b56c4b90 0x5594b5740ef0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9345c32b80 1 --2- 172.21.15.31:0/4204670094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5594b56c54a0 0x5594b5741430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 >> 172.21.15.31:0/4204670094 conn(0x5594b56b2ca0 msgr2=0x5594b56c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.676+0000 7f9312ffd700 0 client.16653 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.677+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 shutdown_connections 2023-12-14T18:32:26.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.677+0000 7f9345c32b80 1 -- 172.21.15.31:0/4204670094 wait complete. 2023-12-14T18:32:26.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.680+0000 7f9345c32b80 20 client.16653 trim_cache size 0 max 16384 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: "e9c9a305-94ae-47d1-85f0-57f435df8e4b": { 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e9c9a305-94ae-47d1-85f0-57f435df8e4b", 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:32:26.684 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:32:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.686+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f931c0a1410 msgr2=0x7f931c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.686+0000 7f9339aa4700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f931c0a1410 0x7f931c0a38c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f933410d500 tx=0x7f93280091c0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.686+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 msgr2=0x7f9334198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:26.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.686+0000 7f9339aa4700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334198de0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f932400c010 tx=0x7f932400f690 comp rx=0 tx=0).stop 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.693+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 shutdown_connections 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.693+0000 7f9339aa4700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9334105550 0x7f933419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.693+0000 7f9339aa4700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9334100c50 0x7f9334198de0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.693+0000 7f9339aa4700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f931c0a1410 0x7f931c0a38c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.693+0000 7f9339aa4700 1 --2- 172.21.15.31:0/594801667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93341015f0 0x7f9334199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.693+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 >> 172.21.15.31:0/594801667 conn(0x7f9334077570 msgr2=0x7f9334103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:26.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.694+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 shutdown_connections 2023-12-14T18:32:26.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:26.697+0000 7f9339aa4700 1 -- 172.21.15.31:0/594801667 wait complete. 2023-12-14T18:32:26.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:26 smithi031 ceph-mon[42085]: pgmap v141: 129 pgs: 129 active+clean; 404 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 879 op/s 2023-12-14T18:32:26.730 INFO:tasks.cephfs.filesystem:scrub status for tag:e9c9a305-94ae-47d1-85f0-57f435df8e4b - {'path': '/', 'tag': 'e9c9a305-94ae-47d1-85f0-57f435df8e4b', 'options': 'recursive,force'} 2023-12-14T18:32:26.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:26 smithi176 ceph-mon[43506]: pgmap v141: 129 pgs: 129 active+clean; 404 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 879 op/s 2023-12-14T18:32:28.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:28 smithi204 ceph-mon[43866]: pgmap v142: 129 pgs: 129 active+clean; 404 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 464 op/s 2023-12-14T18:32:28.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:28 smithi031 ceph-mon[42085]: pgmap v142: 129 pgs: 129 active+clean; 404 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 464 op/s 2023-12-14T18:32:28.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:28 smithi176 ceph-mon[43506]: pgmap v142: 129 pgs: 129 active+clean; 404 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 464 op/s 2023-12-14T18:32:30.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:30 smithi204 ceph-mon[43866]: pgmap v143: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 682 op/s 2023-12-14T18:32:30.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:30 smithi031 ceph-mon[42085]: pgmap v143: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 682 op/s 2023-12-14T18:32:30.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:30 smithi176 ceph-mon[43506]: pgmap v143: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.4 MiB/s wr, 682 op/s 2023-12-14T18:32:32.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:32 smithi204 ceph-mon[43866]: pgmap v144: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2023-12-14T18:32:32.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:32 smithi031 ceph-mon[42085]: pgmap v144: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2023-12-14T18:32:32.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:32 smithi176 ceph-mon[43506]: pgmap v144: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2023-12-14T18:32:33.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:33 smithi204 ceph-mon[43866]: pgmap v145: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2023-12-14T18:32:33.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:33 smithi031 ceph-mon[42085]: pgmap v145: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2023-12-14T18:32:33.745 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:33 smithi176 ceph-mon[43506]: pgmap v145: 129 pgs: 129 active+clean; 406 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2023-12-14T18:32:35.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:35 smithi031 ceph-mon[42085]: pgmap v146: 129 pgs: 129 active+clean; 425 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 660 op/s 2023-12-14T18:32:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:35 smithi176 ceph-mon[43506]: pgmap v146: 129 pgs: 129 active+clean; 425 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 660 op/s 2023-12-14T18:32:35.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:35 smithi204 ceph-mon[43866]: pgmap v146: 129 pgs: 129 active+clean; 425 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 660 op/s 2023-12-14T18:32:37.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:37 smithi031 ceph-mon[42085]: pgmap v147: 129 pgs: 129 active+clean; 425 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 426 op/s 2023-12-14T18:32:37.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:37 smithi176 ceph-mon[43506]: pgmap v147: 129 pgs: 129 active+clean; 425 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 426 op/s 2023-12-14T18:32:37.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:37 smithi204 ceph-mon[43866]: pgmap v147: 129 pgs: 129 active+clean; 425 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 426 op/s 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 -- 172.21.15.31:0/508391167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 msgr2=0x7fa278104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 --2- 172.21.15.31:0/508391167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278104e50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa268009920 tx=0x7fa268009c30 comp rx=0 tx=0).stop 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 -- 172.21.15.31:0/508391167 shutdown_connections 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 --2- 172.21.15.31:0/508391167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa278105580 0x7fa27810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 --2- 172.21.15.31:0/508391167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 0x7fa2780fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 --2- 172.21.15.31:0/508391167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278104e50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.488+0000 7fa280160700 1 -- 172.21.15.31:0/508391167 >> 172.21.15.31:0/508391167 conn(0x7fa278075810 msgr2=0x7fa278075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.489+0000 7fa280160700 1 -- 172.21.15.31:0/508391167 shutdown_connections 2023-12-14T18:32:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.489+0000 7fa280160700 1 -- 172.21.15.31:0/508391167 wait complete. 2023-12-14T18:32:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.489+0000 7fa280160700 1 Processor -- start 2023-12-14T18:32:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.489+0000 7fa280160700 1 -- start start 2023-12-14T18:32:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.489+0000 7fa280160700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 0x7fa2781964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa280160700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa280160700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa278105580 0x7fa27819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa280160700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819b4a0 con 0x7fa2780ff380 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa280160700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819b610 con 0x7fa2780fe9e0 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa280160700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27819b7d0 con 0x7fa278105580 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27defc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 0x7fa2781964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33948/0 (socket says 172.21.15.31:33948) 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 -- 172.21.15.31:0/3788413127 learned_addr learned my addr 172.21.15.31:0/3788413127 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa278105580 0x7fa27819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 -- 172.21.15.31:0/3788413127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa278105580 msgr2=0x7fa27819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa278105580 0x7fa27819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 -- 172.21.15.31:0/3788413127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 msgr2=0x7fa2781964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 0x7fa2781964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 -- 172.21.15.31:0/3788413127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268009520 con 0x7fa2780ff380 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27defc700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 0x7fa2781964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.490+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278196a00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa268009230 tx=0x7fa26800fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.491+0000 7fa26effd700 1 -- 172.21.15.31:0/3788413127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26801c070 con 0x7fa2780ff380 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.491+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa278110320 con 0x7fa2780ff380 2023-12-14T18:32:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.491+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa278110190 con 0x7fa2780ff380 2023-12-14T18:32:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.491+0000 7fa26effd700 1 -- 172.21.15.31:0/3788413127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa26800bb20 con 0x7fa2780ff380 2023-12-14T18:32:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.491+0000 7fa26effd700 1 -- 172.21.15.31:0/3788413127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa268003900 con 0x7fa2780ff380 2023-12-14T18:32:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.493+0000 7fa26effd700 1 -- 172.21.15.31:0/3788413127 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa268020030 con 0x7fa2780ff380 2023-12-14T18:32:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.494+0000 7fa26effd700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa2640a1340 0x7fa2640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.494+0000 7fa26effd700 1 -- 172.21.15.31:0/3788413127 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa2680cdeb0 con 0x7fa2780ff380 2023-12-14T18:32:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.494+0000 7fa27defc700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa2640a1340 0x7fa2640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa27defc700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa2640a1340 0x7fa2640a37f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa274005fd0 tx=0x7fa274009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 Processor -- start 2023-12-14T18:32:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 -- start start 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f8ff8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f8ff97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f90040f0 0x55f4f90064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8ee3580 con 0x55f4f8ff9370 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8f52810 con 0x55f4f8e56430 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.495+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8f4bca0 con 0x55f4f90040f0 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27d6fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f8ff97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f8ff8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27d6fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f8ff97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33958/0 (socket says 172.21.15.31:33958) 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27e6fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f90040f0 0x55f4f90064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f8ff8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44388/0 (socket says 172.21.15.31:44388) 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27d6fb700 1 -- 172.21.15.31:0/3025603688 learned_addr learned my addr 172.21.15.31:0/3025603688 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 -- 172.21.15.31:0/3025603688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f90040f0 msgr2=0x55f4f90064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f90040f0 0x55f4f90064d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 -- 172.21.15.31:0/3025603688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 msgr2=0x55f4f8ff97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f8ff97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 -- 172.21.15.31:0/3025603688 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f8dcd350 con 0x55f4f8e56430 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f8ff97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa27defc700 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f8ff8e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa2740042f0 tx=0x7fa274014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f919aaac23df11e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa263fff700 1 -- 172.21.15.31:0/3025603688 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274003d10 con 0x55f4f8e56430 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa263fff700 1 -- 172.21.15.31:0/3025603688 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa274003e70 con 0x55f4f8e56430 2023-12-14T18:32:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.496+0000 7fa263fff700 1 -- 172.21.15.31:0/3025603688 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27401e7f0 con 0x55f4f8e56430 2023-12-14T18:32:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/3025603688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 msgr2=0x55f4f8ff8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f8ff8e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa2740042f0 tx=0x7fa274014000 comp rx=0 tx=0).stop 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/3025603688 shutdown_connections 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f90040f0 0x55f4f90064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f8ff8e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/3025603688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f8ff97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/3025603688 >> 172.21.15.31:0/3025603688 conn(0x55f4f8fef890 msgr2=0x55f4f8ff5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/3025603688 shutdown_connections 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.497+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/3025603688 wait complete. 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 -- start start 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 Processor -- start 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f9003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8ff9370 0x55f4f9000d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f9001300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8ee7f30 con 0x55f4f90040f0 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8ee7bd0 con 0x55f4f8e56430 2023-12-14T18:32:39.504 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.498+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8dcf640 con 0x55f4f8ff9370 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa28c2eeb80 10 client.0 _ll_register_callbacks cb 0x55f4f8f0b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f9001300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27defc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f9003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27d6fb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8ff9370 0x55f4f9000d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f9001300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33960/0 (socket says 172.21.15.31:33960) 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27defc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f9003a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44398/0 (socket says 172.21.15.31:44398) 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 -- 172.21.15.31:0/1620401894 learned_addr learned my addr 172.21.15.31:0/1620401894 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 -- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8ff9370 msgr2=0x55f4f9000d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8ff9370 0x55f4f9000d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 -- 172.21.15.31:0/1620401894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 msgr2=0x55f4f9003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f9003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 -- 172.21.15.31:0/1620401894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274017070 con 0x55f4f90040f0 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27defc700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f9003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8ff9370 0x55f4f9000d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f9001300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa270011f60 tx=0x7fa270014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8070978abfbb79 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa2617fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa2617fa700 1 -- 172.21.15.31:0/1620401894 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000e040 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa2617fa700 1 -- 172.21.15.31:0/1620401894 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa270014ba0 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.499+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4f8fecea0 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa2617fa700 1 -- 172.21.15.31:0/1620401894 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa270014d60 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 10 client.16665 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4f8f12110 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa2617fa700 1 -- 172.21.15.31:0/1620401894 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4f8fecea0 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa2617fa700 1 -- 172.21.15.31:0/1620401894 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa27001c020 con 0x55f4f90040f0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 10 client.16665 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 20 client.16665 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f9013c70 0x55f4f9016080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 4 client.16665 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4f8f44c80 con 0x55f4f9013c70 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.500+0000 7fa27defc700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f9013c70 0x55f4f9016080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.501+0000 7fa2617fa700 10 client.16665 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.501+0000 7fa27defc700 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f9013c70 0x55f4f9016080 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b249047ce68535f server_cookie=246ee7cb48ac8a59 in_seq=0 out_seq=0 2023-12-14T18:32:39.507 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.502+0000 7fa2617fa700 1 -- 172.21.15.31:0/1620401894 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4f8f44c80 con 0x55f4f9013c70 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.502+0000 7fa2617fa700 10 client.16665 handle_command_reply: tid=0 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.502+0000 7fa28c2eeb80 1 client.16665 shutdown 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 msgr2=0x55f4f9001300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f9001300 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa270011f60 tx=0x7fa270014600 comp rx=0 tx=0).stop 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 shutdown_connections 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f9013c70 0x55f4f9016080 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f9001300 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8e56430 0x55f4f9003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1620401894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8ff9370 0x55f4f9000d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa2617fa700 0 client.16665 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 >> 172.21.15.31:0/1620401894 conn(0x55f4f8fef890 msgr2=0x55f4f8ff4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 shutdown_connections 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1620401894 wait complete. 2023-12-14T18:32:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.503+0000 7fa28c2eeb80 20 client.16665 trim_cache size 0 max 16384 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 Processor -- start 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 -- start start 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f8ff5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8ff9370 0x55f4f9003550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f8ffdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f9011250 con 0x55f4f90040f0 2023-12-14T18:32:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f8ffa930 con 0x55f4f8ff9370 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f9010510 con 0x55f4f8e56430 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa27e6fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f8ffdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa27e6fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f8ffdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33968/0 (socket says 172.21.15.31:33968) 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa27e6fd700 1 -- 172.21.15.31:0/4058000071 learned_addr learned my addr 172.21.15.31:0/4058000071 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa27defc700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f8ff5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.519+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8ff9370 0x55f4f9003550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27e6fd700 1 -- 172.21.15.31:0/4058000071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 msgr2=0x55f4f8ff5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f8ff5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27e6fd700 1 -- 172.21.15.31:0/4058000071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8ff9370 msgr2=0x55f4f9003550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8ff9370 0x55f4f9003550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27e6fd700 1 -- 172.21.15.31:0/4058000071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f8f11e60 con 0x55f4f90040f0 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8ff9370 0x55f4f9003550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27defc700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f8ff5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f8ffdfb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b010 tx=0x7fa2700048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4e879a68190af0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa26cff9700 1 -- 172.21.15.31:0/4058000071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000e070 con 0x55f4f90040f0 2023-12-14T18:32:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa26cff9700 1 -- 172.21.15.31:0/4058000071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa27000f040 con 0x55f4f90040f0 2023-12-14T18:32:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.520+0000 7fa26cff9700 1 -- 172.21.15.31:0/4058000071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270007ea0 con 0x55f4f90040f0 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/4058000071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 msgr2=0x55f4f8ffdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f8ffdfb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b010 tx=0x7fa2700048e0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/4058000071 shutdown_connections 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f90040f0 0x55f4f8ffdfb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f8ff5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/4058000071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f8ff9370 0x55f4f9003550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/4058000071 >> 172.21.15.31:0/4058000071 conn(0x55f4f8fef890 msgr2=0x55f4f8ff4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/4058000071 shutdown_connections 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/4058000071 wait complete. 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 -- start start 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.521+0000 7fa28c2eeb80 1 Processor -- start 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f9064f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f9065450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f90040f0 0x55f4f905f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f905f7e0 con 0x55f4f8ff9370 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f905f950 con 0x55f4f90040f0 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f905fb10 con 0x55f4f8e56430 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa28c2eeb80 10 client.0 _ll_register_callbacks cb 0x7fa2700118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa27defc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f9064f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa27d6fb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f9065450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa27defc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f9064f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45338/0 (socket says 172.21.15.31:45338) 2023-12-14T18:32:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa27defc700 1 -- 172.21.15.31:0/1774219768 learned_addr learned my addr 172.21.15.31:0/1774219768 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.522+0000 7fa27e6fd700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f90040f0 0x55f4f905f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27defc700 1 -- 172.21.15.31:0/1774219768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f90040f0 msgr2=0x55f4f905f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27defc700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f90040f0 0x55f4f905f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27defc700 1 -- 172.21.15.31:0/1774219768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 msgr2=0x55f4f9065450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27defc700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f9065450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27defc700 1 -- 172.21.15.31:0/1774219768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4f8dcd350 con 0x55f4f8e56430 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f9065450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa27defc700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f9064f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa274009720 tx=0x7fa274011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826d76cfb772d7cf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa2617fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa2617fa700 1 -- 172.21.15.31:0/1774219768 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27402b040 con 0x55f4f8e56430 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa2617fa700 1 -- 172.21.15.31:0/1774219768 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fa274033a40 con 0x55f4f8e56430 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa274003e70 con 0x55f4f8e56430 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa2617fa700 1 -- 172.21.15.31:0/1774219768 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa274033600 con 0x55f4f8e56430 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa28c2eeb80 10 client.26512 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa278110320 con 0x55f4f8e56430 2023-12-14T18:32:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.523+0000 7fa2617fa700 1 -- 172.21.15.31:0/1774219768 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa274003e70 con 0x55f4f8e56430 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa2617fa700 1 -- 172.21.15.31:0/1774219768 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa274024020 con 0x55f4f8e56430 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa28c2eeb80 10 client.26512 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa28c2eeb80 10 client.26512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa28c2eeb80 20 client.26512 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f906cd40 0x55f4f906f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa28c2eeb80 4 client.26512 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4f8f449a0 con 0x55f4f906cd40 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f906cd40 0x55f4f906f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa27d6fb700 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f906cd40 0x55f4f906f150 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d914e8ee759f4b server_cookie=69799b69b7af9d6e in_seq=0 out_seq=0 2023-12-14T18:32:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:39.524+0000 7fa2617fa700 10 client.26512 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:39.715 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:39 smithi031 ceph-mon[42085]: pgmap v148: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 656 op/s 2023-12-14T18:32:39.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:39 smithi176 ceph-mon[43506]: pgmap v148: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 656 op/s 2023-12-14T18:32:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:39 smithi204 ceph-mon[43866]: pgmap v148: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.6 MiB/s wr, 656 op/s 2023-12-14T18:32:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.632+0000 7fa2617fa700 1 -- 172.21.15.31:0/1774219768 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4f8f449a0 con 0x55f4f906cd40 2023-12-14T18:32:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.632+0000 7fa2617fa700 10 client.26512 handle_command_reply: tid=0 2023-12-14T18:32:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.632+0000 7fa28c2eeb80 1 client.26512 shutdown 2023-12-14T18:32:40.636 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 msgr2=0x55f4f9064f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f9064f10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa274009720 tx=0x7fa274011ca0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 shutdown_connections 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f4f906cd40 0x55f4f906f150 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f4f90040f0 0x55f4f905f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4f8e56430 0x55f4f9064f10 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa274009720 tx=0x7fa274011ca0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 --2- 172.21.15.31:0/1774219768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f4f8ff9370 0x55f4f9065450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.633+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 >> 172.21.15.31:0/1774219768 conn(0x55f4f8fef890 msgr2=0x55f4f9002390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.635+0000 7fa2617fa700 0 client.26512 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.642+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 shutdown_connections 2023-12-14T18:32:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.642+0000 7fa28c2eeb80 1 -- 172.21.15.31:0/1774219768 wait complete. 2023-12-14T18:32:40.644 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.642+0000 7fa28c2eeb80 20 client.26512 trim_cache size 0 max 16384 2023-12-14T18:32:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:32:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:32:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:32:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:32:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.648+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa2640a1340 msgr2=0x7fa2640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.648+0000 7fa280160700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa2640a1340 0x7fa2640a37f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa274005fd0 tx=0x7fa274009310 comp rx=0 tx=0).stop 2023-12-14T18:32:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.648+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 msgr2=0x7fa278196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.648+0000 7fa280160700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278196a00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa268009230 tx=0x7fa26800fa20 comp rx=0 tx=0).stop 2023-12-14T18:32:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.651+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 shutdown_connections 2023-12-14T18:32:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.651+0000 7fa280160700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa2640a1340 0x7fa2640a37f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.651+0000 7fa280160700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa278105580 0x7fa27819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.651+0000 7fa280160700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa2780fe9e0 0x7fa2781964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.652+0000 7fa280160700 1 --2- 172.21.15.31:0/3788413127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2780ff380 0x7fa278196a00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.652+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 >> 172.21.15.31:0/3788413127 conn(0x7fa278075810 msgr2=0x7fa2780fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.653+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 shutdown_connections 2023-12-14T18:32:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:40.653+0000 7fa280160700 1 -- 172.21.15.31:0/3788413127 wait complete. 2023-12-14T18:32:41.718 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:41 smithi031 ceph-mon[42085]: pgmap v149: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-12-14T18:32:41.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:41 smithi176 ceph-mon[43506]: pgmap v149: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-12-14T18:32:41.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:41 smithi204 ceph-mon[43866]: pgmap v149: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-12-14T18:32:43.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:43 smithi031 ceph-mon[42085]: pgmap v150: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-12-14T18:32:43.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:43 smithi176 ceph-mon[43506]: pgmap v150: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-12-14T18:32:43.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:43 smithi204 ceph-mon[43866]: pgmap v150: 129 pgs: 129 active+clean; 442 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2023-12-14T18:32:45.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:45 smithi031 ceph-mon[42085]: pgmap v151: 129 pgs: 129 active+clean; 448 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 770 op/s 2023-12-14T18:32:45.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:45 smithi176 ceph-mon[43506]: pgmap v151: 129 pgs: 129 active+clean; 448 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 770 op/s 2023-12-14T18:32:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:45 smithi204 ceph-mon[43866]: pgmap v151: 129 pgs: 129 active+clean; 448 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 770 op/s 2023-12-14T18:32:47.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:47 smithi176 ceph-mon[43506]: pgmap v152: 129 pgs: 129 active+clean; 448 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 560 op/s 2023-12-14T18:32:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:47 smithi204 ceph-mon[43866]: pgmap v152: 129 pgs: 129 active+clean; 448 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 560 op/s 2023-12-14T18:32:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:47 smithi031 ceph-mon[42085]: pgmap v152: 129 pgs: 129 active+clean; 448 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 560 op/s 2023-12-14T18:32:49.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:49 smithi176 ceph-mon[43506]: pgmap v153: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 745 op/s 2023-12-14T18:32:49.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:49 smithi204 ceph-mon[43866]: pgmap v153: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 745 op/s 2023-12-14T18:32:49.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:49 smithi031 ceph-mon[42085]: pgmap v153: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 745 op/s 2023-12-14T18:32:51.744 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:51 smithi176 ceph-mon[43506]: pgmap v154: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2023-12-14T18:32:51.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:51 smithi204 ceph-mon[43866]: pgmap v154: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2023-12-14T18:32:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:51 smithi031 ceph-mon[42085]: pgmap v154: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2023-12-14T18:32:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.808+0000 7f16e9c96700 1 -- 172.21.15.31:0/1610326947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e4104400 msgr2=0x7f16e410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.808+0000 7f16e9c96700 1 --2- 172.21.15.31:0/1610326947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e4104400 0x7f16e410bfd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b0d0 tx=0x7f16dc00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 -- 172.21.15.31:0/1610326947 shutdown_connections 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 --2- 172.21.15.31:0/1610326947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e4104400 0x7f16e410bfd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 --2- 172.21.15.31:0/1610326947 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e40ff3c0 0x7f16e40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 --2- 172.21.15.31:0/1610326947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ffd60 0x7f16e4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 -- 172.21.15.31:0/1610326947 >> 172.21.15.31:0/1610326947 conn(0x7f16e40763a0 msgr2=0x7f16e40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 -- 172.21.15.31:0/1610326947 shutdown_connections 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 -- 172.21.15.31:0/1610326947 wait complete. 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.809+0000 7f16e9c96700 1 Processor -- start 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 -- start start 2023-12-14T18:32:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 0x7f16e41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e40ffd60 0x7f16e4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e4104400 0x7f16e41a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41a2890 con 0x7f16e40ffd60 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41a29d0 con 0x7f16e4104400 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e9c96700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41a2b10 con 0x7f16e40ff3c0 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 0x7f16e41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e40ffd60 0x7f16e4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 0x7f16e41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47068/0 (socket says 172.21.15.31:47068) 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e37fe700 1 -- 172.21.15.31:0/2873423187 learned_addr learned my addr 172.21.15.31:0/2873423187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e3fff700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e4104400 0x7f16e41a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e3fff700 1 -- 172.21.15.31:0/2873423187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 msgr2=0x7f16e41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e3fff700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 0x7f16e41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e3fff700 1 -- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e40ffd60 msgr2=0x7f16e4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e3fff700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e40ffd60 0x7f16e4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e3fff700 1 -- 172.21.15.31:0/2873423187 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc009d00 con 0x7f16e4104400 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.810+0000 7f16e37fe700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 0x7f16e41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.811+0000 7f16e3fff700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e4104400 0x7f16e41a2350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009fd0 tx=0x7f16dc0077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.811+0000 7f16e0ff9700 1 -- 172.21.15.31:0/2873423187 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc00e040 con 0x7f16e4104400 2023-12-14T18:32:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.811+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e410f220 con 0x7f16e4104400 2023-12-14T18:32:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.811+0000 7f16e0ff9700 1 -- 172.21.15.31:0/2873423187 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16dc00bcb0 con 0x7f16e4104400 2023-12-14T18:32:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.811+0000 7f16e0ff9700 1 -- 172.21.15.31:0/2873423187 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc005840 con 0x7f16e4104400 2023-12-14T18:32:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.811+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e410f090 con 0x7f16e4104400 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.814+0000 7f16e0ff9700 1 -- 172.21.15.31:0/2873423187 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f16dc004a50 con 0x7f16e4104400 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.814+0000 7f16e0ff9700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f16cc0a12c0 0x7f16cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.815+0000 7f16e0ff9700 1 -- 172.21.15.31:0/2873423187 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f16dc0cde10 con 0x7f16e4104400 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.815+0000 7f16e37fe700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f16cc0a12c0 0x7f16cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.815+0000 7f16e37fe700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f16cc0a12c0 0x7f16cc0a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16d4011550 tx=0x7f16d4009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.815+0000 7f16f5e24b80 1 Processor -- start 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.815+0000 7f16f5e24b80 1 -- start start 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e060de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 0x5560e060e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e061d500 0x5560e061f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e04f8580 con 0x5560e060e370 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e0567810 con 0x5560e046b430 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e0560ca0 con 0x5560e061d500 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 0x5560e060e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e060de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e061d500 0x5560e061f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e060de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35676/0 (socket says 172.21.15.31:35676) 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 0x5560e060e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53556/0 (socket says 172.21.15.31:53556) 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 -- 172.21.15.31:0/474782693 learned_addr learned my addr 172.21.15.31:0/474782693 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 -- 172.21.15.31:0/474782693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e061d500 msgr2=0x5560e061f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e061d500 0x5560e061f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 -- 172.21.15.31:0/474782693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 msgr2=0x5560e060e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 0x5560e060e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 -- 172.21.15.31:0/474782693 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560e03e2350 con 0x5560e046b430 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e2ffd700 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 0x5560e060e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.816+0000 7f16e37fe700 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e060de30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009880 tx=0x7f16d4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbfd9bfb1e239739 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16c9ffb700 1 -- 172.21.15.31:0/474782693 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4026630 con 0x5560e046b430 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16c9ffb700 1 -- 172.21.15.31:0/474782693 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d4021070 con 0x5560e046b430 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16c9ffb700 1 -- 172.21.15.31:0/474782693 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d401d650 con 0x5560e046b430 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 -- 172.21.15.31:0/474782693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 msgr2=0x5560e060de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e060de30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009880 tx=0x7f16d4014000 comp rx=0 tx=0).stop 2023-12-14T18:32:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 -- 172.21.15.31:0/474782693 shutdown_connections 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e061d500 0x5560e061f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e060de30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/474782693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e060e370 0x5560e060e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.817+0000 7f16f5e24b80 1 -- 172.21.15.31:0/474782693 >> 172.21.15.31:0/474782693 conn(0x5560e0604890 msgr2=0x5560e060a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 -- 172.21.15.31:0/474782693 shutdown_connections 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 -- 172.21.15.31:0/474782693 wait complete. 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 -- start start 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 Processor -- start 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 0x5560e061cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e061d500 0x5560e061a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.818+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e04fcf30 con 0x5560e046b430 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e04fcbd0 con 0x5560e061d500 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e03e4640 con 0x5560e060e370 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16f5e24b80 10 client.0 _ll_register_callbacks cb 0x5560e05208b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 0x5560e061cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 0x5560e061cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53570/0 (socket says 172.21.15.31:53570) 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 -- 172.21.15.31:0/229821619 learned_addr learned my addr 172.21.15.31:0/229821619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e2ffd700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e3fff700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e061d500 0x5560e061a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 -- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 msgr2=0x5560e061a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 -- 172.21.15.31:0/229821619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e061d500 msgr2=0x5560e061a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e061d500 0x5560e061a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 -- 172.21.15.31:0/229821619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d4017070 con 0x5560e046b430 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16e37fe700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 0x5560e061cf50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f16d4011ca0 tx=0x7f16d400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981ffdec5831d373 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.819+0000 7f16c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16c2ffd700 1 -- 172.21.15.31:0/229821619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d401c620 con 0x5560e046b430 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560e0601ea0 con 0x5560e046b430 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16c2ffd700 1 -- 172.21.15.31:0/229821619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f16d401c780 con 0x5560e046b430 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16c2ffd700 1 -- 172.21.15.31:0/229821619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d4005040 con 0x5560e046b430 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 10 client.16683 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5560e0527110 con 0x5560e046b430 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16c2ffd700 1 -- 172.21.15.31:0/229821619 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5560e0601ea0 con 0x5560e046b430 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16c2ffd700 1 -- 172.21.15.31:0/229821619 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f16d4004af0 con 0x5560e046b430 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 10 client.16683 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 20 client.16683 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e062cf00 0x5560e062f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 4 client.16683 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.820+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560e0559c80 con 0x5560e062cf00 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.821+0000 7f16e2ffd700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e062cf00 0x5560e062f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.821+0000 7f16e2ffd700 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e062cf00 0x5560e062f310 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86394bbc1e7fcac server_cookie=4667dd6580523ea8 in_seq=0 out_seq=0 2023-12-14T18:32:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.821+0000 7f16c2ffd700 10 client.16683 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16c2ffd700 1 -- 172.21.15.31:0/229821619 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5560e0559c80 con 0x5560e062cf00 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16c2ffd700 10 client.16683 handle_command_reply: tid=0 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 client.16683 shutdown 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 msgr2=0x5560e061cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 0x5560e061cf50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f16d4011ca0 tx=0x7f16d400ee40 comp rx=0 tx=0).stop 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 shutdown_connections 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e062cf00 0x5560e062f310 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e061d500 0x5560e061a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e046b430 0x5560e061cf50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16c2ffd700 0 client.16683 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/229821619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 >> 172.21.15.31:0/229821619 conn(0x5560e0604890 msgr2=0x5560e0609d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 shutdown_connections 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 1 -- 172.21.15.31:0/229821619 wait complete. 2023-12-14T18:32:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.828+0000 7f16f5e24b80 20 client.16683 trim_cache size 0 max 16384 2023-12-14T18:32:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:53 smithi204 ceph-mon[43866]: pgmap v155: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 Processor -- start 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 -- start start 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e061ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0619aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e061af40 con 0x5560e061d500 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e0601a70 con 0x5560e046b430 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e060ebe0 con 0x5560e060e370 2023-12-14T18:32:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0619aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.844+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e061ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e2ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e061ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35696/0 (socket says 172.21.15.31:35696) 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 -- 172.21.15.31:0/4114204897 learned_addr learned my addr 172.21.15.31:0/4114204897 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 -- 172.21.15.31:0/4114204897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 msgr2=0x5560e061cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 -- 172.21.15.31:0/4114204897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 msgr2=0x5560e0619aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0619aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 -- 172.21.15.31:0/4114204897 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560e0526e60 con 0x5560e046b430 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16e37fe700 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e061ca10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16d4016040 tx=0x7f16d4005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2be88fbcb35896 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16ca7fc700 1 -- 172.21.15.31:0/4114204897 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d401cce0 con 0x5560e046b430 2023-12-14T18:32:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16ca7fc700 1 -- 172.21.15.31:0/4114204897 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d4026520 con 0x5560e046b430 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.845+0000 7f16ca7fc700 1 -- 172.21.15.31:0/4114204897 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4005040 con 0x5560e046b430 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 -- 172.21.15.31:0/4114204897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 msgr2=0x5560e061ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e061ca10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16d4016040 tx=0x7f16d4005ce0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 -- 172.21.15.31:0/4114204897 shutdown_connections 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0619aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e046b430 0x5560e061ca10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/4114204897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e060e370 0x5560e061cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 -- 172.21.15.31:0/4114204897 >> 172.21.15.31:0/4114204897 conn(0x5560e0604890 msgr2=0x5560e0628870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 -- 172.21.15.31:0/4114204897 shutdown_connections 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 -- 172.21.15.31:0/4114204897 wait complete. 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.846+0000 7f16f5e24b80 1 -- start start 2023-12-14T18:32:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 Processor -- start 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e046b430 0x5560e0622790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e060e370 0x5560e0622cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0623210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e0680d90 con 0x5560e061d500 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e0680ed0 con 0x5560e060e370 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560e0681010 con 0x5560e046b430 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16f5e24b80 10 client.0 _ll_register_callbacks cb 0x7f16d8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0623210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0623210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53580/0 (socket says 172.21.15.31:53580) 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16e3fff700 1 -- 172.21.15.31:0/3236800153 learned_addr learned my addr 172.21.15.31:0/3236800153 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16e37fe700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e046b430 0x5560e0622790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.847+0000 7f16e2ffd700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e060e370 0x5560e0622cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16e3fff700 1 -- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e046b430 msgr2=0x5560e0622790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16e3fff700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e046b430 0x5560e0622790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16e3fff700 1 -- 172.21.15.31:0/3236800153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e060e370 msgr2=0x5560e0622cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16e3fff700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e060e370 0x5560e0622cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16e3fff700 1 -- 172.21.15.31:0/3236800153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560e0527110 con 0x5560e061d500 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16e3fff700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0623210 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0970e0 tx=0x7f16dc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e628eafcf817225b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16c2ffd700 1 -- 172.21.15.31:0/3236800153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0b3320 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16c2ffd700 1 -- 172.21.15.31:0/3236800153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f16dc00be20 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16d4021070 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16c2ffd700 1 -- 172.21.15.31:0/3236800153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16dc09a540 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16f5e24b80 10 client.16689 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5560e050d940 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.848+0000 7f16c2ffd700 1 -- 172.21.15.31:0/3236800153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16d4021070 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16c2ffd700 1 -- 172.21.15.31:0/3236800153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f16dc09a0d0 con 0x5560e061d500 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16f5e24b80 10 client.16689 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16f5e24b80 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16f5e24b80 20 client.16689 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e06823e0 0x5560e06847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16f5e24b80 4 client.16689 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5560e05599a0 con 0x5560e06823e0 2023-12-14T18:32:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16e37fe700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e06823e0 0x5560e06847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16e37fe700 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e06823e0 0x5560e06847f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fcde1e4ec329fd server_cookie=80fba4568dd8145a in_seq=0 out_seq=0 2023-12-14T18:32:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:53.849+0000 7f16c2ffd700 10 client.16689 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:53.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:53 smithi031 ceph-mon[42085]: pgmap v155: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2023-12-14T18:32:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:53 smithi176 ceph-mon[43506]: pgmap v155: 129 pgs: 129 active+clean; 466 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 515 op/s 2023-12-14T18:32:55.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:55 smithi204 ceph-mon[43866]: pgmap v156: 129 pgs: 129 active+clean; 480 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 726 op/s 2023-12-14T18:32:55.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:55 smithi031 ceph-mon[42085]: pgmap v156: 129 pgs: 129 active+clean; 480 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 726 op/s 2023-12-14T18:32:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:55 smithi176 ceph-mon[43506]: pgmap v156: 129 pgs: 129 active+clean; 480 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 726 op/s 2023-12-14T18:32:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.526+0000 7f16c2ffd700 1 -- 172.21.15.31:0/3236800153 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5560e05599a0 con 0x5560e06823e0 2023-12-14T18:32:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.526+0000 7f16c2ffd700 10 client.16689 handle_command_reply: tid=0 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.526+0000 7f16f5e24b80 1 client.16689 shutdown 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 msgr2=0x5560e0623210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0623210 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f16dc0970e0 tx=0x7f16dc005a20 comp rx=0 tx=0).stop 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 shutdown_connections 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5560e06823e0 0x5560e06847f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5560e061d500 0x5560e0623210 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5560e046b430 0x5560e0622790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 --2- 172.21.15.31:0/3236800153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5560e060e370 0x5560e0622cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 >> 172.21.15.31:0/3236800153 conn(0x5560e0604890 msgr2=0x5560e0619aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.527+0000 7f16c2ffd700 0 client.16689 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.528+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 shutdown_connections 2023-12-14T18:32:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.528+0000 7f16f5e24b80 1 -- 172.21.15.31:0/3236800153 wait complete. 2023-12-14T18:32:56.539 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.536+0000 7f16f5e24b80 20 client.16689 trim_cache size 0 max 16384 2023-12-14T18:32:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:32:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:32:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:32:56.541 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:32:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.541+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f16cc0a12c0 msgr2=0x7f16cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.541+0000 7f16e9c96700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f16cc0a12c0 0x7f16cc0a3770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f16d4011550 tx=0x7f16d4009260 comp rx=0 tx=0).stop 2023-12-14T18:32:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.541+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e4104400 msgr2=0x7f16e41a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.541+0000 7f16e9c96700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e4104400 0x7f16e41a2350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009fd0 tx=0x7f16dc0077f0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 shutdown_connections 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f16e4104400 0x7f16e41a2350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f16cc0a12c0 0x7f16cc0a3770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16e40ff3c0 0x7f16e41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 --2- 172.21.15.31:0/2873423187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16e40ffd60 0x7f16e4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 >> 172.21.15.31:0/2873423187 conn(0x7f16e40763a0 msgr2=0x7f16e40fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 shutdown_connections 2023-12-14T18:32:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:32:56.545+0000 7f16e9c96700 1 -- 172.21.15.31:0/2873423187 wait complete. 2023-12-14T18:32:56.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:32:56.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.997+0000 7f0affe4f700 1 -- 172.21.15.31:0/3123518313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 msgr2=0x7f0af8108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.997+0000 7f0affe4f700 1 --2- 172.21.15.31:0/3123518313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 0x7f0af8108330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009990 tx=0x7f0ae8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.998+0000 7f0affe4f700 1 -- 172.21.15.31:0/3123518313 shutdown_connections 2023-12-14T18:32:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.998+0000 7f0affe4f700 1 --2- 172.21.15.31:0/3123518313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8119a40 0x7f0af811be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.998+0000 7f0affe4f700 1 --2- 172.21.15.31:0/3123518313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af8107af0 0x7f0af810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.998+0000 7f0affe4f700 1 --2- 172.21.15.31:0/3123518313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 0x7f0af8108330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:56.998+0000 7f0affe4f700 1 -- 172.21.15.31:0/3123518313 >> 172.21.15.31:0/3123518313 conn(0x7f0af806b7e0 msgr2=0x7f0af8071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:57.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.004+0000 7f0affe4f700 1 -- 172.21.15.31:0/3123518313 shutdown_connections 2023-12-14T18:32:57.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.008+0000 7f0affe4f700 1 -- 172.21.15.31:0/3123518313 wait complete. 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.008+0000 7f0affe4f700 1 Processor -- start 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 -- start start 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 0x7f0af81a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 0x7f0af81a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af8119a40 0x7f0af81a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af81aa2b0 con 0x7f0af8119a40 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af81aa420 con 0x7f0af8107af0 2023-12-14T18:32:57.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.009+0000 7f0affe4f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af81aa5e0 con 0x7f0af8107ec0 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.010+0000 7f0afdbeb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 0x7f0af81a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.010+0000 7f0afdbeb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 0x7f0af81a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:54612/0 (socket says 172.21.15.31:54612) 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.010+0000 7f0afdbeb700 1 -- 172.21.15.31:0/2693009693 learned_addr learned my addr 172.21.15.31:0/2693009693 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0afdbeb700 1 -- 172.21.15.31:0/2693009693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 msgr2=0x7f0af81a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 0x7f0af81a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0afdbeb700 1 -- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af8119a40 msgr2=0x7f0af81a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af8119a40 0x7f0af81a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0afdbeb700 1 -- 172.21.15.31:0/2693009693 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80095f0 con 0x7f0af8107af0 2023-12-14T18:32:57.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 0x7f0af81a52d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0af400b550 tx=0x7f0af400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0aeeffd700 1 -- 172.21.15.31:0/2693009693 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4011830 con 0x7f0af8107af0 2023-12-14T18:32:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0aeeffd700 1 -- 172.21.15.31:0/2693009693 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af4011e50 con 0x7f0af8107af0 2023-12-14T18:32:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.011+0000 7f0aeeffd700 1 -- 172.21.15.31:0/2693009693 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400f2e0 con 0x7f0af8107af0 2023-12-14T18:32:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.013+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af811f200 con 0x7f0af8107af0 2023-12-14T18:32:57.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.013+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af811f070 con 0x7f0af8107af0 2023-12-14T18:32:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.016+0000 7f0aeeffd700 1 -- 172.21.15.31:0/2693009693 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0af4011990 con 0x7f0af8107af0 2023-12-14T18:32:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.017+0000 7f0aeeffd700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0ae40b24e0 0x7f0ae40b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.017+0000 7f0aeeffd700 1 -- 172.21.15.31:0/2693009693 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0af40cca60 con 0x7f0af8107af0 2023-12-14T18:32:57.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.017+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0ae40b24e0 0x7f0ae40b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.018+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0ae40b24e0 0x7f0ae40b4990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098e0 tx=0x7f0ae8005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:57.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.022+0000 7f0b0bfddb80 1 Processor -- start 2023-12-14T18:32:57.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 -- start start 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fa1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509faef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509e8b5e0 con 0x565509fa1380 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509efa870 con 0x565509fa1c90 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.029+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509ef3d00 con 0x565509facb50 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.030+0000 7f0afd3ea700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.030+0000 7f0afd3ea700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:54620/0 (socket says 172.21.15.31:54620) 2023-12-14T18:32:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.030+0000 7f0afd3ea700 1 -- 172.21.15.31:0/1711472334 learned_addr learned my addr 172.21.15.31:0/1711472334 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:57.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.031+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fa1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.032+0000 7f0afd3ea700 1 -- 172.21.15.31:0/1711472334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 msgr2=0x565509faef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.032+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509faef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.032+0000 7f0afd3ea700 1 -- 172.21.15.31:0/1711472334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 msgr2=0x565509fa1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.032+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fa1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.032+0000 7f0afd3ea700 1 -- 172.21.15.31:0/1711472334 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565509d753b0 con 0x565509fa1c90 2023-12-14T18:32:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.033+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fac610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8003780 tx=0x7f0ae801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc991386d94088ab server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.034+0000 7f0ae3fff700 1 -- 172.21.15.31:0/1711472334 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8029070 con 0x565509fa1c90 2023-12-14T18:32:57.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.035+0000 7f0ae3fff700 1 -- 172.21.15.31:0/1711472334 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae8005a00 con 0x565509fa1c90 2023-12-14T18:32:57.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.035+0000 7f0ae3fff700 1 -- 172.21.15.31:0/1711472334 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae801d070 con 0x565509fa1c90 2023-12-14T18:32:57.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.038+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/1711472334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 msgr2=0x565509fac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.038+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fac610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8003780 tx=0x7f0ae801d000 comp rx=0 tx=0).stop 2023-12-14T18:32:57.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.041+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/1711472334 shutdown_connections 2023-12-14T18:32:57.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.041+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509faef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.041+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fa1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.041+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/1711472334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fac610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.041+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/1711472334 >> 172.21.15.31:0/1711472334 conn(0x565509f97ca0 msgr2=0x565509f9d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:57.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.046+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/1711472334 shutdown_connections 2023-12-14T18:32:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.046+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/1711472334 wait complete. 2023-12-14T18:32:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.046+0000 7f0b0bfddb80 1 -- start start 2023-12-14T18:32:57.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 Processor -- start 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fa9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509fa9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509e8ff90 con 0x565509fa1380 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509e8fc30 con 0x565509fa1c90 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.047+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509d776a0 con 0x565509facb50 2023-12-14T18:32:57.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.048+0000 7f0b0bfddb80 10 client.0 _ll_register_callbacks cb 0x565509eb3910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.048+0000 7f0afe3ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509fa9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.048+0000 7f0afe3ec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509fa9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:43102/0 (socket says 172.21.15.31:43102) 2023-12-14T18:32:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.048+0000 7f0afe3ec700 1 -- 172.21.15.31:0/3300867769 learned_addr learned my addr 172.21.15.31:0/3300867769 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.048+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fa9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.049+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.050+0000 7f0afd3ea700 1 -- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 msgr2=0x565509fa9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.050+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509fa9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.050+0000 7f0afd3ea700 1 -- 172.21.15.31:0/3300867769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 msgr2=0x565509fac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.050+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.050+0000 7f0afd3ea700 1 -- 172.21.15.31:0/3300867769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af811c6b0 con 0x565509fa1c90 2023-12-14T18:32:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.054+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fa9840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005e50 tx=0x7f0ae8005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c93866da0707b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.054+0000 7f0ae17fa700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:32:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.054+0000 7f0ae17fa700 1 -- 172.21.15.31:0/3300867769 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8029040 con 0x565509fa1c90 2023-12-14T18:32:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.054+0000 7f0ae17fa700 1 -- 172.21.15.31:0/3300867769 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0ae802e870 con 0x565509fa1c90 2023-12-14T18:32:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.054+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565509f952b0 con 0x565509fa1c90 2023-12-14T18:32:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.055+0000 7f0ae17fa700 1 -- 172.21.15.31:0/3300867769 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae800bb30 con 0x565509fa1c90 2023-12-14T18:32:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.055+0000 7f0b0bfddb80 10 client.26522 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.055+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x565509eba170 con 0x565509fa1c90 2023-12-14T18:32:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.055+0000 7f0ae17fa700 1 -- 172.21.15.31:0/3300867769 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565509f952b0 con 0x565509fa1c90 2023-12-14T18:32:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.055+0000 7f0ae17fa700 1 -- 172.21.15.31:0/3300867769 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0ae800fc10 con 0x565509fa1c90 2023-12-14T18:32:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.056+0000 7f0b0bfddb80 10 client.26522 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.056+0000 7f0b0bfddb80 10 client.26522 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.056+0000 7f0b0bfddb80 20 client.26522 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.056+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565509fbc010 0x565509fbe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.056+0000 7f0b0bfddb80 4 client.26522 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.056+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565509eecce0 con 0x565509fbc010 2023-12-14T18:32:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.057+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565509fbc010 0x565509fbe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.058+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565509fbc010 0x565509fbe3f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819bb7d8a6193e92 server_cookie=586997f7974e4ad in_seq=0 out_seq=0 2023-12-14T18:32:57.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.058+0000 7f0ae17fa700 10 client.26522 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.059+0000 7f0ae17fa700 1 -- 172.21.15.31:0/3300867769 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565509eecce0 con 0x565509fbc010 2023-12-14T18:32:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.059+0000 7f0ae17fa700 10 client.26522 handle_command_reply: tid=0 2023-12-14T18:32:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.059+0000 7f0b0bfddb80 1 client.26522 shutdown 2023-12-14T18:32:57.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 msgr2=0x565509fa9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fa9840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005e50 tx=0x7f0ae8005670 comp rx=0 tx=0).stop 2023-12-14T18:32:57.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 shutdown_connections 2023-12-14T18:32:57.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x565509fbc010 0x565509fbe3f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509facb50 0x565509fa9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3300867769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1c90 0x565509fa9840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 >> 172.21.15.31:0/3300867769 conn(0x565509f97ca0 msgr2=0x565509f9d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0ae17fa700 0 client.26522 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 shutdown_connections 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.067+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3300867769 wait complete. 2023-12-14T18:32:57.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.068+0000 7f0b0bfddb80 20 client.26522 trim_cache size 0 max 16384 2023-12-14T18:32:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.102+0000 7f0b0bfddb80 1 Processor -- start 2023-12-14T18:32:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.102+0000 7f0b0bfddb80 1 -- start start 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509facb50 0x565509fac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509faa560 con 0x565509fa1380 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509fa42c0 con 0x565509facb50 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509fa2580 con 0x565509fa1c90 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afd3ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afe3ec700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509facb50 0x565509fac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afd3ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:43108/0 (socket says 172.21.15.31:43108) 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afd3ea700 1 -- 172.21.15.31:0/3928194821 learned_addr learned my addr 172.21.15.31:0/3928194821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afe3ec700 1 -- 172.21.15.31:0/3928194821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 msgr2=0x565509fac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afe3ec700 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afe3ec700 1 -- 172.21.15.31:0/3928194821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 msgr2=0x565509fb3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afe3ec700 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fb3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afe3ec700 1 -- 172.21.15.31:0/3928194821 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565509eb9ec0 con 0x565509facb50 2023-12-14T18:32:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.103+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:32:57.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.104+0000 7f0afe3ec700 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509facb50 0x565509fac590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009c00 tx=0x7f0af0009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecee7248016ba5fb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.104+0000 7f0aecff9700 1 -- 172.21.15.31:0/3928194821 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0011070 con 0x565509facb50 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.104+0000 7f0aecff9700 1 -- 172.21.15.31:0/3928194821 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af000cb60 con 0x565509facb50 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.104+0000 7f0aecff9700 1 -- 172.21.15.31:0/3928194821 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0015660 con 0x565509facb50 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3928194821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509facb50 msgr2=0x565509fac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509facb50 0x565509fac590 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009c00 tx=0x7f0af0009f10 comp rx=0 tx=0).stop 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3928194821 shutdown_connections 2023-12-14T18:32:57.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509facb50 0x565509fac590 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509fa1380 0x565509fb3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/3928194821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.105+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3928194821 >> 172.21.15.31:0/3928194821 conn(0x565509f97ca0 msgr2=0x565509fb8840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3928194821 shutdown_connections 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/3928194821 wait complete. 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 -- start start 2023-12-14T18:32:57.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 Processor -- start 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1380 0x565509fb35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fb1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509facb50 0x565509fb1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509fb2640 con 0x565509facb50 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x565509fb27b0 con 0x565509fa1380 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.106+0000 7f0b0bfddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56550a00fe60 con 0x565509fa1c90 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0b0bfddb80 10 client.0 _ll_register_callbacks cb 0x7f0af4095b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:57.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afdbeb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1380 0x565509fb35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fb1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afdbeb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1380 0x565509fb35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:54636/0 (socket says 172.21.15.31:54636) 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fb1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:43120/0 (socket says 172.21.15.31:43120) 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afdbeb700 1 -- 172.21.15.31:0/2886528532 learned_addr learned my addr 172.21.15.31:0/2886528532 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afe3ec700 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509facb50 0x565509fb1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 -- 172.21.15.31:0/2886528532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1380 msgr2=0x565509fb35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1380 0x565509fb35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 -- 172.21.15.31:0/2886528532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509facb50 msgr2=0x565509fb1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509facb50 0x565509fb1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 -- 172.21.15.31:0/2886528532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af0009880 con 0x565509fa1c90 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0afd3ea700 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fb1a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005850 tx=0x7f0ae8003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9909f59ad591b854 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.107+0000 7f0ae17fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:32:57.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0ae17fa700 1 -- 172.21.15.31:0/2886528532 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8029040 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0ae17fa700 1 -- 172.21.15.31:0/2886528532 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0ae800bab0 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ae8005a00 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0ae17fa700 1 -- 172.21.15.31:0/2886528532 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae801d040 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 10 client.26542 fetch_fsmap learned FSMap version 28 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x565509ea09a0 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0ae17fa700 1 -- 172.21.15.31:0/2886528532 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ae8005a00 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0ae17fa700 1 -- 172.21.15.31:0/2886528532 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0ae802e7d0 con 0x565509fa1c90 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 10 client.26542 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 10 client.26542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 20 client.26542 populate_metadata read hostname 'smithi031' 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56550a010ef0 0x56550a0132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 4 client.26542 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:32:57.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.108+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565509eeca00 con 0x56550a010ef0 2023-12-14T18:32:57.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.109+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56550a010ef0 0x56550a0132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0afdbeb700 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56550a010ef0 0x56550a0132d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c1fceba2803a3e server_cookie=1ce6fe848f0400bf in_seq=0 out_seq=0 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0ae17fa700 10 client.26542 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0ae17fa700 1 -- 172.21.15.31:0/2886528532 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x565509eeca00 con 0x56550a010ef0 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0ae17fa700 10 client.26542 handle_command_reply: tid=0 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 client.26542 shutdown 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 msgr2=0x565509fb1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fb1a20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005850 tx=0x7f0ae8003f50 comp rx=0 tx=0).stop 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 shutdown_connections 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56550a010ef0 0x56550a0132d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x565509facb50 0x565509fb1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x565509fa1380 0x565509fb35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 --2- 172.21.15.31:0/2886528532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565509fa1c90 0x565509fb1a20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0ae17fa700 0 client.26542 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 >> 172.21.15.31:0/2886528532 conn(0x565509f97ca0 msgr2=0x565509fa4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 shutdown_connections 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.110+0000 7f0b0bfddb80 1 -- 172.21.15.31:0/2886528532 wait complete. 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.111+0000 7f0b0bfddb80 20 client.26542 trim_cache size 0 max 16384 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:32:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "e9c9a305-94ae-47d1-85f0-57f435df8e4b": { 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "e9c9a305-94ae-47d1-85f0-57f435df8e4b", 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:32:57.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.111+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0ae40b24e0 msgr2=0x7f0ae40b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.111+0000 7f0affe4f700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0ae40b24e0 0x7f0ae40b4990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098e0 tx=0x7f0ae8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.111+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 msgr2=0x7f0af81a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.111+0000 7f0affe4f700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 0x7f0af81a52d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0af400b550 tx=0x7f0af400b910 comp rx=0 tx=0).stop 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 shutdown_connections 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af8119a40 0x7f0af81a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0af8107af0 0x7f0af81a52d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0ae40b24e0 0x7f0ae40b4990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 --2- 172.21.15.31:0/2693009693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0af8107ec0 0x7f0af81a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 >> 172.21.15.31:0/2693009693 conn(0x7f0af806b7e0 msgr2=0x7f0af8071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 shutdown_connections 2023-12-14T18:32:57.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:32:57.112+0000 7f0affe4f700 1 -- 172.21.15.31:0/2693009693 wait complete. 2023-12-14T18:32:57.129 INFO:tasks.cephfs.filesystem:scrub status for tag:e9c9a305-94ae-47d1-85f0-57f435df8e4b - {'path': '/', 'tag': 'e9c9a305-94ae-47d1-85f0-57f435df8e4b', 'options': 'recursive,force'} 2023-12-14T18:32:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:57 smithi204 ceph-mon[43866]: pgmap v157: 129 pgs: 129 active+clean; 480 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 396 op/s 2023-12-14T18:32:57.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:57 smithi031 ceph-mon[42085]: pgmap v157: 129 pgs: 129 active+clean; 480 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 396 op/s 2023-12-14T18:32:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:57 smithi176 ceph-mon[43506]: pgmap v157: 129 pgs: 129 active+clean; 480 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 396 op/s 2023-12-14T18:32:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:32:59 smithi204 ceph-mon[43866]: pgmap v158: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 699 op/s 2023-12-14T18:32:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:32:59 smithi031 ceph-mon[42085]: pgmap v158: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 699 op/s 2023-12-14T18:32:59.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:32:59 smithi176 ceph-mon[43506]: pgmap v158: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.1 MiB/s wr, 699 op/s 2023-12-14T18:33:01.843 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:01 smithi204 ceph-mon[43866]: pgmap v159: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 514 op/s 2023-12-14T18:33:01.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:01 smithi031 ceph-mon[42085]: pgmap v159: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 514 op/s 2023-12-14T18:33:01.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:01 smithi176 ceph-mon[43506]: pgmap v159: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 514 op/s 2023-12-14T18:33:03.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:03 smithi204 ceph-mon[43866]: pgmap v160: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 514 op/s 2023-12-14T18:33:03.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:03 smithi031 ceph-mon[42085]: pgmap v160: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 514 op/s 2023-12-14T18:33:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:03 smithi176 ceph-mon[43506]: pgmap v160: 129 pgs: 129 active+clean; 480 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 514 op/s 2023-12-14T18:33:05.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:05 smithi204 ceph-mon[43866]: pgmap v161: 129 pgs: 129 active+clean; 499 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 702 op/s 2023-12-14T18:33:05.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:05 smithi031 ceph-mon[42085]: pgmap v161: 129 pgs: 129 active+clean; 499 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 702 op/s 2023-12-14T18:33:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:05 smithi176 ceph-mon[43506]: pgmap v161: 129 pgs: 129 active+clean; 499 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 702 op/s 2023-12-14T18:33:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:07 smithi204 ceph-mon[43866]: pgmap v162: 129 pgs: 129 active+clean; 499 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 491 op/s 2023-12-14T18:33:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:07 smithi031 ceph-mon[42085]: pgmap v162: 129 pgs: 129 active+clean; 499 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 491 op/s 2023-12-14T18:33:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:07 smithi176 ceph-mon[43506]: pgmap v162: 129 pgs: 129 active+clean; 499 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 491 op/s 2023-12-14T18:33:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.753+0000 7f9dd7684700 1 -- 172.21.15.31:0/2883592974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107ec0 msgr2=0x7f9dd0108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.753+0000 7f9dd7684700 1 --2- 172.21.15.31:0/2883592974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107ec0 0x7f9dd0108330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009f90 tx=0x7f9dc800b330 comp rx=0 tx=0).stop 2023-12-14T18:33:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.754+0000 7f9dd7684700 1 -- 172.21.15.31:0/2883592974 shutdown_connections 2023-12-14T18:33:09.755 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.754+0000 7f9dd7684700 1 --2- 172.21.15.31:0/2883592974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 0x7f9dd011be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.754+0000 7f9dd7684700 1 --2- 172.21.15.31:0/2883592974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0107af0 0x7f9dd010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.754+0000 7f9dd7684700 1 --2- 172.21.15.31:0/2883592974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107ec0 0x7f9dd0108330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.756 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.754+0000 7f9dd7684700 1 -- 172.21.15.31:0/2883592974 >> 172.21.15.31:0/2883592974 conn(0x7f9dd006c510 msgr2=0x7f9dd00716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.756+0000 7f9dd7684700 1 -- 172.21.15.31:0/2883592974 shutdown_connections 2023-12-14T18:33:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.757+0000 7f9dd7684700 1 -- 172.21.15.31:0/2883592974 wait complete. 2023-12-14T18:33:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.757+0000 7f9dd7684700 1 Processor -- start 2023-12-14T18:33:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 -- start start 2023-12-14T18:33:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107af0 0x7f9dd01a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0107ec0 0x7f9dd01a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 0x7f9dd01a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd01aa2f0 con 0x7f9dd0107ec0 2023-12-14T18:33:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd01aa460 con 0x7f9dd0119a40 2023-12-14T18:33:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd7684700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd01aa620 con 0x7f9dd0107af0 2023-12-14T18:33:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd5420700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107af0 0x7f9dd01a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd5420700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107af0 0x7f9dd01a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:55098/0 (socket says 172.21.15.31:55098) 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.762+0000 7f9dd5420700 1 -- 172.21.15.31:0/1904701030 learned_addr learned my addr 172.21.15.31:0/1904701030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 0x7f9dd01a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0107ec0 0x7f9dd01a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 -- 172.21.15.31:0/1904701030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107af0 msgr2=0x7f9dd01a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107af0 0x7f9dd01a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 -- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0107ec0 msgr2=0x7f9dd01a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0107ec0 0x7f9dd01a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 -- 172.21.15.31:0/1904701030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8009c40 con 0x7f9dd0119a40 2023-12-14T18:33:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.763+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 0x7f9dd01a9cb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e970 tx=0x7f9dcc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.764+0000 7f9dc67fc700 1 -- 172.21.15.31:0/1904701030 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc004220 con 0x7f9dd0119a40 2023-12-14T18:33:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.764+0000 7f9dc67fc700 1 -- 172.21.15.31:0/1904701030 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dcc004cf0 con 0x7f9dd0119a40 2023-12-14T18:33:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.764+0000 7f9dc67fc700 1 -- 172.21.15.31:0/1904701030 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc005700 con 0x7f9dd0119a40 2023-12-14T18:33:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.764+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd011f0f0 con 0x7f9dd0119a40 2023-12-14T18:33:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.764+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd011ef60 con 0x7f9dd0119a40 2023-12-14T18:33:09.771 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.769+0000 7f9dc67fc700 1 -- 172.21.15.31:0/1904701030 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9dcc004750 con 0x7f9dd0119a40 2023-12-14T18:33:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.772+0000 7f9de3812b80 1 Processor -- start 2023-12-14T18:33:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.773+0000 7f9dc67fc700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dbc0a14d0 0x7f9dbc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9dc67fc700 1 -- 172.21.15.31:0/1904701030 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9dcc014070 con 0x7f9dd0119a40 2023-12-14T18:33:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 -- start start 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112b72430 0x557112d14e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112bff580 con 0x557112d200f0 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112c6e810 con 0x557112b72430 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9de3812b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112c67ca0 con 0x557112d15370 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dbc0a14d0 0x7f9dbc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9dd4c1f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9dd4c1f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:55102/0 (socket says 172.21.15.31:55102) 2023-12-14T18:33:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.774+0000 7f9dd4c1f700 1 -- 172.21.15.31:0/2926987031 learned_addr learned my addr 172.21.15.31:0/2926987031 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.776+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.776+0000 7f9dd4c1f700 1 -- 172.21.15.31:0/2926987031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112b72430 msgr2=0x557112d14e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.776+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112b72430 0x557112d14e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.776+0000 7f9dd4c1f700 1 -- 172.21.15.31:0/2926987031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 msgr2=0x557112d224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.776+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d224d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.776+0000 7f9dd4c1f700 1 -- 172.21.15.31:0/2926987031 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557112ae9350 con 0x557112d15370 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.778+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dbc0a14d0 0x7f9dbc0a3980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009f90 tx=0x7f9dc80074c0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.778+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d157e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80076f0 tx=0x7f9dc8007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a1965e5eaae740 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.778+0000 7f9dbb7fe700 1 -- 172.21.15.31:0/2926987031 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8026070 con 0x557112d15370 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.778+0000 7f9dbb7fe700 1 -- 172.21.15.31:0/2926987031 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc8009460 con 0x557112d15370 2023-12-14T18:33:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.778+0000 7f9dbb7fe700 1 -- 172.21.15.31:0/2926987031 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc800e040 con 0x557112d15370 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 -- 172.21.15.31:0/2926987031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 msgr2=0x557112d157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d157e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80076f0 tx=0x7f9dc8007930 comp rx=0 tx=0).stop 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 -- 172.21.15.31:0/2926987031 shutdown_connections 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112b72430 0x557112d14e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 --2- 172.21.15.31:0/2926987031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d157e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 -- 172.21.15.31:0/2926987031 >> 172.21.15.31:0/2926987031 conn(0x557112d0b890 msgr2=0x557112d11200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 -- 172.21.15.31:0/2926987031 shutdown_connections 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.779+0000 7f9de3812b80 1 -- 172.21.15.31:0/2926987031 wait complete. 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.780+0000 7f9de3812b80 1 -- start start 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.780+0000 7f9de3812b80 1 Processor -- start 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112b72430 0x557112d1faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d1cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d200f0 0x557112d1d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112c03f30 con 0x557112b72430 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112c03bd0 con 0x557112d200f0 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112aeb640 con 0x557112d15370 2023-12-14T18:33:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9de3812b80 10 client.0 _ll_register_callbacks cb 0x557112c278b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9dd4c1f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d1cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9dd4c1f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d1cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:55112/0 (socket says 172.21.15.31:55112) 2023-12-14T18:33:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.781+0000 7f9dd4c1f700 1 -- 172.21.15.31:0/397240266 learned_addr learned my addr 172.21.15.31:0/397240266 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.782+0000 7f9dd5420700 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112b72430 0x557112d1faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.782+0000 7f9dd5420700 1 -- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 msgr2=0x557112d1cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.782+0000 7f9dd5420700 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d1cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.782+0000 7f9dd5420700 1 -- 172.21.15.31:0/397240266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d200f0 msgr2=0x557112d1d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.782+0000 7f9dd5420700 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d200f0 0x557112d1d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.782+0000 7f9dd5420700 1 -- 172.21.15.31:0/397240266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd011c6e0 con 0x557112b72430 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.783+0000 7f9dd5420700 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112b72430 0x557112d1faa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000bf70 tx=0x7f9dc00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eaf9ff87dd5fe9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.783+0000 7f9db8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.783+0000 7f9db8ff9700 1 -- 172.21.15.31:0/397240266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0004230 con 0x557112b72430 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.783+0000 7f9db8ff9700 1 -- 172.21.15.31:0/397240266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9dc0004390 con 0x557112b72430 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.783+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557112d08ea0 con 0x557112b72430 2023-12-14T18:33:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.784+0000 7f9db8ff9700 1 -- 172.21.15.31:0/397240266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc0004550 con 0x557112b72430 2023-12-14T18:33:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.785+0000 7f9de3812b80 10 client.16713 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.785+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557112c2e110 con 0x557112b72430 2023-12-14T18:33:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9db8ff9700 1 -- 172.21.15.31:0/397240266 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557112d08ea0 con 0x557112b72430 2023-12-14T18:33:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9db8ff9700 1 -- 172.21.15.31:0/397240266 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9dc001a8f0 con 0x557112b72430 2023-12-14T18:33:09.787 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9de3812b80 10 client.16713 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9de3812b80 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9de3812b80 20 client.16713 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9de3812b80 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d2fb10 0x557112d31f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9de3812b80 4 client.16713 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.786+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557112c60c80 con 0x557112d2fb10 2023-12-14T18:33:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.787+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d2fb10 0x557112d31f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.788+0000 7f9db8ff9700 10 client.16713 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.788+0000 7f9dd4c1f700 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d2fb10 0x557112d31f20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876cd715d788832b server_cookie=ca14a94e7c9f7bf7 in_seq=0 out_seq=0 2023-12-14T18:33:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.790+0000 7f9db8ff9700 1 -- 172.21.15.31:0/397240266 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557112c60c80 con 0x557112d2fb10 2023-12-14T18:33:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.790+0000 7f9db8ff9700 10 client.16713 handle_command_reply: tid=0 2023-12-14T18:33:09.792 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.790+0000 7f9de3812b80 1 client.16713 shutdown 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112b72430 msgr2=0x557112d1faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112b72430 0x557112d1faa0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000bf70 tx=0x7f9dc00106b0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 shutdown_connections 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d2fb10 0x557112d31f20 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d200f0 0x557112d1d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112b72430 0x557112d1faa0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 --2- 172.21.15.31:0/397240266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112d15370 0x557112d1cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 >> 172.21.15.31:0/397240266 conn(0x557112d0b890 msgr2=0x557112d10df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.794+0000 7f9db8ff9700 0 client.16713 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.795+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 shutdown_connections 2023-12-14T18:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.795+0000 7f9de3812b80 1 -- 172.21.15.31:0/397240266 wait complete. 2023-12-14T18:33:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.799+0000 7f9de3812b80 20 client.16713 trim_cache size 0 max 16384 2023-12-14T18:33:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 Processor -- start 2023-12-14T18:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 -- start start 2023-12-14T18:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 0x557112d1f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d1faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112d2d1f0 con 0x557112d200f0 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112d1b320 con 0x557112d15370 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9de3812b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112d18c00 con 0x557112b72430 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.827+0000 7f9dd5420700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5420700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d31b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:55120/0 (socket says 172.21.15.31:55120) 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5420700 1 -- 172.21.15.31:0/1387572449 learned_addr learned my addr 172.21.15.31:0/1387572449 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d1faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5c21700 1 -- 172.21.15.31:0/1387572449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 msgr2=0x557112d31b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d31b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5c21700 1 -- 172.21.15.31:0/1387572449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 msgr2=0x557112d1f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 0x557112d1f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.828+0000 7f9dd5c21700 1 -- 172.21.15.31:0/1387572449 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557112c2de60 con 0x557112d200f0 2023-12-14T18:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.829+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d1faa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0ca050 tx=0x7f9dcc099870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed75cb5deb5c626 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.829+0000 7f9dbbfff700 1 -- 172.21.15.31:0/1387572449 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0b1070 con 0x557112d200f0 2023-12-14T18:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.829+0000 7f9dbbfff700 1 -- 172.21.15.31:0/1387572449 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dcc004e60 con 0x557112d200f0 2023-12-14T18:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.829+0000 7f9dbbfff700 1 -- 172.21.15.31:0/1387572449 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc097d70 con 0x557112d200f0 2023-12-14T18:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.830+0000 7f9de3812b80 1 -- 172.21.15.31:0/1387572449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 msgr2=0x557112d1faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.830+0000 7f9de3812b80 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d1faa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0ca050 tx=0x7f9dcc099870 comp rx=0 tx=0).stop 2023-12-14T18:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.832+0000 7f9de3812b80 1 -- 172.21.15.31:0/1387572449 shutdown_connections 2023-12-14T18:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.832+0000 7f9de3812b80 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d1faa0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.832+0000 7f9de3812b80 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d31b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.832+0000 7f9de3812b80 1 --2- 172.21.15.31:0/1387572449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 0x557112d1f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.832+0000 7f9de3812b80 1 -- 172.21.15.31:0/1387572449 >> 172.21.15.31:0/1387572449 conn(0x557112d0b890 msgr2=0x557112d0f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.833+0000 7f9de3812b80 1 -- 172.21.15.31:0/1387572449 shutdown_connections 2023-12-14T18:33:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.833+0000 7f9de3812b80 1 -- 172.21.15.31:0/1387572449 wait complete. 2023-12-14T18:33:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 -- start start 2023-12-14T18:33:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 Processor -- start 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 0x557112d25340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112d82f10 con 0x557112d200f0 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112d83050 con 0x557112d15370 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557112d83190 con 0x557112b72430 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.834+0000 7f9de3812b80 10 client.0 _ll_register_callbacks cb 0x7f9dc000ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.835+0000 7f9dd5c21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.835+0000 7f9dd5c21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d829d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43616/0 (socket says 172.21.15.31:43616) 2023-12-14T18:33:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.835+0000 7f9dd5c21700 1 -- 172.21.15.31:0/228404345 learned_addr learned my addr 172.21.15.31:0/228404345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9dd5c21700 1 -- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 msgr2=0x557112d24e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d24e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9dd5c21700 1 -- 172.21.15.31:0/228404345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 msgr2=0x557112d25340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 0x557112d25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9dd5c21700 1 -- 172.21.15.31:0/228404345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd011ef60 con 0x557112d200f0 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9dd5c21700 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d829d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004430 tx=0x7f9dcc099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e369df6be9f252fa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.836+0000 7f9db8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.837+0000 7f9db8ff9700 1 -- 172.21.15.31:0/228404345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0b1040 con 0x557112d200f0 2023-12-14T18:33:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.837+0000 7f9db8ff9700 1 -- 172.21.15.31:0/228404345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f9dcc0a1e20 con 0x557112d200f0 2023-12-14T18:33:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.837+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557112d15200 con 0x557112d200f0 2023-12-14T18:33:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.837+0000 7f9db8ff9700 1 -- 172.21.15.31:0/228404345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dcc01e380 con 0x557112d200f0 2023-12-14T18:33:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.837+0000 7f9de3812b80 10 client.16719 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.837+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557112c14940 con 0x557112d200f0 2023-12-14T18:33:09.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:09 smithi204 ceph-mon[43866]: pgmap v163: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 653 op/s 2023-12-14T18:33:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.838+0000 7f9db8ff9700 1 -- 172.21.15.31:0/228404345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557112d15200 con 0x557112d200f0 2023-12-14T18:33:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.838+0000 7f9db8ff9700 1 -- 172.21.15.31:0/228404345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9dcc0c7070 con 0x557112d200f0 2023-12-14T18:33:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.838+0000 7f9de3812b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.838+0000 7f9de3812b80 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.839+0000 7f9de3812b80 20 client.16719 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.839+0000 7f9de3812b80 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d843f0 0x557112d86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.839+0000 7f9de3812b80 4 client.16719 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:33:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.839+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f9dc8009460 con 0x557112d843f0 2023-12-14T18:33:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.843+0000 7f9dd5420700 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d843f0 0x557112d86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.848+0000 7f9dd5420700 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d843f0 0x557112d86800 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac52f0ec419d5e server_cookie=20a99b32bb7258e6 in_seq=0 out_seq=0 2023-12-14T18:33:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:09.848+0000 7f9db8ff9700 10 client.16719 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:09 smithi031 ceph-mon[42085]: pgmap v163: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 653 op/s 2023-12-14T18:33:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:09 smithi176 ceph-mon[43506]: pgmap v163: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 653 op/s 2023-12-14T18:33:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9db8ff9700 1 -- 172.21.15.31:0/228404345 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f9dc8009460 con 0x557112d843f0 2023-12-14T18:33:10.114 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9db8ff9700 10 client.16719 handle_command_reply: tid=0 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 client.16719 shutdown 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 msgr2=0x557112d829d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d829d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc004430 tx=0x7f9dcc099b20 comp rx=0 tx=0).stop 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 shutdown_connections 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557112d843f0 0x557112d86800 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557112d200f0 0x557112d829d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557112b72430 0x557112d24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 --2- 172.21.15.31:0/228404345 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557112d15370 0x557112d25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 >> 172.21.15.31:0/228404345 conn(0x557112d0b890 msgr2=0x557112d31b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.112+0000 7f9db8ff9700 0 client.16719 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:10.115 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.113+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 shutdown_connections 2023-12-14T18:33:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.113+0000 7f9de3812b80 1 -- 172.21.15.31:0/228404345 wait complete. 2023-12-14T18:33:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.113+0000 7f9de3812b80 20 client.16719 trim_cache size 0 max 16384 2023-12-14T18:33:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:33:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:33:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:33:10.116 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:33:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dbc0a14d0 msgr2=0x7f9dbc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dbc0a14d0 0x7f9dbc0a3980 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009f90 tx=0x7f9dc80074c0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 msgr2=0x7f9dd01a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 0x7f9dd01a9cb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e970 tx=0x7f9dcc00ec80 comp rx=0 tx=0).stop 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 shutdown_connections 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9dd0119a40 0x7f9dd01a9cb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dbc0a14d0 0x7f9dbc0a3980 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dd0107af0 0x7f9dd01a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 --2- 172.21.15.31:0/1904701030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dd0107ec0 0x7f9dd01a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 >> 172.21.15.31:0/1904701030 conn(0x7f9dd006c510 msgr2=0x7f9dd011a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 shutdown_connections 2023-12-14T18:33:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:10.114+0000 7f9dd7684700 1 -- 172.21.15.31:0/1904701030 wait complete. 2023-12-14T18:33:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:10 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:33:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:10 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:10 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:10 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:10 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:10 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:10 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:33:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:10 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:10 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:11 smithi204 ceph-mon[43866]: pgmap v164: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 350 op/s 2023-12-14T18:33:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:11 smithi031 ceph-mon[42085]: pgmap v164: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 350 op/s 2023-12-14T18:33:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:11 smithi176 ceph-mon[43506]: pgmap v164: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 350 op/s 2023-12-14T18:33:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:13 smithi204 ceph-mon[43866]: pgmap v165: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 350 op/s 2023-12-14T18:33:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:13 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:33:13.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:13 smithi031 ceph-mon[42085]: pgmap v165: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 350 op/s 2023-12-14T18:33:13.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:13 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:33:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:13 smithi176 ceph-mon[43506]: pgmap v165: 129 pgs: 129 active+clean; 518 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.2 MiB/s wr, 350 op/s 2023-12-14T18:33:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:13 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:33:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:33:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:33:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:33:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:33:14.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:33:14.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:33:14.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:14 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:33:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:14 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:33:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:33:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:33:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:33:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:33:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:33:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:33:14.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:14 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:33:16.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:16 smithi204 ceph-mon[43866]: pgmap v166: 129 pgs: 129 active+clean; 521 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 540 op/s 2023-12-14T18:33:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:16 smithi031 ceph-mon[42085]: pgmap v166: 129 pgs: 129 active+clean; 521 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 540 op/s 2023-12-14T18:33:16.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:16 smithi176 ceph-mon[43506]: pgmap v166: 129 pgs: 129 active+clean; 521 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.2 MiB/s wr, 540 op/s 2023-12-14T18:33:18.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:18 smithi204 ceph-mon[43866]: pgmap v167: 129 pgs: 129 active+clean; 521 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2023-12-14T18:33:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:18 smithi031 ceph-mon[42085]: pgmap v167: 129 pgs: 129 active+clean; 521 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2023-12-14T18:33:18.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:18 smithi176 ceph-mon[43506]: pgmap v167: 129 pgs: 129 active+clean; 521 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 351 op/s 2023-12-14T18:33:20.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:20 smithi204 ceph-mon[43866]: pgmap v168: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 490 op/s 2023-12-14T18:33:20.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:20 smithi031 ceph-mon[42085]: pgmap v168: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 490 op/s 2023-12-14T18:33:20.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:20 smithi176 ceph-mon[43506]: pgmap v168: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 490 op/s 2023-12-14T18:33:22.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:22 smithi204 ceph-mon[43866]: pgmap v169: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-12-14T18:33:22.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:22 smithi031 ceph-mon[42085]: pgmap v169: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-12-14T18:33:22.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:22 smithi176 ceph-mon[43506]: pgmap v169: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-12-14T18:33:23.271 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.266+0000 7f25680b9700 1 -- 172.21.15.31:0/2992187536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f256010a820 msgr2=0x7f256010cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 --2- 172.21.15.31:0/2992187536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f256010a820 0x7f256010cc40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f255800b0a0 tx=0x7f255800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 -- 172.21.15.31:0/2992187536 shutdown_connections 2023-12-14T18:33:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 --2- 172.21.15.31:0/2992187536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f256010a820 0x7f256010cc40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 --2- 172.21.15.31:0/2992187536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2560101300 0x7f25601016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 --2- 172.21.15.31:0/2992187536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2560101c10 0x7f256010a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.272 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 -- 172.21.15.31:0/2992187536 >> 172.21.15.31:0/2992187536 conn(0x7f25600763a0 msgr2=0x7f25600767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 -- 172.21.15.31:0/2992187536 shutdown_connections 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 -- 172.21.15.31:0/2992187536 wait complete. 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 Processor -- start 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.267+0000 7f25680b9700 1 -- start start 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f25680b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 0x7f25600ff130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f25680b9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2560101c10 0x7f25600ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f25680b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256010a820 0x7f25601a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f25680b9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600ffea0 con 0x7f2560101300 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f25680b9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a4b70 con 0x7f2560101c10 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f25680b9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a4cb0 con 0x7f256010a820 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2565e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 0x7f25600ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2565e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 0x7f25600ff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35866/0 (socket says 172.21.15.31:35866) 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2565e55700 1 -- 172.21.15.31:0/2528108415 learned_addr learned my addr 172.21.15.31:0/2528108415 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256010a820 0x7f25601a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2565654700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2560101c10 0x7f25600ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 -- 172.21.15.31:0/2528108415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2560101c10 msgr2=0x7f25600ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2560101c10 0x7f25600ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 -- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 msgr2=0x7f25600ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 0x7f25600ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 -- 172.21.15.31:0/2528108415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009520 con 0x7f256010a820 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2565e55700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 0x7f25600ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.268+0000 7f2566656700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256010a820 0x7f25601a4630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f255800b070 tx=0x7f25580078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:23.275 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.269+0000 7f2556ffd700 1 -- 172.21.15.31:0/2528108415 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255800e040 con 0x7f256010a820 2023-12-14T18:33:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.269+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2558009d00 con 0x7f256010a820 2023-12-14T18:33:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.269+0000 7f2556ffd700 1 -- 172.21.15.31:0/2528108415 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255801d470 con 0x7f256010a820 2023-12-14T18:33:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.269+0000 7f2556ffd700 1 -- 172.21.15.31:0/2528108415 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580052a0 con 0x7f256010a820 2023-12-14T18:33:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.269+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2560110270 con 0x7f256010a820 2023-12-14T18:33:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.271+0000 7f2556ffd700 1 -- 172.21.15.31:0/2528108415 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2558004a50 con 0x7f256010a820 2023-12-14T18:33:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.272+0000 7f2556ffd700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f254c0a12c0 0x7f254c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.272+0000 7f2565e55700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f254c0a12c0 0x7f254c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.272+0000 7f2556ffd700 1 -- 172.21.15.31:0/2528108415 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f25580ce310 con 0x7f256010a820 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.272+0000 7f2565e55700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f254c0a12c0 0x7f254c0a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f255c009fd0 tx=0x7f255c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 Processor -- start 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 -- start start 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87c92e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c93370 0x557a87c937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87ca04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87b7d580 con 0x557a87af0430 2023-12-14T18:33:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87bec810 con 0x557a87c93370 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2574247b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87be5ca0 con 0x557a87c9e0f0 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2565e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87c92e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2565e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87c92e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35878/0 (socket says 172.21.15.31:35878) 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.273+0000 7f2565e55700 1 -- 172.21.15.31:0/2214156706 learned_addr learned my addr 172.21.15.31:0/2214156706 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2566656700 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87ca04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565654700 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c93370 0x557a87c937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565e55700 1 -- 172.21.15.31:0/2214156706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 msgr2=0x557a87ca04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565e55700 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87ca04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565e55700 1 -- 172.21.15.31:0/2214156706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c93370 msgr2=0x557a87c937e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565e55700 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c93370 0x557a87c937e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565e55700 1 -- 172.21.15.31:0/2214156706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a87a67350 con 0x557a87af0430 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f2565e55700 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87c92e30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f255c00f7d0 tx=0x7f255c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7397d72b1d00f23e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f254bfff700 1 -- 172.21.15.31:0/2214156706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c01b630 con 0x557a87af0430 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f254bfff700 1 -- 172.21.15.31:0/2214156706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255c01bc50 con 0x557a87af0430 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.274+0000 7f254bfff700 1 -- 172.21.15.31:0/2214156706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c01cc40 con 0x557a87af0430 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 -- 172.21.15.31:0/2214156706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 msgr2=0x557a87c92e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87c92e30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f255c00f7d0 tx=0x7f255c014000 comp rx=0 tx=0).stop 2023-12-14T18:33:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 -- 172.21.15.31:0/2214156706 shutdown_connections 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87ca04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87c92e30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 --2- 172.21.15.31:0/2214156706 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c93370 0x557a87c937e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 -- 172.21.15.31:0/2214156706 >> 172.21.15.31:0/2214156706 conn(0x557a87c89890 msgr2=0x557a87c8f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 -- 172.21.15.31:0/2214156706 shutdown_connections 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.275+0000 7f2574247b80 1 -- 172.21.15.31:0/2214156706 wait complete. 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 -- start start 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 Processor -- start 2023-12-14T18:33:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87c9da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c93370 0x557a87c9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87c9b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87b81f30 con 0x557a87c93370 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87b81bd0 con 0x557a87af0430 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.276+0000 7f2574247b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87a69640 con 0x557a87c9e0f0 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2574247b80 10 client.0 _ll_register_callbacks cb 0x557a87ba58b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565654700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c93370 0x557a87c9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565654700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c93370 0x557a87c9ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35880/0 (socket says 172.21.15.31:35880) 2023-12-14T18:33:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565654700 1 -- 172.21.15.31:0/3589240265 learned_addr learned my addr 172.21.15.31:0/3589240265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87c9da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2566656700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87c9b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 -- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 msgr2=0x557a87c9b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87c9b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 -- 172.21.15.31:0/3589240265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c93370 msgr2=0x557a87c9ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c93370 0x557a87c9ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 -- 172.21.15.31:0/3589240265 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c017050 con 0x557a87af0430 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2566656700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87c9b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2565e55700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87c9da50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f255c011e50 tx=0x7f255c01cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ed989c69c9f3cd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f25497fa700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f25497fa700 1 -- 172.21.15.31:0/3589240265 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c005240 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f25497fa700 1 -- 172.21.15.31:0/3589240265 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f255c01bdc0 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.277+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a87c86ea0 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f25497fa700 1 -- 172.21.15.31:0/3589240265 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f255c0053a0 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 10 client.26546 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a87bac110 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f25497fa700 1 -- 172.21.15.31:0/3589240265 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a87c86ea0 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f25497fa700 1 -- 172.21.15.31:0/3589240265 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f255c02c070 con 0x557a87af0430 2023-12-14T18:33:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 10 client.26546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 20 client.26546 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87cadaa0 0x557a87cafeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 4 client.26546 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a87bdec80 con 0x557a87cadaa0 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.278+0000 7f2565654700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87cadaa0 0x557a87cafeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.279+0000 7f25497fa700 10 client.26546 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.279+0000 7f2565654700 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87cadaa0 0x557a87cafeb0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47333127c6f4bb8c server_cookie=be89bb9f8ed5ece3 in_seq=0 out_seq=0 2023-12-14T18:33:23.293 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.279+0000 7f25497fa700 1 -- 172.21.15.31:0/3589240265 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557a87bdec80 con 0x557a87cadaa0 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.279+0000 7f25497fa700 10 client.26546 handle_command_reply: tid=0 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.279+0000 7f2574247b80 1 client.26546 shutdown 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 msgr2=0x557a87c9da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87c9da50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f255c011e50 tx=0x7f255c01cc60 comp rx=0 tx=0).stop 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 shutdown_connections 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87cadaa0 0x557a87cafeb0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c9e0f0 0x557a87c9b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87c9da50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 --2- 172.21.15.31:0/3589240265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c93370 0x557a87c9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 >> 172.21.15.31:0/3589240265 conn(0x557a87c89890 msgr2=0x557a87c8ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f25497fa700 0 client.26546 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 shutdown_connections 2023-12-14T18:33:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 1 -- 172.21.15.31:0/3589240265 wait complete. 2023-12-14T18:33:23.295 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.280+0000 7f2574247b80 20 client.26546 trim_cache size 0 max 16384 2023-12-14T18:33:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.296+0000 7f2574247b80 1 Processor -- start 2023-12-14T18:33:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 -- start start 2023-12-14T18:33:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87ca4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87c8e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c9e0f0 0x557a87c93740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87c949d0 con 0x557a87c9e0f0 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87c99340 con 0x557a87af0430 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2574247b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87c96c50 con 0x557a87c93370 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87c8e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2566656700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c9e0f0 0x557a87c93740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87c8e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45870/0 (socket says 172.21.15.31:45870) 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 -- 172.21.15.31:0/3348040611 learned_addr learned my addr 172.21.15.31:0/3348040611 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:23.301 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565e55700 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87ca4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 -- 172.21.15.31:0/3348040611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 msgr2=0x557a87ca4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87ca4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 -- 172.21.15.31:0/3348040611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c9e0f0 msgr2=0x557a87c93740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c9e0f0 0x557a87c93740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.297+0000 7f2565654700 1 -- 172.21.15.31:0/3348040611 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a87c8b0d0 con 0x557a87c93370 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.298+0000 7f2565654700 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87c8e3d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x557a87cb0780 tx=0x7f2550009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=786bc242738ff27d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.298+0000 7f2554ff9700 1 -- 172.21.15.31:0/3348040611 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550017040 con 0x557a87c93370 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.298+0000 7f2554ff9700 1 -- 172.21.15.31:0/3348040611 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2550018040 con 0x557a87c93370 2023-12-14T18:33:23.302 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.298+0000 7f2554ff9700 1 -- 172.21.15.31:0/3348040611 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550010b90 con 0x557a87c93370 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.298+0000 7f2574247b80 1 -- 172.21.15.31:0/3348040611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 msgr2=0x557a87c8e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.298+0000 7f2574247b80 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87c8e3d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x557a87cb0780 tx=0x7f2550009260 comp rx=0 tx=0).stop 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 -- 172.21.15.31:0/3348040611 shutdown_connections 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87c9e0f0 0x557a87c93740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87af0430 0x557a87ca4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 --2- 172.21.15.31:0/3348040611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87c8e3d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 -- 172.21.15.31:0/3348040611 >> 172.21.15.31:0/3348040611 conn(0x557a87c89890 msgr2=0x557a87c96390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 -- 172.21.15.31:0/3348040611 shutdown_connections 2023-12-14T18:33:23.303 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 -- 172.21.15.31:0/3348040611 wait complete. 2023-12-14T18:33:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 -- start start 2023-12-14T18:33:23.304 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.299+0000 7f2574247b80 1 Processor -- start 2023-12-14T18:33:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87ca4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87ca4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c9e0f0 0x557a87ca2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87ca35b0 con 0x557a87af0430 2023-12-14T18:33:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87ca3720 con 0x557a87c9e0f0 2023-12-14T18:33:23.307 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a87ca38e0 con 0x557a87c93370 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 10 client.0 _ll_register_callbacks cb 0x7f2550009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87ca4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87ca4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35904/0 (socket says 172.21.15.31:35904) 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 -- 172.21.15.31:0/3332611871 learned_addr learned my addr 172.21.15.31:0/3332611871 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565654700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87ca4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2566656700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c9e0f0 0x557a87ca2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 -- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 msgr2=0x557a87ca4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87ca4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 -- 172.21.15.31:0/3332611871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c9e0f0 msgr2=0x557a87ca2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:23.308 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c9e0f0 0x557a87ca2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 -- 172.21.15.31:0/3332611871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2550009c50 con 0x557a87af0430 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565654700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87ca4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2565e55700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87ca4950 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f255c01c7f0 tx=0x7f255c005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134dc04e5de29b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f25497fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f25497fa700 1 -- 172.21.15.31:0/3332611871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c020070 con 0x557a87af0430 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f25497fa700 1 -- 172.21.15.31:0/3332611871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f255c004b20 con 0x557a87af0430 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.300+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f255c01bc50 con 0x557a87af0430 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f25497fa700 1 -- 172.21.15.31:0/3332611871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f255c0046d0 con 0x557a87af0430 2023-12-14T18:33:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 10 client.16737 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a87a13b00 con 0x557a87af0430 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f25497fa700 1 -- 172.21.15.31:0/3332611871 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f255c01bc50 con 0x557a87af0430 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f25497fa700 1 -- 172.21.15.31:0/3332611871 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f255c00fae0 con 0x557a87af0430 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 10 client.16737 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 20 client.16737 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87d02970 0x557a87d04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 4 client.16737 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.301+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a87bde9a0 con 0x557a87d02970 2023-12-14T18:33:23.310 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.302+0000 7f2565654700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87d02970 0x557a87d04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.302+0000 7f2565654700 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87d02970 0x557a87d04d80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9a8d4170f8bac5 server_cookie=3f06f4f4c75cf1c5 in_seq=0 out_seq=0 2023-12-14T18:33:23.311 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:23.302+0000 7f25497fa700 10 client.16737 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f25497fa700 1 -- 172.21.15.31:0/3332611871 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a87bde9a0 con 0x557a87d02970 2023-12-14T18:33:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f25497fa700 10 client.16737 handle_command_reply: tid=0 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 client.16737 shutdown 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 msgr2=0x557a87ca4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87ca4950 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f255c01c7f0 tx=0x7f255c005d00 comp rx=0 tx=0).stop 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 shutdown_connections 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x557a87d02970 0x557a87d04d80 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557a87c9e0f0 0x557a87ca2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557a87af0430 0x557a87ca4950 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f25497fa700 0 client.16737 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 --2- 172.21.15.31:0/3332611871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a87c93370 0x557a87ca4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 >> 172.21.15.31:0/3332611871 conn(0x557a87c89890 msgr2=0x557a87c93740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.021+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 shutdown_connections 2023-12-14T18:33:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f2574247b80 1 -- 172.21.15.31:0/3332611871 wait complete. 2023-12-14T18:33:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f2574247b80 20 client.16737 trim_cache size 0 max 16384 2023-12-14T18:33:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:33:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:33:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:33:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:33:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f254c0a12c0 msgr2=0x7f254c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f25680b9700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f254c0a12c0 0x7f254c0a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f255c009fd0 tx=0x7f255c009260 comp rx=0 tx=0).stop 2023-12-14T18:33:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256010a820 msgr2=0x7f25601a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f25680b9700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256010a820 0x7f25601a4630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f255800b070 tx=0x7f25580078e0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 shutdown_connections 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.022+0000 7f25680b9700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f254c0a12c0 0x7f254c0a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.023+0000 7f25680b9700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f256010a820 0x7f25601a4630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.023+0000 7f25680b9700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2560101300 0x7f25600ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.023+0000 7f25680b9700 1 --2- 172.21.15.31:0/2528108415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2560101c10 0x7f25600ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.023+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 >> 172.21.15.31:0/2528108415 conn(0x7f25600763a0 msgr2=0x7f25600fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.023+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 shutdown_connections 2023-12-14T18:33:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:24.023+0000 7f25680b9700 1 -- 172.21.15.31:0/2528108415 wait complete. 2023-12-14T18:33:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:24 smithi204 ceph-mon[43866]: pgmap v170: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-12-14T18:33:24.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:24 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:33:24.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:24 smithi031 ceph-mon[42085]: pgmap v170: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-12-14T18:33:24.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:24 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:33:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:24 smithi176 ceph-mon[43506]: pgmap v170: 129 pgs: 129 active+clean; 529 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 328 op/s 2023-12-14T18:33:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:24 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:33:25.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:25 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:33:25.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:25 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:33:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:25 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:33:25.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:25 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:33:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:25 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:33:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:25 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:33:26.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:26 smithi204 ceph-mon[43866]: pgmap v171: 129 pgs: 129 active+clean; 533 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 471 op/s 2023-12-14T18:33:26.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:26 smithi031 ceph-mon[42085]: pgmap v171: 129 pgs: 129 active+clean; 533 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 471 op/s 2023-12-14T18:33:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:26 smithi176 ceph-mon[43506]: pgmap v171: 129 pgs: 129 active+clean; 533 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 471 op/s 2023-12-14T18:33:27.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:33:27.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 -- 172.21.15.31:0/3622426412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 msgr2=0x7f7308101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 --2- 172.21.15.31:0/3622426412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f7308101020 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7304009990 tx=0x7f7304009ca0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 -- 172.21.15.31:0/3622426412 shutdown_connections 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 --2- 172.21.15.31:0/3622426412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7308105550 0x7f730810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 --2- 172.21.15.31:0/3622426412 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f7308101020 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 --2- 172.21.15.31:0/3622426412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73081015f0 0x7f7308104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 -- 172.21.15.31:0/3622426412 >> 172.21.15.31:0/3622426412 conn(0x7f7308077570 msgr2=0x7f7308077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 -- 172.21.15.31:0/3622426412 shutdown_connections 2023-12-14T18:33:27.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.352+0000 7f73106fd700 1 -- 172.21.15.31:0/3622426412 wait complete. 2023-12-14T18:33:27.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 Processor -- start 2023-12-14T18:33:27.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 -- start start 2023-12-14T18:33:27.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f73081964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73081015f0 0x7f7308196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7308105550 0x7f730819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819b4d0 con 0x7f7308105550 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819b640 con 0x7f7308100c50 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.353+0000 7f73106fd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819b800 con 0x7f73081015f0 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f73081964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f73081964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34954/0 (socket says 172.21.15.31:34954) 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 -- 172.21.15.31:0/2361790313 learned_addr learned my addr 172.21.15.31:0/2361790313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730dc98700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73081015f0 0x7f7308196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 -- 172.21.15.31:0/2361790313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73081015f0 msgr2=0x7f7308196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73081015f0 0x7f7308196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 -- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7308105550 msgr2=0x7f730819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7308105550 0x7f730819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 -- 172.21.15.31:0/2361790313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040095f0 con 0x7f7308100c50 2023-12-14T18:33:27.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.354+0000 7f730e499700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f73081964f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7304005f40 tx=0x7f73040036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.361+0000 7f72ff7fe700 1 -- 172.21.15.31:0/2361790313 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304005310 con 0x7f7308100c50 2023-12-14T18:33:27.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.361+0000 7f72ff7fe700 1 -- 172.21.15.31:0/2361790313 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f730403f070 con 0x7f7308100c50 2023-12-14T18:33:27.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.361+0000 7f72ff7fe700 1 -- 172.21.15.31:0/2361790313 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304004c70 con 0x7f7308100c50 2023-12-14T18:33:27.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.362+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73081103a0 con 0x7f7308100c50 2023-12-14T18:33:27.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.362+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7308110210 con 0x7f7308100c50 2023-12-14T18:33:27.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.367+0000 7f731c88bb80 1 Processor -- start 2023-12-14T18:33:27.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 -- start start 2023-12-14T18:33:27.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbcd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 0x55c39dbe0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbe3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dab75e0 con 0x55c39dbe1360 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39db26870 con 0x55c39dbcdc90 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f731c88bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39db1fd00 con 0x55c39dbcd380 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.368+0000 7f730ec9a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbcd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730ec9a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbcd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46834/0 (socket says 172.21.15.31:46834) 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730ec9a700 1 -- 172.21.15.31:0/3944484751 learned_addr learned my addr 172.21.15.31:0/3944484751 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730dc98700 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbe3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730dc98700 1 -- 172.21.15.31:0/3944484751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 msgr2=0x55c39dbcd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730dc98700 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbcd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730dc98700 1 -- 172.21.15.31:0/3944484751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 msgr2=0x55c39dbe0e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:33:27.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730dc98700 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 0x55c39dbe0e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.369+0000 7f730dc98700 1 -- 172.21.15.31:0/3944484751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c39d9a13b0 con 0x55c39dbe1360 2023-12-14T18:33:27.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.371+0000 7f72ff7fe700 1 -- 172.21.15.31:0/2361790313 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7304041070 con 0x7f7308100c50 2023-12-14T18:33:27.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.372+0000 7f72ff7fe700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f72f40a1410 0x7f72f40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.372+0000 7f72ff7fe700 1 -- 172.21.15.31:0/2361790313 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f73040ef930 con 0x7f7308100c50 2023-12-14T18:33:27.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.373+0000 7f730dc98700 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbe3740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7308067ab0 tx=0x7f72f8009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c586d85519c4e00 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.373+0000 7f72fcff9700 1 -- 172.21.15.31:0/3944484751 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8024680 con 0x55c39dbe1360 2023-12-14T18:33:27.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.373+0000 7f72fcff9700 1 -- 172.21.15.31:0/3944484751 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72f8024ca0 con 0x55c39dbe1360 2023-12-14T18:33:27.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.373+0000 7f72fcff9700 1 -- 172.21.15.31:0/3944484751 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8023d40 con 0x55c39dbe1360 2023-12-14T18:33:27.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.374+0000 7f731c88bb80 1 -- 172.21.15.31:0/3944484751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 msgr2=0x55c39dbe3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.374+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbe3740 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7308067ab0 tx=0x7f72f8009b40 comp rx=0 tx=0).stop 2023-12-14T18:33:27.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.375+0000 7f731c88bb80 1 -- 172.21.15.31:0/3944484751 shutdown_connections 2023-12-14T18:33:27.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.375+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbe3740 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.375+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbcd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.375+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3944484751 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 0x55c39dbe0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.375+0000 7f731c88bb80 1 -- 172.21.15.31:0/3944484751 >> 172.21.15.31:0/3944484751 conn(0x55c39dbc3ca0 msgr2=0x55c39dbc9720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.377+0000 7f730ec9a700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f72f40a1410 0x7f72f40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.381+0000 7f730ec9a700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f72f40a1410 0x7f72f40a38c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7300009fa0 tx=0x7f730000b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.382+0000 7f731c88bb80 1 -- 172.21.15.31:0/3944484751 shutdown_connections 2023-12-14T18:33:27.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.382+0000 7f731c88bb80 1 -- 172.21.15.31:0/3944484751 wait complete. 2023-12-14T18:33:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.382+0000 7f731c88bb80 1 -- start start 2023-12-14T18:33:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.382+0000 7f731c88bb80 1 Processor -- start 2023-12-14T18:33:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbd8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 0x55c39dbcfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbd0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dabbf90 con 0x55c39dbe1360 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dabbc30 con 0x55c39dbcdc90 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39d9a36a0 con 0x55c39dbcd380 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.383+0000 7f731c88bb80 10 client.0 _ll_register_callbacks cb 0x55c39dadf910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbd8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbd8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46844/0 (socket says 172.21.15.31:46844) 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 -- 172.21.15.31:0/3734825654 learned_addr learned my addr 172.21.15.31:0/3734825654 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730ec9a700 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbd0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 -- 172.21.15.31:0/3734825654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 msgr2=0x55c39dbcfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 0x55c39dbcfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 -- 172.21.15.31:0/3734825654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 msgr2=0x55c39dbd0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbd0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.384+0000 7f730e499700 1 -- 172.21.15.31:0/3734825654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f801e040 con 0x55c39dbcd380 2023-12-14T18:33:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.385+0000 7f730e499700 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbd8750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73040b9170 tx=0x7f73040bc0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dec1fba1a7f12c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.385+0000 7f72f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:33:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.385+0000 7f72f1ffb700 1 -- 172.21.15.31:0/3734825654 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040c4ac0 con 0x55c39dbcd380 2023-12-14T18:33:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.385+0000 7f72f1ffb700 1 -- 172.21.15.31:0/3734825654 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f73040bc960 con 0x55c39dbcd380 2023-12-14T18:33:27.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.385+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c39dbc12b0 con 0x55c39dbcd380 2023-12-14T18:33:27.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.385+0000 7f72f1ffb700 1 -- 172.21.15.31:0/3734825654 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73040bcb20 con 0x55c39dbcd380 2023-12-14T18:33:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.389+0000 7f731c88bb80 10 client.26584 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.389+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c39dae6170 con 0x55c39dbcd380 2023-12-14T18:33:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f72f1ffb700 1 -- 172.21.15.31:0/3734825654 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c39dbc12b0 con 0x55c39dbcd380 2023-12-14T18:33:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f72f1ffb700 1 -- 172.21.15.31:0/3734825654 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f73040bdb60 con 0x55c39dbcd380 2023-12-14T18:33:27.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f731c88bb80 10 client.26584 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f731c88bb80 10 client.26584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f731c88bb80 20 client.26584 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dbe80f0 0x55c39dbea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f731c88bb80 4 client.26584 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:27.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c39db18ce0 con 0x55c39dbe80f0 2023-12-14T18:33:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.390+0000 7f730dc98700 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dbe80f0 0x55c39dbea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.391+0000 7f730dc98700 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dbe80f0 0x55c39dbea500 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cf9cc6694053c3 server_cookie=64f4954a5719b530 in_seq=0 out_seq=0 2023-12-14T18:33:27.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.391+0000 7f72f1ffb700 10 client.26584 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:27.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.392+0000 7f72f1ffb700 1 -- 172.21.15.31:0/3734825654 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c39db18ce0 con 0x55c39dbe80f0 2023-12-14T18:33:27.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.392+0000 7f72f1ffb700 10 client.26584 handle_command_reply: tid=0 2023-12-14T18:33:27.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 client.26584 shutdown 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 msgr2=0x55c39dbd8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbd8750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f73040b9170 tx=0x7f73040bc0e0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 shutdown_connections 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dbe80f0 0x55c39dbea500 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbe1360 0x55c39dbd0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbd8750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 --2- 172.21.15.31:0/3734825654 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcdc90 0x55c39dbcfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.394+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 >> 172.21.15.31:0/3734825654 conn(0x55c39dbc3ca0 msgr2=0x55c39dbc92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.395+0000 7f72f1ffb700 0 client.26584 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:27.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.395+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 shutdown_connections 2023-12-14T18:33:27.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.395+0000 7f731c88bb80 1 -- 172.21.15.31:0/3734825654 wait complete. 2023-12-14T18:33:27.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.395+0000 7f731c88bb80 20 client.26584 trim_cache size 0 max 16384 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.426+0000 7f731c88bb80 1 Processor -- start 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.426+0000 7f731c88bb80 1 -- start start 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 0x55c39dbdbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbc9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbe1360 0x55c39dbcf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f731c88bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dbd4950 con 0x55c39dbcdc90 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f731c88bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dbdd450 con 0x55c39dbcd380 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f731c88bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dbcf240 con 0x55c39dbe1360 2023-12-14T18:33:27.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 0x55c39dbdbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 0x55c39dbdbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34980/0 (socket says 172.21.15.31:34980) 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 -- 172.21.15.31:0/6348772 learned_addr learned my addr 172.21.15.31:0/6348772 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730dc98700 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbc9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730ec9a700 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbe1360 0x55c39dbcf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 -- 172.21.15.31:0/6348772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbe1360 msgr2=0x55c39dbcf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbe1360 0x55c39dbcf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 -- 172.21.15.31:0/6348772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 msgr2=0x55c39dbc9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbc9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.427+0000 7f730e499700 1 -- 172.21.15.31:0/6348772 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c39dae5ec0 con 0x55c39dbcd380 2023-12-14T18:33:27.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.428+0000 7f730e499700 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 0x55c39dbdbd50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73040ecda0 tx=0x7f73040bacb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea2f3debde5fc8a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.429+0000 7f72fd7fa700 1 -- 172.21.15.31:0/6348772 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040c25f0 con 0x55c39dbcd380 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.429+0000 7f72fd7fa700 1 -- 172.21.15.31:0/6348772 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73040c4ac0 con 0x55c39dbcd380 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.429+0000 7f72fd7fa700 1 -- 172.21.15.31:0/6348772 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040bbe10 con 0x55c39dbcd380 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 -- 172.21.15.31:0/6348772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 msgr2=0x55c39dbdbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 0x55c39dbdbd50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f73040ecda0 tx=0x7f73040bacb0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 -- 172.21.15.31:0/6348772 shutdown_connections 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbe1360 0x55c39dbcf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbcd380 0x55c39dbdbd50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 --2- 172.21.15.31:0/6348772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbc9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.430+0000 7f731c88bb80 1 -- 172.21.15.31:0/6348772 >> 172.21.15.31:0/6348772 conn(0x55c39dbc3ca0 msgr2=0x55c39dbd4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.436+0000 7f731c88bb80 1 -- 172.21.15.31:0/6348772 shutdown_connections 2023-12-14T18:33:27.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.438+0000 7f731c88bb80 1 -- 172.21.15.31:0/6348772 wait complete. 2023-12-14T18:33:27.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.438+0000 7f731c88bb80 1 -- start start 2023-12-14T18:33:27.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.438+0000 7f731c88bb80 1 Processor -- start 2023-12-14T18:33:27.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbf54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbf59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 0x55c39dbf5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dc3b770 con 0x55c39dbcdc90 2023-12-14T18:33:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dc3b8e0 con 0x55c39dbe1360 2023-12-14T18:33:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c39dc3baa0 con 0x55c39dbcd380 2023-12-14T18:33:27.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.440+0000 7f731c88bb80 10 client.0 _ll_register_callbacks cb 0x7f73040bc4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.447+0000 7f730ec9a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 0x55c39dbf5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.447+0000 7f730ec9a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 0x55c39dbf5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34984/0 (socket says 172.21.15.31:34984) 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.447+0000 7f730ec9a700 1 -- 172.21.15.31:0/192701490 learned_addr learned my addr 172.21.15.31:0/192701490 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.447+0000 7f730dc98700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbf59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.447+0000 7f730e499700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbf54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f730ec9a700 1 -- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 msgr2=0x55c39dbf54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f730ec9a700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbf54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f730ec9a700 1 -- 172.21.15.31:0/192701490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 msgr2=0x55c39dbf59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f730ec9a700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbf59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f730ec9a700 1 -- 172.21.15.31:0/192701490 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c39dae6170 con 0x55c39dbe1360 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f730ec9a700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 0x55c39dbf5f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7300015bb0 tx=0x7f7300004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397fbab17b758a16 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f72f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f72f1ffb700 1 -- 172.21.15.31:0/192701490 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300013a60 con 0x55c39dbe1360 2023-12-14T18:33:27.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.448+0000 7f72f1ffb700 1 -- 172.21.15.31:0/192701490 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7300013bc0 con 0x55c39dbe1360 2023-12-14T18:33:27.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.449+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72f8024ca0 con 0x55c39dbe1360 2023-12-14T18:33:27.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.457+0000 7f72f1ffb700 1 -- 172.21.15.31:0/192701490 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7300013d80 con 0x55c39dbe1360 2023-12-14T18:33:27.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.457+0000 7f731c88bb80 10 client.26564 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:27.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.457+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c39dacc9a0 con 0x55c39dbe1360 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f72f1ffb700 1 -- 172.21.15.31:0/192701490 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72f8024ca0 con 0x55c39dbe1360 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f72f1ffb700 1 -- 172.21.15.31:0/192701490 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7300022500 con 0x55c39dbe1360 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f731c88bb80 10 client.26564 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f731c88bb80 10 client.26564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f731c88bb80 20 client.26564 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f731c88bb80 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dc3d2d0 0x55c39dc3f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f731c88bb80 4 client.26564 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:33:27.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.459+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c39db18a00 con 0x55c39dc3d2d0 2023-12-14T18:33:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.460+0000 7f730e499700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dc3d2d0 0x55c39dc3f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.460+0000 7f730e499700 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dc3d2d0 0x55c39dc3f6e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe58fad50e41245 server_cookie=5d99c1d0bca16006 in_seq=0 out_seq=0 2023-12-14T18:33:27.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.460+0000 7f72f1ffb700 10 client.26564 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.463+0000 7f72f1ffb700 1 -- 172.21.15.31:0/192701490 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c39db18a00 con 0x55c39dc3d2d0 2023-12-14T18:33:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.463+0000 7f72f1ffb700 10 client.26564 handle_command_reply: tid=0 2023-12-14T18:33:27.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.463+0000 7f731c88bb80 1 client.26564 shutdown 2023-12-14T18:33:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.464+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 msgr2=0x55c39dbf5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.464+0000 7f731c88bb80 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 0x55c39dbf5f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7300015bb0 tx=0x7f7300004bd0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 shutdown_connections 2023-12-14T18:33:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f731c88bb80 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c39dc3d2d0 0x55c39dc3f6e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f731c88bb80 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c39dbe1360 0x55c39dbf5f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f731c88bb80 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c39dbcd380 0x55c39dbf54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f731c88bb80 1 --2- 172.21.15.31:0/192701490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c39dbcdc90 0x55c39dbf59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 >> 172.21.15.31:0/192701490 conn(0x55c39dbc3ca0 msgr2=0x55c39dbcf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.465+0000 7f72f1ffb700 0 client.26564 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.467+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 shutdown_connections 2023-12-14T18:33:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.470+0000 7f731c88bb80 1 -- 172.21.15.31:0/192701490 wait complete. 2023-12-14T18:33:27.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.472+0000 7f731c88bb80 20 client.26564 trim_cache size 0 max 16384 2023-12-14T18:33:27.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:33:27.474 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:33:27.474 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:33:27.474 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:33:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.477+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f72f40a1410 msgr2=0x7f72f40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.477+0000 7f73106fd700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f72f40a1410 0x7f72f40a38c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7300009fa0 tx=0x7f730000b040 comp rx=0 tx=0).stop 2023-12-14T18:33:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.477+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 msgr2=0x7f73081964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.477+0000 7f73106fd700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f73081964f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7304005f40 tx=0x7f73040036f0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.478+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 shutdown_connections 2023-12-14T18:33:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.478+0000 7f73106fd700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7308105550 0x7f730819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.478+0000 7f73106fd700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7308100c50 0x7f73081964f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.478+0000 7f73106fd700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f72f40a1410 0x7f72f40a38c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.478+0000 7f73106fd700 1 --2- 172.21.15.31:0/2361790313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73081015f0 0x7f7308196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.478+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 >> 172.21.15.31:0/2361790313 conn(0x7f7308077570 msgr2=0x7f73080fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.479+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 shutdown_connections 2023-12-14T18:33:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.479+0000 7f73106fd700 1 -- 172.21.15.31:0/2361790313 wait complete. 2023-12-14T18:33:27.501 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:33:27.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:33:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 -- 172.21.15.31:0/2998494508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc105510 msgr2=0x7f7ffc10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 --2- 172.21.15.31:0/2998494508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc105510 0x7f7ffc10d0e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400b0d0 tx=0x7f7ff400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 -- 172.21.15.31:0/2998494508 shutdown_connections 2023-12-14T18:33:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 --2- 172.21.15.31:0/2998494508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc105510 0x7f7ffc10d0e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 --2- 172.21.15.31:0/2998494508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc069320 0x7f7ffc0696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 --2- 172.21.15.31:0/2998494508 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 -- 172.21.15.31:0/2998494508 >> 172.21.15.31:0/2998494508 conn(0x7f7ffc075fa0 msgr2=0x7f7ffc0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 -- 172.21.15.31:0/2998494508 shutdown_connections 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 -- 172.21.15.31:0/2998494508 wait complete. 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.641+0000 7f8003514700 1 Processor -- start 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 -- start start 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 0x7f7ffc196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc105510 0x7f7ffc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc19b470 con 0x7f7ffc069cc0 2023-12-14T18:33:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc19b5e0 con 0x7f7ffc069320 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8003514700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc19b7a0 con 0x7f7ffc105510 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f80012b0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 0x7f7ffc196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc1969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56818/0 (socket says 172.21.15.31:56818) 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f80012b0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 0x7f7ffc196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35010/0 (socket says 172.21.15.31:35010) 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 -- 172.21.15.31:0/1610857068 learned_addr learned my addr 172.21.15.31:0/1610857068 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8001ab1700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc105510 0x7f7ffc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 -- 172.21.15.31:0/1610857068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc105510 msgr2=0x7f7ffc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc105510 0x7f7ffc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 -- 172.21.15.31:0/1610857068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 msgr2=0x7f7ffc196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 0x7f7ffc196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f8000aaf700 1 -- 172.21.15.31:0/1610857068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4009d00 con 0x7f7ffc069cc0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.642+0000 7f80012b0700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 0x7f7ffc196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.643+0000 7f8000aaf700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc1969d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00e940 tx=0x7f7fec00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.643+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec009730 con 0x7f7ffc069cc0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.643+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fec010430 con 0x7f7ffc069cc0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.643+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc1102e0 con 0x7f7ffc069cc0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.643+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00cea0 con 0x7f7ffc069cc0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.643+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc110150 con 0x7f7ffc069cc0 2023-12-14T18:33:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.644+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ffc110470 con 0x7f7ffc069cc0 2023-12-14T18:33:27.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.645+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7fec010810 con 0x7f7ffc069cc0 2023-12-14T18:33:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.646+0000 7f7ff27fc700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7fe80a12c0 0x7f7fe80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.646+0000 7f80012b0700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7fe80a12c0 0x7f7fe80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.646+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f7fec014070 con 0x7f7ffc069cc0 2023-12-14T18:33:27.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.647+0000 7f80012b0700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7fe80a12c0 0x7f7fe80a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8005950 tx=0x7f7ff800b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:27.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.650+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7fec0963f0 con 0x7f7ffc069cc0 2023-12-14T18:33:27.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.910+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ffc067ec0 con 0x7f7ffc069cc0 2023-12-14T18:33:27.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.911+0000 7f7ff27fc700 1 -- 172.21.15.31:0/1610857068 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10822 (secure 0 0 0) 0x7f7fec099a10 con 0x7f7ffc069cc0 2023-12-14T18:33:27.912 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:33:27.912 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:31:40.963020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7fe80a12c0 msgr2=0x7f7fe80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7fe80a12c0 0x7f7fe80a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8005950 tx=0x7f7ff800b410 comp rx=0 tx=0).stop 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 msgr2=0x7f7ffc1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc1969d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00e940 tx=0x7f7fec00ec50 comp rx=0 tx=0).stop 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 shutdown_connections 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7fe80a12c0 0x7f7fe80a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ffc105510 0x7f7ffc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ffc069320 0x7f7ffc196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 --2- 172.21.15.31:0/1610857068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ffc069cc0 0x7f7ffc1969d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:27.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 >> 172.21.15.31:0/1610857068 conn(0x7f7ffc075fa0 msgr2=0x7f7ffc0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 shutdown_connections 2023-12-14T18:33:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:27.913+0000 7f8003514700 1 -- 172.21.15.31:0/1610857068 wait complete. 2023-12-14T18:33:27.916 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-12-14T18:33:27.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:33:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.071+0000 7f17c70e5700 1 -- 172.21.15.31:0/2513947542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 msgr2=0x7f17c00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.071+0000 7f17c70e5700 1 --2- 172.21.15.31:0/2513947542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c00ff760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f17b00099c0 tx=0x7f17b0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 -- 172.21.15.31:0/2513947542 shutdown_connections 2023-12-14T18:33:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 --2- 172.21.15.31:0/2513947542 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17c01043d0 0x7f17c010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 --2- 172.21.15.31:0/2513947542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c00ff760 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 --2- 172.21.15.31:0/2513947542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c00ffd30 0x7f17c0103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 -- 172.21.15.31:0/2513947542 >> 172.21.15.31:0/2513947542 conn(0x7f17c0075fd0 msgr2=0x7f17c00763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 -- 172.21.15.31:0/2513947542 shutdown_connections 2023-12-14T18:33:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 -- 172.21.15.31:0/2513947542 wait complete. 2023-12-14T18:33:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 Processor -- start 2023-12-14T18:33:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 -- start start 2023-12-14T18:33:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.072+0000 7f17c70e5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c01963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c70e5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17c00ffd30 0x7f17c01968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c70e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c01043d0 0x7f17c019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c70e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019b380 con 0x7f17c01043d0 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c70e5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019b4f0 con 0x7f17c00ffd30 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c70e5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019b6b0 con 0x7f17c00ff390 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c5682700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c01043d0 0x7f17c019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c01963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c01963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46920/0 (socket says 172.21.15.31:46920) 2023-12-14T18:33:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 -- 172.21.15.31:0/3921937758 learned_addr learned my addr 172.21.15.31:0/3921937758 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17bffff700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17c00ffd30 0x7f17c01968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 -- 172.21.15.31:0/3921937758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17c00ffd30 msgr2=0x7f17c01968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17c00ffd30 0x7f17c01968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 -- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c01043d0 msgr2=0x7f17c019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c01043d0 0x7f17c019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 -- 172.21.15.31:0/3921937758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8009c40 con 0x7f17c00ff390 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c5682700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c01043d0 0x7f17c019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c4e81700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c01963a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b580 tx=0x7f17b000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17bdffb700 1 -- 172.21.15.31:0/3921937758 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b001d070 con 0x7f17c00ff390 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b00095f0 con 0x7f17c00ff390 2023-12-14T18:33:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17bdffb700 1 -- 172.21.15.31:0/3921937758 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b0005050 con 0x7f17c00ff390 2023-12-14T18:33:28.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.073+0000 7f17bdffb700 1 -- 172.21.15.31:0/3921937758 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b00049b0 con 0x7f17c00ff390 2023-12-14T18:33:28.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.074+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c010f1e0 con 0x7f17c00ff390 2023-12-14T18:33:28.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.076+0000 7f17bdffb700 1 -- 172.21.15.31:0/3921937758 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f17b00051c0 con 0x7f17c00ff390 2023-12-14T18:33:28.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.076+0000 7f17bdffb700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17ac0a12c0 0x7f17ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.077+0000 7f17bdffb700 1 -- 172.21.15.31:0/3921937758 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f17b00cde60 con 0x7f17c00ff390 2023-12-14T18:33:28.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.077+0000 7f17bffff700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17ac0a12c0 0x7f17ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.077+0000 7f17bffff700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17ac0a12c0 0x7f17ac0a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17c01979b0 tx=0x7f17a8009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.077+0000 7f17d3273b80 1 Processor -- start 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 -- start start 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32e380 0x5620ff32e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff339610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff339b50 0x5620ff33bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff2185e0 con 0x5620ff339b50 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff287870 con 0x5620ff32e380 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17d3273b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff280d00 con 0x5620ff32ec90 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff339610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17c5682700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff339b50 0x5620ff33bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff339610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46936/0 (socket says 172.21.15.31:46936) 2023-12-14T18:33:28.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 -- 172.21.15.31:0/3510973720 learned_addr learned my addr 172.21.15.31:0/3510973720 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17c4e81700 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32e380 0x5620ff32e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 -- 172.21.15.31:0/3510973720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32e380 msgr2=0x5620ff32e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32e380 0x5620ff32e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 -- 172.21.15.31:0/3510973720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff339b50 msgr2=0x5620ff33bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff339b50 0x5620ff33bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17bffff700 1 -- 172.21.15.31:0/3510973720 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ff1023b0 con 0x5620ff32ec90 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.078+0000 7f17c5682700 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff339b50 0x5620ff33bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.079+0000 7f17bffff700 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff339610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f17a8011cf0 tx=0x7f17a8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4f72f5cccbdd3f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.079+0000 7f17b6ffd700 1 -- 172.21.15.31:0/3510973720 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a800ea40 con 0x5620ff32ec90 2023-12-14T18:33:28.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.079+0000 7f17b6ffd700 1 -- 172.21.15.31:0/3510973720 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a8021070 con 0x5620ff32ec90 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.079+0000 7f17b6ffd700 1 -- 172.21.15.31:0/3510973720 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a801d410 con 0x5620ff32ec90 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.079+0000 7f17d3273b80 1 -- 172.21.15.31:0/3510973720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 msgr2=0x5620ff339610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.079+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff339610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f17a8011cf0 tx=0x7f17a8014000 comp rx=0 tx=0).stop 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 -- 172.21.15.31:0/3510973720 shutdown_connections 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff339b50 0x5620ff33bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32e380 0x5620ff32e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3510973720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff339610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 -- 172.21.15.31:0/3510973720 >> 172.21.15.31:0/3510973720 conn(0x5620ff324ca0 msgr2=0x5620ff32a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 -- 172.21.15.31:0/3510973720 shutdown_connections 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 -- 172.21.15.31:0/3510973720 wait complete. 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 -- start start 2023-12-14T18:33:28.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.080+0000 7f17d3273b80 1 Processor -- start 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff3367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff339b50 0x5620ff336dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff21cf90 con 0x5620ff32e380 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff21cc30 con 0x5620ff339b50 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff1046a0 con 0x5620ff32ec90 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17d3273b80 10 client.0 _ll_register_callbacks cb 0x5620ff240910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56840/0 (socket says 172.21.15.31:56840) 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 -- 172.21.15.31:0/1780713247 learned_addr learned my addr 172.21.15.31:0/1780713247 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17bffff700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff3367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c5682700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff339b50 0x5620ff336dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 -- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 msgr2=0x5620ff3367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff3367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 -- 172.21.15.31:0/1780713247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff339b50 msgr2=0x5620ff336dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff339b50 0x5620ff336dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 -- 172.21.15.31:0/1780713247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a8017070 con 0x5620ff32e380 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f17c4e81700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17b00036c0 tx=0x7f17b009ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b465f18250105d75 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.081+0000 7f179ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f179ffff700 1 -- 172.21.15.31:0/1780713247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b009b0a0 con 0x5620ff32e380 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f179ffff700 1 -- 172.21.15.31:0/1780713247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f17b009a120 con 0x5620ff32e380 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620ff3222b0 con 0x5620ff32e380 2023-12-14T18:33:28.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f179ffff700 1 -- 172.21.15.31:0/1780713247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b009a2e0 con 0x5620ff32e380 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 10 client.16767 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620ff247170 con 0x5620ff32e380 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f179ffff700 1 -- 172.21.15.31:0/1780713247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620ff3222b0 con 0x5620ff32e380 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f179ffff700 1 -- 172.21.15.31:0/1780713247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f17b0004b50 con 0x5620ff32e380 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 20 client.16767 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff349050 0x5620ff34b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 4 client.16767 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.082+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620ff279ce0 con 0x5620ff349050 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.083+0000 7f17bffff700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff349050 0x5620ff34b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.083+0000 7f17bffff700 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff349050 0x5620ff34b460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0beb5314ddbc95c server_cookie=adde21f4ed98ef5b in_seq=0 out_seq=0 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.083+0000 7f179ffff700 10 client.16767 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f179ffff700 1 -- 172.21.15.31:0/1780713247 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620ff279ce0 con 0x5620ff349050 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f179ffff700 10 client.16767 handle_command_reply: tid=0 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 client.16767 shutdown 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 msgr2=0x5620ff339530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f17b00036c0 tx=0x7f17b009ad70 comp rx=0 tx=0).stop 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 shutdown_connections 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff349050 0x5620ff34b460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff339b50 0x5620ff336dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f179ffff700 0 client.16767 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:33:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 --2- 172.21.15.31:0/1780713247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff3367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 >> 172.21.15.31:0/1780713247 conn(0x5620ff324ca0 msgr2=0x5620ff32a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 shutdown_connections 2023-12-14T18:33:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.084+0000 7f17d3273b80 1 -- 172.21.15.31:0/1780713247 wait complete. 2023-12-14T18:33:28.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.085+0000 7f17d3273b80 20 client.16767 trim_cache size 0 max 16384 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 Processor -- start 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 -- start start 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff324ca0 0x5620ff338ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32ec90 0x5620ff331ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff279ce0 con 0x5620ff32e380 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.099+0000 7f17d3273b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff32f390 con 0x5620ff32ec90 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17d3273b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff32f5b0 con 0x5620ff324ca0 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56852/0 (socket says 172.21.15.31:56852) 2023-12-14T18:33:28.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 -- 172.21.15.31:0/3107755730 learned_addr learned my addr 172.21.15.31:0/3107755730 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17c5682700 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32ec90 0x5620ff331ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17c4e81700 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff324ca0 0x5620ff338ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 -- 172.21.15.31:0/3107755730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff324ca0 msgr2=0x5620ff338ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff324ca0 0x5620ff338ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 -- 172.21.15.31:0/3107755730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32ec90 msgr2=0x5620ff331ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32ec90 0x5620ff331ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 -- 172.21.15.31:0/3107755730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620ff246ec0 con 0x5620ff32e380 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17bffff700 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17a8011cf0 tx=0x7f17a801caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc59831ce01f065 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17b77fe700 1 -- 172.21.15.31:0/3107755730 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a801ce30 con 0x5620ff32e380 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17b77fe700 1 -- 172.21.15.31:0/3107755730 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a8033160 con 0x5620ff32e380 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.100+0000 7f17b77fe700 1 -- 172.21.15.31:0/3107755730 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a801c540 con 0x5620ff32e380 2023-12-14T18:33:28.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 -- 172.21.15.31:0/3107755730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 msgr2=0x5620ff339530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f17a8011cf0 tx=0x7f17a801caf0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 -- 172.21.15.31:0/3107755730 shutdown_connections 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff32ec90 0x5620ff331ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff324ca0 0x5620ff338ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 --2- 172.21.15.31:0/3107755730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff339530 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 -- 172.21.15.31:0/3107755730 >> 172.21.15.31:0/3107755730 conn(0x5620ff33c200 msgr2=0x5620ff345860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 -- 172.21.15.31:0/3107755730 shutdown_connections 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.101+0000 7f17d3273b80 1 -- 172.21.15.31:0/3107755730 wait complete. 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 -- start start 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 Processor -- start 2023-12-14T18:33:28.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 0x5620ff340130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff340670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff33e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff33ec00 con 0x5620ff32e380 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff33ed70 con 0x5620ff324ca0 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620ff33ef30 con 0x5620ff32ec90 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17d3273b80 10 client.0 _ll_register_callbacks cb 0x7f17b009a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff340670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17c5682700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff33e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17c4e81700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 0x5620ff340130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17c5682700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff33e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46966/0 (socket says 172.21.15.31:46966) 2023-12-14T18:33:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17c4e81700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 0x5620ff340130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35080/0 (socket says 172.21.15.31:35080) 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17c4e81700 1 -- 172.21.15.31:0/2392420203 learned_addr learned my addr 172.21.15.31:0/2392420203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.102+0000 7f17bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff340670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56860/0 (socket says 172.21.15.31:56860) 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17bffff700 1 -- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 msgr2=0x5620ff33e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17bffff700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff33e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17bffff700 1 -- 172.21.15.31:0/2392420203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 msgr2=0x5620ff340130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17bffff700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 0x5620ff340130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17bffff700 1 -- 172.21.15.31:0/2392420203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a802a040 con 0x5620ff32e380 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17c5682700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff33e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17c4e81700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 0x5620ff340130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17bffff700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff340670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x5620ff32ffe0 tx=0x7f17a8011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88103bf32361045a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f179ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f179ffff700 1 -- 172.21.15.31:0/2392420203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a80332d0 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f179ffff700 1 -- 172.21.15.31:0/2392420203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f17a8033430 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17a8021070 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f179ffff700 1 -- 172.21.15.31:0/2392420203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17a80335f0 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17d3273b80 10 client.16773 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.103+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620ff22d9a0 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f179ffff700 1 -- 172.21.15.31:0/2392420203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17a8021070 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f179ffff700 1 -- 172.21.15.31:0/2392420203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f17a803a020 con 0x5620ff32e380 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17d3273b80 10 client.16773 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17d3273b80 10 client.16773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17d3273b80 20 client.16773 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17d3273b80 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff339b50 0x5620ff39fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17d3273b80 4 client.16773 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:33:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620ff279a00 con 0x5620ff339b50 2023-12-14T18:33:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17c4e81700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff339b50 0x5620ff39fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f17c4e81700 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff339b50 0x5620ff39fd90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45f830063a52421a server_cookie=4c7d2fe82bb4e61e in_seq=0 out_seq=0 2023-12-14T18:33:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.104+0000 7f179ffff700 10 client.16773 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:33:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f179ffff700 1 -- 172.21.15.31:0/2392420203 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620ff279a00 con 0x5620ff339b50 2023-12-14T18:33:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f179ffff700 10 client.16773 handle_command_reply: tid=0 2023-12-14T18:33:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 client.16773 shutdown 2023-12-14T18:33:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 msgr2=0x5620ff340670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff340670 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x5620ff32ffe0 tx=0x7f17a8011860 comp rx=0 tx=0).stop 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 shutdown_connections 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5620ff339b50 0x5620ff39fd90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5620ff32ec90 0x5620ff33e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5620ff324ca0 0x5620ff340130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f179ffff700 0 client.16773 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 --2- 172.21.15.31:0/2392420203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5620ff32e380 0x5620ff340670 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 >> 172.21.15.31:0/2392420203 conn(0x5620ff33c200 msgr2=0x5620ff331ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 shutdown_connections 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 1 -- 172.21.15.31:0/2392420203 wait complete. 2023-12-14T18:33:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.106+0000 7f17d3273b80 20 client.16773 trim_cache size 0 max 16384 2023-12-14T18:33:28.115 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:33:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17ac0a12c0 msgr2=0x7f17ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17ac0a12c0 0x7f17ac0a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f17c01979b0 tx=0x7f17a8009260 comp rx=0 tx=0).stop 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 msgr2=0x7f17c01963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c01963a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f17b000b580 tx=0x7f17b000bfd0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 shutdown_connections 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17c01043d0 0x7f17c019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17ac0a12c0 0x7f17ac0a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c00ff390 0x7f17c01963a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 --2- 172.21.15.31:0/3921937758 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17c00ffd30 0x7f17c01968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 >> 172.21.15.31:0/3921937758 conn(0x7f17c0075fd0 msgr2=0x7f17c0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 shutdown_connections 2023-12-14T18:33:28.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.107+0000 7f17c70e5700 1 -- 172.21.15.31:0/3921937758 wait complete. 2023-12-14T18:33:28.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:33:28.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:28 smithi031 ceph-mon[42085]: pgmap v172: 129 pgs: 129 active+clean; 533 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 281 op/s 2023-12-14T18:33:28.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:28 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1610857068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:33:28.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 -- 172.21.15.31:0/3214247661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 msgr2=0x7f3f3c0ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 --2- 172.21.15.31:0/3214247661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c0ff760 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0099c0 tx=0x7f3f2c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 -- 172.21.15.31:0/3214247661 shutdown_connections 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 --2- 172.21.15.31:0/3214247661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c1043d0 0x7f3f3c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 --2- 172.21.15.31:0/3214247661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c0ff760 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 --2- 172.21.15.31:0/3214247661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 -- 172.21.15.31:0/3214247661 >> 172.21.15.31:0/3214247661 conn(0x7f3f3c075fd0 msgr2=0x7f3f3c0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 -- 172.21.15.31:0/3214247661 shutdown_connections 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.265+0000 7f3f41ced700 1 -- 172.21.15.31:0/3214247661 wait complete. 2023-12-14T18:33:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 Processor -- start 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 -- start start 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f3c1043d0 0x7f3f3c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c19b410 con 0x7f3f3c0ff390 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c19b580 con 0x7f3f3c1043d0 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f41ced700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c19b740 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:46978/0 (socket says 172.21.15.31:46978) 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f3affd700 1 -- 172.21.15.31:0/510501905 learned_addr learned my addr 172.21.15.31:0/510501905 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56868/0 (socket says 172.21.15.31:56868) 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.266+0000 7f3f3bfff700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f3c1043d0 0x7f3f3c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3affd700 1 -- 172.21.15.31:0/510501905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f3c1043d0 msgr2=0x7f3f3c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3affd700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f3c1043d0 0x7f3f3c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3affd700 1 -- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 msgr2=0x7f3f3c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3affd700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3affd700 1 -- 172.21.15.31:0/510501905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f30009520 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f3affd700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c196970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009fd0 tx=0x7f3f3000ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f38ff9700 1 -- 172.21.15.31:0/510501905 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000cbd0 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f38ff9700 1 -- 172.21.15.31:0/510501905 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3000eea0 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2c0095f0 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f38ff9700 1 -- 172.21.15.31:0/510501905 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30003cf0 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.267+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c10f1e0 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.270+0000 7f3f38ff9700 1 -- 172.21.15.31:0/510501905 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3f30003eb0 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.271+0000 7f3f38ff9700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f240a12c0 0x7f3f240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.271+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f240a12c0 0x7f3f240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.271+0000 7f3f38ff9700 1 -- 172.21.15.31:0/510501905 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f30014070 con 0x7f3f3c0ffd30 2023-12-14T18:33:28.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.271+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f240a12c0 0x7f3f240a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009990 tx=0x7f3f2c01a040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 Processor -- start 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 -- start start 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329dde750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 0x562329df0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329cc85e0 con 0x562329dedf60 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329d37870 con 0x562329dde380 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329d30d00 con 0x562329ddec90 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 0x562329df0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 0x562329df0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56872/0 (socket says 172.21.15.31:56872) 2023-12-14T18:33:28.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f3bfff700 1 -- 172.21.15.31:0/1672963455 learned_addr learned my addr 172.21.15.31:0/1672963455 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.272+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329dde750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3affd700 1 -- 172.21.15.31:0/1672963455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 msgr2=0x562329dde750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3affd700 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329dde750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3affd700 1 -- 172.21.15.31:0/1672963455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 msgr2=0x562329df0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3affd700 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 0x562329df0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3affd700 1 -- 172.21.15.31:0/1672963455 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562329bb23b0 con 0x562329ddec90 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3bfff700 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 0x562329df0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f3affd700 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deda20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ca6e0 tx=0x7f3f300098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a66e4efc0e5080c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f21ffb700 1 -- 172.21.15.31:0/1672963455 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300cd160 con 0x562329ddec90 2023-12-14T18:33:28.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f21ffb700 1 -- 172.21.15.31:0/1672963455 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3009e090 con 0x562329ddec90 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.273+0000 7f3f21ffb700 1 -- 172.21.15.31:0/1672963455 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3009a520 con 0x562329ddec90 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1672963455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 msgr2=0x562329deda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deda20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3f300ca6e0 tx=0x7f3f300098e0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1672963455 shutdown_connections 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dedf60 0x562329df0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329dde750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1672963455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deda20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1672963455 >> 172.21.15.31:0/1672963455 conn(0x562329dd4ca0 msgr2=0x562329dda7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1672963455 shutdown_connections 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1672963455 wait complete. 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.274+0000 7f3f4de7bb80 1 -- start start 2023-12-14T18:33:28.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 Processor -- start 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 0x562329ded9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dedf60 0x562329deb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329cccf90 con 0x562329dde380 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329cccc30 con 0x562329dedf60 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329bb46a0 con 0x562329ddec90 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f4de7bb80 10 client.0 _ll_register_callbacks cb 0x562329cf0910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 0x562329ded9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 0x562329ded9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56884/0 (socket says 172.21.15.31:56884) 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/4084855443 learned_addr learned my addr 172.21.15.31:0/4084855443 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.275+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3bfff700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dedf60 0x562329deb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/4084855443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 msgr2=0x562329deaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dedf60 msgr2=0x562329deb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dedf60 0x562329deb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/4084855443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c10f1e0 con 0x562329dde380 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3affd700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 0x562329ded9f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009fd0 tx=0x7f3f2c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5a4e02a6093967 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f1affd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f1affd700 1 -- 172.21.15.31:0/4084855443 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c02c040 con 0x562329dde380 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f1affd700 1 -- 172.21.15.31:0/4084855443 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f2c004e20 con 0x562329dde380 2023-12-14T18:33:28.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562329dd22b0 con 0x562329dde380 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f1affd700 1 -- 172.21.15.31:0/4084855443 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f2c00f460 con 0x562329dde380 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f4de7bb80 10 client.16785 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.276+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562329cf7170 con 0x562329dde380 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f1affd700 1 -- 172.21.15.31:0/4084855443 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562329dd22b0 con 0x562329dde380 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f1affd700 1 -- 172.21.15.31:0/4084855443 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3f2c030650 con 0x562329dde380 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f4de7bb80 10 client.16785 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f4de7bb80 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f4de7bb80 20 client.16785 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dfda00 0x562329dffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f4de7bb80 4 client.16785 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:28.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562329d29ce0 con 0x562329dfda00 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.277+0000 7f3f3affd700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dfda00 0x562329dffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f3affd700 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dfda00 0x562329dffe10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73e4c2e88fdf0fd4 server_cookie=6d42bf9ebd5b37a2 in_seq=0 out_seq=0 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f1affd700 10 client.16785 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f1affd700 1 -- 172.21.15.31:0/4084855443 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562329d29ce0 con 0x562329dfda00 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f1affd700 10 client.16785 handle_command_reply: tid=0 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 client.16785 shutdown 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 msgr2=0x562329ded9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 0x562329ded9f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009fd0 tx=0x7f3f2c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 shutdown_connections 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dedf60 0x562329deb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dde380 0x562329ded9f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dfda00 0x562329dffe10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/4084855443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329deaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.278+0000 7f3f1affd700 0 client.16785 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.279+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 >> 172.21.15.31:0/4084855443 conn(0x562329dd4ca0 msgr2=0x562329dda3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.279+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 shutdown_connections 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.279+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/4084855443 wait complete. 2023-12-14T18:33:28.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.279+0000 7f3f4de7bb80 20 client.16785 trim_cache size 0 max 16384 2023-12-14T18:33:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 Processor -- start 2023-12-14T18:33:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 -- start start 2023-12-14T18:33:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 0x562329df4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329ddec90 0x562329ded4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329dfa380 con 0x562329ddec90 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329de6b30 con 0x562329dde380 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329df4fe0 con 0x562329dd4ca0 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 0x562329df4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329ddec90 0x562329ded4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 0x562329df4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47010/0 (socket says 172.21.15.31:47010) 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/1180419534 learned_addr learned my addr 172.21.15.31:0/1180419534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.293+0000 7f3f3affd700 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/1180419534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 msgr2=0x562329df9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/1180419534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329ddec90 msgr2=0x562329ded4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329ddec90 0x562329ded4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f3b7fe700 1 -- 172.21.15.31:0/1180419534 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562329cf6ec0 con 0x562329dd4ca0 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 0x562329df4aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098e0 tx=0x7f3f2c005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99b7bea975e930b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f227fc700 1 -- 172.21.15.31:0/1180419534 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c02c050 con 0x562329dd4ca0 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f227fc700 1 -- 172.21.15.31:0/1180419534 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f2c01e040 con 0x562329dd4ca0 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.294+0000 7f3f227fc700 1 -- 172.21.15.31:0/1180419534 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c040790 con 0x562329dd4ca0 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1180419534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 msgr2=0x562329df4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 0x562329df4aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c0098e0 tx=0x7f3f2c005cc0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1180419534 shutdown_connections 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329ddec90 0x562329ded4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329dd4ca0 0x562329df4aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/1180419534 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1180419534 >> 172.21.15.31:0/1180419534 conn(0x562329df0610 msgr2=0x562329df9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1180419534 shutdown_connections 2023-12-14T18:33:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/1180419534 wait complete. 2023-12-14T18:33:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.295+0000 7f3f4de7bb80 1 -- start start 2023-12-14T18:33:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 Processor -- start 2023-12-14T18:33:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dd4ca0 0x562329df2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329df3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329e509f0 con 0x562329dd4ca0 2023-12-14T18:33:28.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329e50b60 con 0x562329dde380 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562329e50d20 con 0x562329ddec90 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f4de7bb80 10 client.0 _ll_register_callbacks cb 0x7f3f2c0048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dd4ca0 0x562329df2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329df3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329df3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47018/0 (socket says 172.21.15.31:47018) 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dd4ca0 0x562329df2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56898/0 (socket says 172.21.15.31:56898) 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3affd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:35140/0 (socket says 172.21.15.31:35140) 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 -- 172.21.15.31:0/2151182550 learned_addr learned my addr 172.21.15.31:0/2151182550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 -- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 msgr2=0x562329df2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 -- 172.21.15.31:0/2151182550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dd4ca0 msgr2=0x562329df2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.296+0000 7f3f3bfff700 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dd4ca0 0x562329df2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f3bfff700 1 -- 172.21.15.31:0/2151182550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562329cf7170 con 0x562329ddec90 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f3bfff700 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329df3500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f34004610 tx=0x7f3f34004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70f7df9d7f4213b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f1affd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f1affd700 1 -- 172.21.15.31:0/2151182550 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3401c070 con 0x562329ddec90 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f1affd700 1 -- 172.21.15.31:0/2151182550 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3f34007b00 con 0x562329ddec90 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f3009e090 con 0x562329ddec90 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f1affd700 1 -- 172.21.15.31:0/2151182550 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f3400e040 con 0x562329ddec90 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f4de7bb80 10 client.26632 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562329cdd9a0 con 0x562329ddec90 2023-12-14T18:33:28.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f1affd700 1 -- 172.21.15.31:0/2151182550 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f3009e090 con 0x562329ddec90 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f1affd700 1 -- 172.21.15.31:0/2151182550 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3f34017410 con 0x562329ddec90 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f4de7bb80 10 client.26632 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.297+0000 7f3f4de7bb80 10 client.26632 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f4de7bb80 20 client.26632 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dedf60 0x562329e545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f4de7bb80 4 client.26632 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562329d29a00 con 0x562329dedf60 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dedf60 0x562329e545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f1affd700 10 client.26632 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f3b7fe700 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dedf60 0x562329e545e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbe6e1c7b64a0dde server_cookie=24fa742a59d7bb5e in_seq=0 out_seq=0 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f1affd700 1 -- 172.21.15.31:0/2151182550 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562329d29a00 con 0x562329dedf60 2023-12-14T18:33:28.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f1affd700 10 client.26632 handle_command_reply: tid=0 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.298+0000 7f3f4de7bb80 1 client.26632 shutdown 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 msgr2=0x562329df3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329df3500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f34004610 tx=0x7f3f34004920 comp rx=0 tx=0).stop 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 shutdown_connections 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562329dedf60 0x562329e545e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562329ddec90 0x562329df3500 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562329dd4ca0 0x562329df2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f1affd700 0 client.26632 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 --2- 172.21.15.31:0/2151182550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562329dde380 0x562329df2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 >> 172.21.15.31:0/2151182550 conn(0x562329df0610 msgr2=0x562329df9240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 shutdown_connections 2023-12-14T18:33:28.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 1 -- 172.21.15.31:0/2151182550 wait complete. 2023-12-14T18:33:28.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f4de7bb80 20 client.26632 trim_cache size 0 max 16384 2023-12-14T18:33:28.308 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f240a12c0 msgr2=0x7f3f240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f240a12c0 0x7f3f240a3770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009990 tx=0x7f3f2c01a040 comp rx=0 tx=0).stop 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 msgr2=0x7f3f3c196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c196970 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009fd0 tx=0x7f3f3000ed10 comp rx=0 tx=0).stop 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 shutdown_connections 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f3c1043d0 0x7f3f3c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f3c0ff390 0x7f3f3c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3f240a12c0 0x7f3f240a3770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 --2- 172.21.15.31:0/510501905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f3c0ffd30 0x7f3f3c196970 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.299+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 >> 172.21.15.31:0/510501905 conn(0x7f3f3c075fd0 msgr2=0x7f3f3c067e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.300+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 shutdown_connections 2023-12-14T18:33:28.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.300+0000 7f3f41ced700 1 -- 172.21.15.31:0/510501905 wait complete. 2023-12-14T18:33:28.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:33:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:28 smithi204 ceph-mon[43866]: pgmap v172: 129 pgs: 129 active+clean; 533 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 281 op/s 2023-12-14T18:33:28.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:28 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1610857068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 -- 172.21.15.31:0/1987917360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 msgr2=0x7fddec10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/1987917360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec10bfa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdde400b0d0 tx=0x7fdde400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 -- 172.21.15.31:0/1987917360 shutdown_connections 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/1987917360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec10bfa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/1987917360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/1987917360 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fddec0ffd30 0x7fddec103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 -- 172.21.15.31:0/1987917360 >> 172.21.15.31:0/1987917360 conn(0x7fddec075fd0 msgr2=0x7fddec0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 -- 172.21.15.31:0/1987917360 shutdown_connections 2023-12-14T18:33:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.458+0000 7fddf3ef8700 1 -- 172.21.15.31:0/1987917360 wait complete. 2023-12-14T18:33:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 Processor -- start 2023-12-14T18:33:28.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 -- start start 2023-12-14T18:33:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fddec0ffd30 0x7fddec1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec101b00 con 0x7fddec1043d0 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1a4bd0 con 0x7fddec0ffd30 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf3ef8700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1a4d10 con 0x7fddec0ff390 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47032/0 (socket says 172.21.15.31:47032) 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf1c94700 1 -- 172.21.15.31:0/2218530773 learned_addr learned my addr 172.21.15.31:0/2218530773 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf2495700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.459+0000 7fddf1493700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fddec0ffd30 0x7fddec1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf2495700 1 -- 172.21.15.31:0/2218530773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 msgr2=0x7fddec100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf2495700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf2495700 1 -- 172.21.15.31:0/2218530773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fddec0ffd30 msgr2=0x7fddec1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf2495700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fddec0ffd30 0x7fddec1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf2495700 1 -- 172.21.15.31:0/2218530773 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4009d00 con 0x7fddec1043d0 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf1c94700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf2495700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec1a4690 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdde400b770 tx=0x7fdde4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fdde2ffd700 1 -- 172.21.15.31:0/2218530773 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde400e040 con 0x7fddec1043d0 2023-12-14T18:33:28.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fdde2ffd700 1 -- 172.21.15.31:0/2218530773 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde401d930 con 0x7fddec1043d0 2023-12-14T18:33:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec10f1a0 con 0x7fddec1043d0 2023-12-14T18:33:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fdde2ffd700 1 -- 172.21.15.31:0/2218530773 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde401b820 con 0x7fddec1043d0 2023-12-14T18:33:28.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.460+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec10f010 con 0x7fddec1043d0 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.463+0000 7fdde2ffd700 1 -- 172.21.15.31:0/2218530773 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdde401d470 con 0x7fddec1043d0 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.463+0000 7fdde2ffd700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fddd80a12c0 0x7fddd80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.464+0000 7fddf1c94700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fddd80a12c0 0x7fddd80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.464+0000 7fdde2ffd700 1 -- 172.21.15.31:0/2218530773 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fdde40ce450 con 0x7fddec1043d0 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.464+0000 7fddf1c94700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fddd80a12c0 0x7fddd80a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdde8011550 tx=0x7fdde8009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.464+0000 7fde00086b80 1 Processor -- start 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 -- start start 2023-12-14T18:33:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde166750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde173f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde0505e0 con 0x55bfde171b50 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde0bf870 con 0x55bfde166c90 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fde00086b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde0b8d00 con 0x55bfde166380 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf2495700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde173f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf2495700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde173f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56926/0 (socket says 172.21.15.31:56926) 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf2495700 1 -- 172.21.15.31:0/3124525541 learned_addr learned my addr 172.21.15.31:0/3124525541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1493700 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1c94700 1 -- 172.21.15.31:0/3124525541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 msgr2=0x55bfde171610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1c94700 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1c94700 1 -- 172.21.15.31:0/3124525541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 msgr2=0x55bfde173f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1c94700 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde173f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1c94700 1 -- 172.21.15.31:0/3124525541 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfddf3a3b0 con 0x55bfde166380 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf1493700 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.465+0000 7fddf2495700 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde173f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.466+0000 7fddf1c94700 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde166750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdde8011ca0 tx=0x7fdde8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b01106319bfbcd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.466+0000 7fddd7fff700 1 -- 172.21.15.31:0/3124525541 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8004830 con 0x55bfde166380 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.466+0000 7fddd7fff700 1 -- 172.21.15.31:0/3124525541 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde8021070 con 0x55bfde166380 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.466+0000 7fddd7fff700 1 -- 172.21.15.31:0/3124525541 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde801c610 con 0x55bfde166380 2023-12-14T18:33:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 -- 172.21.15.31:0/3124525541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 msgr2=0x55bfde166750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde166750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdde8011ca0 tx=0x7fdde8014000 comp rx=0 tx=0).stop 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 -- 172.21.15.31:0/3124525541 shutdown_connections 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde173f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde166750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 --2- 172.21.15.31:0/3124525541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 -- 172.21.15.31:0/3124525541 >> 172.21.15.31:0/3124525541 conn(0x55bfde15cca0 msgr2=0x55bfde1627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 -- 172.21.15.31:0/3124525541 shutdown_connections 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 -- 172.21.15.31:0/3124525541 wait complete. 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 -- start start 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.467+0000 7fde00086b80 1 Processor -- start 2023-12-14T18:33:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde1715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde16e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde16efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde054f90 con 0x55bfde171b50 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde054c30 con 0x55bfde166c90 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfddf3c6a0 con 0x55bfde166380 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fde00086b80 10 client.0 _ll_register_callbacks cb 0x55bfde078910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf2495700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde16efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde1715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde1715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47056/0 (socket says 172.21.15.31:47056) 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 -- 172.21.15.31:0/2199517923 learned_addr learned my addr 172.21.15.31:0/2199517923 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf2495700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde16efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56938/0 (socket says 172.21.15.31:56938) 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1493700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde16e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 -- 172.21.15.31:0/2199517923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 msgr2=0x55bfde16e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde16e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 -- 172.21.15.31:0/2199517923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 msgr2=0x55bfde16efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde16efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 -- 172.21.15.31:0/2199517923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde8017070 con 0x55bfde166380 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf2495700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde16efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddf1c94700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde1715b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdde8005720 tx=0x7fdde8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=346c300403d8a63a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.468+0000 7fddd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fddd57fa700 1 -- 172.21.15.31:0/2199517923 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8026890 con 0x55bfde166380 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fddd57fa700 1 -- 172.21.15.31:0/2199517923 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fdde80269f0 con 0x55bfde166380 2023-12-14T18:33:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfde15a2b0 con 0x55bfde166380 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fddd57fa700 1 -- 172.21.15.31:0/2199517923 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde800ee90 con 0x55bfde166380 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 10 client.26644 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55bfde07f170 con 0x55bfde166380 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fddd57fa700 1 -- 172.21.15.31:0/2199517923 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfde15a2b0 con 0x55bfde166380 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fddd57fa700 1 -- 172.21.15.31:0/2199517923 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdde801d410 con 0x55bfde166380 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 10 client.26644 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 10 client.26644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 20 client.26644 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde181460 0x55bfde183870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 4 client.26644 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.469+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfde0b1ce0 con 0x55bfde181460 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.470+0000 7fddf1493700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde181460 0x55bfde183870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.470+0000 7fddd57fa700 10 client.26644 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.470+0000 7fddf1493700 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde181460 0x55bfde183870 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402cba40703ddeec server_cookie=48e34c28e1ae5037 in_seq=0 out_seq=0 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.470+0000 7fddd57fa700 1 -- 172.21.15.31:0/2199517923 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bfde0b1ce0 con 0x55bfde181460 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.470+0000 7fddd57fa700 10 client.26644 handle_command_reply: tid=0 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 client.26644 shutdown 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 msgr2=0x55bfde1715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde1715b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdde8005720 tx=0x7fdde8005750 comp rx=0 tx=0).stop 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 shutdown_connections 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde181460 0x55bfde183870 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde171b50 0x55bfde16efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde166380 0x55bfde1715b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fddd57fa700 0 client.26644 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 --2- 172.21.15.31:0/2199517923 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde16e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 >> 172.21.15.31:0/2199517923 conn(0x55bfde15cca0 msgr2=0x55bfde1623d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 shutdown_connections 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 1 -- 172.21.15.31:0/2199517923 wait complete. 2023-12-14T18:33:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.471+0000 7fde00086b80 20 client.26644 trim_cache size 0 max 16384 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 Processor -- start 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 -- start start 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 0x55bfde178690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 0x55bfde17ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde183ed0 con 0x55bfde166380 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde16a6f0 con 0x55bfde166c90 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.485+0000 7fde00086b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde167d30 con 0x55bfde15cca0 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 0x55bfde17ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 0x55bfde178690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 0x55bfde17ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56954/0 (socket says 172.21.15.31:56954) 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 -- 172.21.15.31:0/3468936260 learned_addr learned my addr 172.21.15.31:0/3468936260 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1c94700 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 0x55bfde178690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47060/0 (socket says 172.21.15.31:47060) 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf2495700 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 -- 172.21.15.31:0/3468936260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 msgr2=0x55bfde178690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 0x55bfde178690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 -- 172.21.15.31:0/3468936260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 msgr2=0x55bfde171070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 -- 172.21.15.31:0/3468936260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfde07eec0 con 0x55bfde166380 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1c94700 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 0x55bfde178690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fddf1493700 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 0x55bfde17ce30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fddec06a350 tx=0x7fdddc0073d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d26f012da1d8b3c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fdde0ff9700 1 -- 172.21.15.31:0/3468936260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc016070 con 0x55bfde166380 2023-12-14T18:33:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fdde0ff9700 1 -- 172.21.15.31:0/3468936260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdddc017040 con 0x55bfde166380 2023-12-14T18:33:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.486+0000 7fdde0ff9700 1 -- 172.21.15.31:0/3468936260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc004bb0 con 0x55bfde166380 2023-12-14T18:33:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 -- 172.21.15.31:0/3468936260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 msgr2=0x55bfde17ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 0x55bfde17ce30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fddec06a350 tx=0x7fdddc0073d0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 -- 172.21.15.31:0/3468936260 shutdown_connections 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde171070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde15cca0 0x55bfde178690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 --2- 172.21.15.31:0/3468936260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde166380 0x55bfde17ce30 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 -- 172.21.15.31:0/3468936260 >> 172.21.15.31:0/3468936260 conn(0x55bfde174200 msgr2=0x55bfde17d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 -- 172.21.15.31:0/3468936260 shutdown_connections 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 -- 172.21.15.31:0/3468936260 wait complete. 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.487+0000 7fde00086b80 1 -- start start 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 Processor -- start 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 0x55bfde177f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde178470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde171b50 0x55bfde176670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde176d80 con 0x55bfde15cca0 2023-12-14T18:33:28.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde176ef0 con 0x55bfde166c90 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfde1770b0 con 0x55bfde171b50 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fde00086b80 10 client.0 _ll_register_callbacks cb 0x7fdde4097500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 0x55bfde177f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fddf1c94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 0x55bfde177f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56958/0 (socket says 172.21.15.31:56958) 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fddf1c94700 1 -- 172.21.15.31:0/16307596 learned_addr learned my addr 172.21.15.31:0/16307596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fddf2495700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde171b50 0x55bfde176670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.488+0000 7fddf1493700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde178470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf1c94700 1 -- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde171b50 msgr2=0x55bfde176670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf1c94700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde171b50 0x55bfde176670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf1c94700 1 -- 172.21.15.31:0/16307596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 msgr2=0x55bfde178470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf1c94700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde178470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf1c94700 1 -- 172.21.15.31:0/16307596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfde07f170 con 0x55bfde15cca0 2023-12-14T18:33:28.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf2495700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde171b50 0x55bfde176670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddf1c94700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 0x55bfde177f30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdde8011ca0 tx=0x7fdde800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7cc54d8b0977d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddd57fa700 1 -- 172.21.15.31:0/16307596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde801da90 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdde8021070 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddd57fa700 1 -- 172.21.15.31:0/16307596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fdde80268a0 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddd57fa700 1 -- 172.21.15.31:0/16307596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdde8026a60 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fde00086b80 10 client.16809 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fdddc007e30 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.489+0000 7fddd57fa700 1 -- 172.21.15.31:0/16307596 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdde8021070 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fddd57fa700 1 -- 172.21.15.31:0/16307596 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdde8045070 con 0x55bfde15cca0 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fde00086b80 10 client.16809 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:28.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fde00086b80 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fde00086b80 20 client.16809 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fde00086b80 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde1d60b0 0x55bfde1d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fde00086b80 4 client.16809 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfde0b1a00 con 0x55bfde1d60b0 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fddf1493700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde1d60b0 0x55bfde1d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fddd57fa700 10 client.16809 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fddf1493700 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde1d60b0 0x55bfde1d84c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b744232057ae9212 server_cookie=9a884f706c11ff5 in_seq=0 out_seq=0 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.490+0000 7fddd57fa700 1 -- 172.21.15.31:0/16307596 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfde0b1a00 con 0x55bfde1d60b0 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fddd57fa700 10 client.16809 handle_command_reply: tid=0 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 client.16809 shutdown 2023-12-14T18:33:28.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 msgr2=0x55bfde177f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 0x55bfde177f30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdde8011ca0 tx=0x7fdde800ee70 comp rx=0 tx=0).stop 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 shutdown_connections 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bfde1d60b0 0x55bfde1d84c0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bfde171b50 0x55bfde176670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bfde15cca0 0x55bfde177f30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fddd57fa700 0 client.16809 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 --2- 172.21.15.31:0/16307596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bfde166c90 0x55bfde178470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 >> 172.21.15.31:0/16307596 conn(0x55bfde174200 msgr2=0x55bfde17ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 shutdown_connections 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 1 -- 172.21.15.31:0/16307596 wait complete. 2023-12-14T18:33:28.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.491+0000 7fde00086b80 20 client.16809 trim_cache size 0 max 16384 2023-12-14T18:33:28.500 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:33:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fddd80a12c0 msgr2=0x7fddd80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fddd80a12c0 0x7fddd80a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdde8011550 tx=0x7fdde8009260 comp rx=0 tx=0).stop 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 msgr2=0x7fddec1a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec1a4690 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdde400b770 tx=0x7fdde4007870 comp rx=0 tx=0).stop 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 shutdown_connections 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec1043d0 0x7fddec1a4690 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fddd80a12c0 0x7fddd80a3770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddec0ff390 0x7fddec100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 --2- 172.21.15.31:0/2218530773 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fddec0ffd30 0x7fddec1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 >> 172.21.15.31:0/2218530773 conn(0x7fddec075fd0 msgr2=0x7fddec067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 shutdown_connections 2023-12-14T18:33:28.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:28.492+0000 7fddf3ef8700 1 -- 172.21.15.31:0/2218530773 wait complete. 2023-12-14T18:33:28.502 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:28 smithi176 ceph-mon[43506]: pgmap v172: 129 pgs: 129 active+clean; 533 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 281 op/s 2023-12-14T18:33:28.502 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:28 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1610857068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:33:29.507 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:33:29.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 -- 172.21.15.31:0/2510055509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4880fefb0 msgr2=0x7fd4880ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 --2- 172.21.15.31:0/2510055509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4880fefb0 0x7fd4880ff380 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fd484009990 tx=0x7fd484009ca0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 -- 172.21.15.31:0/2510055509 shutdown_connections 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 --2- 172.21.15.31:0/2510055509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd48810cb60 0x7fd48810ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 --2- 172.21.15.31:0/2510055509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4880fefb0 0x7fd4880ff380 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 --2- 172.21.15.31:0/2510055509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4880ff8c0 0x7fd48810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 -- 172.21.15.31:0/2510055509 >> 172.21.15.31:0/2510055509 conn(0x7fd4880763a0 msgr2=0x7fd4880767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 -- 172.21.15.31:0/2510055509 shutdown_connections 2023-12-14T18:33:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.672+0000 7fd48fae2700 1 -- 172.21.15.31:0/2510055509 wait complete. 2023-12-14T18:33:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 Processor -- start 2023-12-14T18:33:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 -- start start 2023-12-14T18:33:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4880fefb0 0x7fd4881985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4880ff8c0 0x7fd488198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 0x7fd48819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819d5b0 con 0x7fd48810cb60 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819d720 con 0x7fd4880fefb0 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.673+0000 7fd48fae2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48819d8e0 con 0x7fd4880ff8c0 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48e07f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 0x7fd48819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d07d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4880ff8c0 0x7fd488198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4880fefb0 0x7fd4881985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48e07f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 0x7fd48819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56970/0 (socket says 172.21.15.31:56970) 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48e07f700 1 -- 172.21.15.31:0/727024495 learned_addr learned my addr 172.21.15.31:0/727024495 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:29.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 -- 172.21.15.31:0/727024495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4880ff8c0 msgr2=0x7fd488198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4880ff8c0 0x7fd488198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 -- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 msgr2=0x7fd48819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 0x7fd48819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 -- 172.21.15.31:0/727024495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840095f0 con 0x7fd4880fefb0 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48e07f700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 0x7fd48819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48d87e700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4880fefb0 0x7fd4881985d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd484000c00 tx=0x7fd484005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd47effd700 1 -- 172.21.15.31:0/727024495 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48401d070 con 0x7fd4880fefb0 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488112510 con 0x7fd4880fefb0 2023-12-14T18:33:29.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd47effd700 1 -- 172.21.15.31:0/727024495 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd484021410 con 0x7fd4880fefb0 2023-12-14T18:33:29.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd47effd700 1 -- 172.21.15.31:0/727024495 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484017650 con 0x7fd4880fefb0 2023-12-14T18:33:29.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.674+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd488112380 con 0x7fd4880fefb0 2023-12-14T18:33:29.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.677+0000 7fd47effd700 1 -- 172.21.15.31:0/727024495 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd4840217f0 con 0x7fd4880fefb0 2023-12-14T18:33:29.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.677+0000 7fd47effd700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4740a12c0 0x7fd4740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.677+0000 7fd47effd700 1 -- 172.21.15.31:0/727024495 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd4840ce0c0 con 0x7fd4880fefb0 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.677+0000 7fd48d07d700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4740a12c0 0x7fd4740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.678+0000 7fd48d07d700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4740a12c0 0x7fd4740a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd488199be0 tx=0x7fd478009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 Processor -- start 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 -- start start 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 0x55b145f217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21d10 0x55b145f2c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f2efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145e0b660 con 0x55b145f21400 2023-12-14T18:33:29.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145e7a8f0 con 0x55b145f2cbd0 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145e73d80 con 0x55b145f21d10 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 0x55b145f217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 0x55b145f217d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56982/0 (socket says 172.21.15.31:56982) 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd48d07d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21d10 0x55b145f2c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.679+0000 7fd48d87e700 1 -- 172.21.15.31:0/995728520 learned_addr learned my addr 172.21.15.31:0/995728520 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48e07f700 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f2efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d07d700 1 -- 172.21.15.31:0/995728520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 msgr2=0x55b145f2efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d07d700 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f2efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d07d700 1 -- 172.21.15.31:0/995728520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 msgr2=0x55b145f217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d07d700 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 0x55b145f217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d07d700 1 -- 172.21.15.31:0/995728520 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b145cf5430 con 0x55b145f21d10 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d87e700 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 0x55b145f217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd48d07d700 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21d10 0x55b145f2c690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd478011c90 tx=0x7fd478014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2439c3fe07fe49e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd473fff700 1 -- 172.21.15.31:0/995728520 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478004230 con 0x55b145f21d10 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd473fff700 1 -- 172.21.15.31:0/995728520 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd478021070 con 0x55b145f21d10 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.680+0000 7fd473fff700 1 -- 172.21.15.31:0/995728520 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478003b20 con 0x55b145f21d10 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 -- 172.21.15.31:0/995728520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21d10 msgr2=0x55b145f2c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21d10 0x55b145f2c690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd478011c90 tx=0x7fd478014000 comp rx=0 tx=0).stop 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 -- 172.21.15.31:0/995728520 shutdown_connections 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f2efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21400 0x55b145f217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/995728520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21d10 0x55b145f2c690 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 -- 172.21.15.31:0/995728520 >> 172.21.15.31:0/995728520 conn(0x55b145f17d20 msgr2=0x55b145f1d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 -- 172.21.15.31:0/995728520 shutdown_connections 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.681+0000 7fd49bc70b80 1 -- 172.21.15.31:0/995728520 wait complete. 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 -- start start 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 Processor -- start 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 0x55b145f298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f29f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145e10010 con 0x55b145f21d10 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145e0fcb0 con 0x55b145f2cbd0 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145cf7720 con 0x55b145f21400 2023-12-14T18:33:29.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd49bc70b80 10 client.0 _ll_register_callbacks cb 0x55b145e33990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd48d07d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 0x55b145f298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47128/0 (socket says 172.21.15.31:47128) 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd48d07d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 0x55b145f298f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56984/0 (socket says 172.21.15.31:56984) 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.682+0000 7fd48d87e700 1 -- 172.21.15.31:0/3147434212 learned_addr learned my addr 172.21.15.31:0/3147434212 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48e07f700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f29f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d87e700 1 -- 172.21.15.31:0/3147434212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 msgr2=0x55b145f29f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f29f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d87e700 1 -- 172.21.15.31:0/3147434212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 msgr2=0x55b145f298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 0x55b145f298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d87e700 1 -- 172.21.15.31:0/3147434212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd478017070 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 0x55b145f298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48e07f700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f29f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c5d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd484097a20 tx=0x7fd48409aaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132026e37b31fd88 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd4717fa700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd4717fa700 1 -- 172.21.15.31:0/3147434212 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48409a6a0 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd4717fa700 1 -- 172.21.15.31:0/3147434212 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd48409afb0 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b145f15330 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd4717fa700 1 -- 172.21.15.31:0/3147434212 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd48409efe0 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd49bc70b80 10 client.26662 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.683+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b145e3a1f0 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd4717fa700 1 -- 172.21.15.31:0/3147434212 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b145f15330 con 0x55b145f21400 2023-12-14T18:33:29.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd4717fa700 1 -- 172.21.15.31:0/3147434212 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd4840cbd20 con 0x55b145f21400 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd49bc70b80 10 client.26662 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd49bc70b80 10 client.26662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd49bc70b80 20 client.26662 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f3c280 0x55b145f3e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd49bc70b80 4 client.26662 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b145e6cd60 con 0x55b145f3c280 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f3c280 0x55b145f3e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd4717fa700 10 client.26662 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.684+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f3c280 0x55b145f3e660 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cee70fd78be93f83 server_cookie=8d67bf7976fae236 in_seq=0 out_seq=0 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd4717fa700 1 -- 172.21.15.31:0/3147434212 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b145e6cd60 con 0x55b145f3c280 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd4717fa700 10 client.26662 handle_command_reply: tid=0 2023-12-14T18:33:29.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 client.26662 shutdown 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 msgr2=0x55b145f2c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c5d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd484097a20 tx=0x7fd48409aaf0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 shutdown_connections 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f3c280 0x55b145f3e660 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f2cbd0 0x55b145f29f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c5d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3147434212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f21d10 0x55b145f298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd4717fa700 0 client.26662 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 >> 172.21.15.31:0/3147434212 conn(0x55b145f17d20 msgr2=0x55b145f1d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 shutdown_connections 2023-12-14T18:33:29.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.685+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3147434212 wait complete. 2023-12-14T18:33:29.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.686+0000 7fd49bc70b80 20 client.26662 trim_cache size 0 max 16384 2023-12-14T18:33:29.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 Processor -- start 2023-12-14T18:33:29.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 -- start start 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145f17b30 con 0x55b145f17d20 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145f22410 con 0x55b145f21d10 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145f27e90 con 0x55b145f21400 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f298a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56990/0 (socket says 172.21.15.31:56990) 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd48d87e700 1 -- 172.21.15.31:0/3819657550 learned_addr learned my addr 172.21.15.31:0/3819657550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.704+0000 7fd48e07f700 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd48d87e700 1 -- 172.21.15.31:0/3819657550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 msgr2=0x55b145f2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd48d87e700 1 -- 172.21.15.31:0/3819657550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 msgr2=0x55b145f2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f2c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd48d87e700 1 -- 172.21.15.31:0/3819657550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b145e39f40 con 0x55b145f17d20 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f298a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd484099300 tx=0x7fd48400b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a346d5ac71e3e5f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd47cdac700 1 -- 172.21.15.31:0/3819657550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840a3170 con 0x55b145f17d20 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd47cdac700 1 -- 172.21.15.31:0/3819657550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd484021e90 con 0x55b145f17d20 2023-12-14T18:33:29.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.705+0000 7fd47cdac700 1 -- 172.21.15.31:0/3819657550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840b9360 con 0x55b145f17d20 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3819657550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 msgr2=0x55b145f298a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f298a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd484099300 tx=0x7fd48400b490 comp rx=0 tx=0).stop 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3819657550 shutdown_connections 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f298a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3819657550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3819657550 >> 172.21.15.31:0/3819657550 conn(0x55b145f2f280 msgr2=0x55b145f29190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3819657550 shutdown_connections 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3819657550 wait complete. 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 -- start start 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.706+0000 7fd49bc70b80 1 Processor -- start 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f31e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f32370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145f90120 con 0x55b145f17d20 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145f90260 con 0x55b145f21d10 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145f903d0 con 0x55b145f21400 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd49bc70b80 10 client.0 _ll_register_callbacks cb 0x7fd484096c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:29.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f31e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f31e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57004/0 (socket says 172.21.15.31:57004) 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 -- 172.21.15.31:0/3120731796 learned_addr learned my addr 172.21.15.31:0/3120731796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f32370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48e07f700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 -- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 msgr2=0x55b145f32370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f32370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 -- 172.21.15.31:0/3120731796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 msgr2=0x55b145f328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d87e700 1 -- 172.21.15.31:0/3120731796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b145e3a1f0 con 0x55b145f17d20 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48e07f700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.707+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f32370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:29.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd48d87e700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f31e30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd484009990 tx=0x7fd484099560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fb43577c1790dd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd4717fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd4717fa700 1 -- 172.21.15.31:0/3120731796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840a3170 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd4717fa700 1 -- 172.21.15.31:0/3120731796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fd48409afb0 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd478021070 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd4717fa700 1 -- 172.21.15.31:0/3120731796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd48409efe0 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd49bc70b80 10 client.16827 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b145e20a20 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.708+0000 7fd4717fa700 1 -- 172.21.15.31:0/3120731796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd478021070 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd4717fa700 1 -- 172.21.15.31:0/3120731796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd484095c50 con 0x55b145f17d20 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd49bc70b80 10 client.16827 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:29.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd49bc70b80 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd49bc70b80 20 client.16827 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f2cbd0 0x55b145f93670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd49bc70b80 4 client.16827 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b145e6ca80 con 0x55b145f2cbd0 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f2cbd0 0x55b145f93670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd4717fa700 10 client.16827 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.709+0000 7fd48d07d700 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f2cbd0 0x55b145f93670 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b68515c6dca8a10 server_cookie=7ecd05d691427f76 in_seq=0 out_seq=0 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd4717fa700 1 -- 172.21.15.31:0/3120731796 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b145e6ca80 con 0x55b145f2cbd0 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd4717fa700 10 client.16827 handle_command_reply: tid=0 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 client.16827 shutdown 2023-12-14T18:33:29.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 msgr2=0x55b145f31e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f31e30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd484009990 tx=0x7fd484099560 comp rx=0 tx=0).stop 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 shutdown_connections 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b145f2cbd0 0x55b145f93670 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b145f21d10 0x55b145f328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd4717fa700 0 client.16827 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b145f17d20 0x55b145f31e30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 --2- 172.21.15.31:0/3120731796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b145f21400 0x55b145f32370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 >> 172.21.15.31:0/3120731796 conn(0x55b145f2f280 msgr2=0x55b145f23b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 shutdown_connections 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 1 -- 172.21.15.31:0/3120731796 wait complete. 2023-12-14T18:33:29.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.710+0000 7fd49bc70b80 20 client.16827 trim_cache size 0 max 16384 2023-12-14T18:33:29.720 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:33:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:33:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "20a26b1f-c873-47dd-bed6-5463c1efeb50", 2023-12-14T18:33:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:33:29.720 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4740a12c0 msgr2=0x7fd4740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4740a12c0 0x7fd4740a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd488199be0 tx=0x7fd478009260 comp rx=0 tx=0).stop 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4880fefb0 msgr2=0x7fd4881985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4880fefb0 0x7fd4881985d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd484000c00 tx=0x7fd484005740 comp rx=0 tx=0).stop 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 shutdown_connections 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd48810cb60 0x7fd48819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4880fefb0 0x7fd4881985d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4740a12c0 0x7fd4740a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 --2- 172.21.15.31:0/727024495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4880ff8c0 0x7fd488198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 >> 172.21.15.31:0/727024495 conn(0x7fd4880763a0 msgr2=0x7fd4880fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 shutdown_connections 2023-12-14T18:33:29.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.711+0000 7fd48fae2700 1 -- 172.21.15.31:0/727024495 wait complete. 2023-12-14T18:33:29.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:33:29.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 -- 172.21.15.31:0/4092994085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b841004b0 msgr2=0x7f6b84100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:29.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 --2- 172.21.15.31:0/4092994085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b841004b0 0x7f6b84100880 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6b740099c0 tx=0x7f6b74009cd0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 -- 172.21.15.31:0/4092994085 shutdown_connections 2023-12-14T18:33:29.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 --2- 172.21.15.31:0/4092994085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b841054f0 0x7f6b8410d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 --2- 172.21.15.31:0/4092994085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b841004b0 0x7f6b84100880 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 --2- 172.21.15.31:0/4092994085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b84100e50 0x7f6b84104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:29.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.869+0000 7f6b891d9700 1 -- 172.21.15.31:0/4092994085 >> 172.21.15.31:0/4092994085 conn(0x7f6b840fbd40 msgr2=0x7f6b840fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:29.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.870+0000 7f6b891d9700 1 -- 172.21.15.31:0/4092994085 shutdown_connections 2023-12-14T18:33:29.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.870+0000 7f6b891d9700 1 -- 172.21.15.31:0/4092994085 wait complete. 2023-12-14T18:33:29.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.870+0000 7f6b891d9700 1 Processor -- start 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 -- start start 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b841004b0 0x7f6b84196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 0x7f6b84196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b841054f0 0x7f6b8419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8419b430 con 0x7f6b84100e50 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8419b5a0 con 0x7f6b841004b0 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b891d9700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8419b760 con 0x7f6b841054f0 2023-12-14T18:33:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b8259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 0x7f6b84196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b8259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 0x7f6b84196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57032/0 (socket says 172.21.15.31:57032) 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b8359e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b841054f0 0x7f6b8419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b841004b0 0x7f6b84196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.871+0000 7f6b8259c700 1 -- 172.21.15.31:0/154563531 learned_addr learned my addr 172.21.15.31:0/154563531 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b8259c700 1 -- 172.21.15.31:0/154563531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b841054f0 msgr2=0x7f6b8419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b8259c700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b841054f0 0x7f6b8419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b8259c700 1 -- 172.21.15.31:0/154563531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b841004b0 msgr2=0x7f6b84196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b8259c700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b841004b0 0x7f6b84196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b8259c700 1 -- 172.21.15.31:0/154563531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740095f0 con 0x7f6b84100e50 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b841004b0 0x7f6b84196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b8259c700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 0x7f6b84196990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800d950 tx=0x7f6b7800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b6bfff700 1 -- 172.21.15.31:0/154563531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800f890 con 0x7f6b84100e50 2023-12-14T18:33:30.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b6bfff700 1 -- 172.21.15.31:0/154563531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b78010460 con 0x7f6b84100e50 2023-12-14T18:33:30.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b84110300 con 0x7f6b84100e50 2023-12-14T18:33:30.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b6bfff700 1 -- 172.21.15.31:0/154563531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7800e610 con 0x7f6b84100e50 2023-12-14T18:33:30.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.872+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b84110170 con 0x7f6b84100e50 2023-12-14T18:33:30.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.875+0000 7f6b6bfff700 1 -- 172.21.15.31:0/154563531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6b780105d0 con 0x7f6b84100e50 2023-12-14T18:33:30.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.876+0000 7f6b6bfff700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6b6c0a1340 0x7f6b6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.876+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6b6c0a1340 0x7f6b6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.876+0000 7f6b6bfff700 1 -- 172.21.15.31:0/154563531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6b780ccb30 con 0x7f6b84100e50 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.876+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6b6c0a1340 0x7f6b6c0a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009990 tx=0x7f6b7401a040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 Processor -- start 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 -- start start 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x56032728a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x560327295610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560327295b50 0x560327297f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603271745e0 con 0x56032728a380 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603271e3870 con 0x56032728ac90 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b95367b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603271dcd00 con 0x560327295b50 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x56032728a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x56032728a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57042/0 (socket says 172.21.15.31:57042) 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b82d9d700 1 -- 172.21.15.31:0/1509135074 learned_addr learned my addr 172.21.15.31:0/1509135074 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b8359e700 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560327295b50 0x560327297f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.877+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x560327295610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8259c700 1 -- 172.21.15.31:0/1509135074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560327295b50 msgr2=0x560327297f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560327295b50 0x560327297f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8259c700 1 -- 172.21.15.31:0/1509135074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 msgr2=0x56032728a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x56032728a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8259c700 1 -- 172.21.15.31:0/1509135074 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56032705e3b0 con 0x56032728ac90 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8359e700 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560327295b50 0x560327297f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x560327295610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b780c9ef0 tx=0x7f6b78099870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d075d1248fe13 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b697fa700 1 -- 172.21.15.31:0/1509135074 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780b7ea0 con 0x56032728ac90 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b697fa700 1 -- 172.21.15.31:0/1509135074 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b780a1cc0 con 0x56032728ac90 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.878+0000 7f6b697fa700 1 -- 172.21.15.31:0/1509135074 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78098fc0 con 0x56032728ac90 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 -- 172.21.15.31:0/1509135074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 msgr2=0x560327295610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x560327295610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b780c9ef0 tx=0x7f6b78099870 comp rx=0 tx=0).stop 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 -- 172.21.15.31:0/1509135074 shutdown_connections 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560327295b50 0x560327297f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x56032728a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1509135074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x560327295610 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 -- 172.21.15.31:0/1509135074 >> 172.21.15.31:0/1509135074 conn(0x560327280ca0 msgr2=0x5603272867e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 -- 172.21.15.31:0/1509135074 shutdown_connections 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 -- 172.21.15.31:0/1509135074 wait complete. 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 -- start start 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.879+0000 7f6b95367b80 1 Processor -- start 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x560327295530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728ac90 0x560327292850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x560327292eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560327178f90 con 0x56032728a380 2023-12-14T18:33:30.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560327178c30 con 0x560327295b50 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603270606a0 con 0x56032728ac90 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x560327295530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b95367b80 10 client.0 _ll_register_callbacks cb 0x56032719c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x560327295530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57058/0 (socket says 172.21.15.31:57058) 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 -- 172.21.15.31:0/578837239 learned_addr learned my addr 172.21.15.31:0/578837239 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b8259c700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728ac90 0x560327292850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b8359e700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x560327292eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 -- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728ac90 msgr2=0x560327292850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728ac90 0x560327292850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 -- 172.21.15.31:0/578837239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 msgr2=0x560327292eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x560327292eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b82d9d700 1 -- 172.21.15.31:0/578837239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b84110170 con 0x56032728a380 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.880+0000 7f6b8359e700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x560327292eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x560327295530 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6b740058e0 tx=0x7f6b74009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72aeab9a884f680a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b66ff5700 1 -- 172.21.15.31:0/578837239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7402c040 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b66ff5700 1 -- 172.21.15.31:0/578837239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6b74004e30 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56032727e2b0 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b66ff5700 1 -- 172.21.15.31:0/578837239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b7400f460 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b95367b80 10 client.16839 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5603271a3170 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b66ff5700 1 -- 172.21.15.31:0/578837239 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56032727e2b0 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b66ff5700 1 -- 172.21.15.31:0/578837239 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6b74030790 con 0x56032728a380 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b95367b80 10 client.16839 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.881+0000 7f6b95367b80 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b95367b80 20 client.16839 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b95367b80 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272a98e0 0x5603272abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b95367b80 4 client.16839 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603271d5ce0 con 0x5603272a98e0 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b8259c700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272a98e0 0x5603272abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b66ff5700 10 client.16839 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.882+0000 7f6b8259c700 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272a98e0 0x5603272abcf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49c764dc5c7e998 server_cookie=9a02501533a8903c in_seq=0 out_seq=0 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.884+0000 7f6b66ff5700 1 -- 172.21.15.31:0/578837239 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5603271d5ce0 con 0x5603272a98e0 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b66ff5700 10 client.16839 handle_command_reply: tid=0 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 client.16839 shutdown 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 msgr2=0x560327295530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x560327295530 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6b740058e0 tx=0x7f6b74009cd0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 shutdown_connections 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272a98e0 0x5603272abcf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x560327292eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728a380 0x560327295530 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b66ff5700 0 client.16839 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 --2- 172.21.15.31:0/578837239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728ac90 0x560327292850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 >> 172.21.15.31:0/578837239 conn(0x560327280ca0 msgr2=0x5603272863d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 shutdown_connections 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 1 -- 172.21.15.31:0/578837239 wait complete. 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.885+0000 7f6b95367b80 20 client.16839 trim_cache size 0 max 16384 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.903+0000 7f6b95367b80 1 Processor -- start 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.903+0000 7f6b95367b80 1 -- start start 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.903+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x560327295060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.903+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 0x56032728db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.903+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x56032728e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b95367b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603272a5890 con 0x56032728ac90 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b95367b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560327290dd0 con 0x560327295b50 2023-12-14T18:33:30.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b95367b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032728e6e0 con 0x56032728a380 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 0x56032728db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 0x56032728db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57062/0 (socket says 172.21.15.31:57062) 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 -- 172.21.15.31:0/3591946770 learned_addr learned my addr 172.21.15.31:0/3591946770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x560327295060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8359e700 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x56032728e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 -- 172.21.15.31:0/3591946770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 msgr2=0x560327295060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x560327295060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 -- 172.21.15.31:0/3591946770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 msgr2=0x56032728e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x56032728e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 -- 172.21.15.31:0/3591946770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603271a2ec0 con 0x56032728ac90 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b8259c700 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 0x56032728db70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6b780c9fa0 tx=0x7f6b78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d08d01cc70d19f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b69ffb700 1 -- 172.21.15.31:0/3591946770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b780b7ea0 con 0x56032728ac90 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b69ffb700 1 -- 172.21.15.31:0/3591946770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b780108b0 con 0x56032728ac90 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.904+0000 7f6b69ffb700 1 -- 172.21.15.31:0/3591946770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78098fc0 con 0x56032728ac90 2023-12-14T18:33:30.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.905+0000 7f6b95367b80 1 -- 172.21.15.31:0/3591946770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 msgr2=0x56032728db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.905+0000 7f6b95367b80 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 0x56032728db70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6b780c9fa0 tx=0x7f6b78000bc0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- 172.21.15.31:0/3591946770 shutdown_connections 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560327295b50 0x56032728e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x560327295060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 --2- 172.21.15.31:0/3591946770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56032728ac90 0x56032728db70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- 172.21.15.31:0/3591946770 >> 172.21.15.31:0/3591946770 conn(0x560327280ca0 msgr2=0x5603272a5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- 172.21.15.31:0/3591946770 shutdown_connections 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- 172.21.15.31:0/3591946770 wait complete. 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- start start 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 Processor -- start 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x5603272a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x5603272a0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 0x56032729e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032729ed80 con 0x560327295b50 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032729eef0 con 0x56032728ac90 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56032729f0b0 con 0x56032728a380 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b95367b80 10 client.0 _ll_register_callbacks cb 0x56032719c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b8359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 0x56032729e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b8359e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 0x56032729e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57064/0 (socket says 172.21.15.31:57064) 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.906+0000 7f6b8359e700 1 -- 172.21.15.31:0/1926734550 learned_addr learned my addr 172.21.15.31:0/1926734550 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x5603272a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x5603272a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 -- 172.21.15.31:0/1926734550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 msgr2=0x5603272a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x5603272a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 -- 172.21.15.31:0/1926734550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 msgr2=0x56032729e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 0x56032729e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 -- 172.21.15.31:0/1926734550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56032705e3b0 con 0x56032728a380 2023-12-14T18:33:30.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x5603272a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b8359e700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 0x56032729e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b82d9d700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x5603272a0310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6b740036c0 tx=0x7f6b74004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461439a51b59672d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b66ff5700 1 -- 172.21.15.31:0/1926734550 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7402c040 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b780a1cc0 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.907+0000 7f6b66ff5700 1 -- 172.21.15.31:0/1926734550 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f6b74030dc0 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b66ff5700 1 -- 172.21.15.31:0/1926734550 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b74030630 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 10 client.26686 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5603271899a0 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b66ff5700 1 -- 172.21.15.31:0/1926734550 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b780a1cc0 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b66ff5700 1 -- 172.21.15.31:0/1926734550 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6b74021070 con 0x56032728a380 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 10 client.26686 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 20 client.26686 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:30.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272fe3b0 0x5603273007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 4 client.26686 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5603271d5a00 con 0x5603272fe3b0 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.908+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272fe3b0 0x5603273007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.909+0000 7f6b8259c700 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272fe3b0 0x5603273007c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fbf53ff354a78fc server_cookie=b0046c8624d2b279 in_seq=0 out_seq=0 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.909+0000 7f6b66ff5700 10 client.26686 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b66ff5700 1 -- 172.21.15.31:0/1926734550 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x5603271d5a00 con 0x5603272fe3b0 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b66ff5700 10 client.26686 handle_command_reply: tid=0 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 client.26686 shutdown 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 msgr2=0x5603272a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x5603272a0310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6b740036c0 tx=0x7f6b74004800 comp rx=0 tx=0).stop 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 shutdown_connections 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5603272fe3b0 0x5603273007c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560327295b50 0x56032729e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b66ff5700 0 client.26686 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:30.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56032728a380 0x5603272a0310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 --2- 172.21.15.31:0/1926734550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56032728ac90 0x5603272a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 >> 172.21.15.31:0/1926734550 conn(0x560327280ca0 msgr2=0x56032728db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 shutdown_connections 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 1 -- 172.21.15.31:0/1926734550 wait complete. 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.915+0000 7f6b95367b80 20 client.26686 trim_cache size 0 max 16384 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "scrub active (195 inodes in the stack)", 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: "20a26b1f-c873-47dd-bed6-5463c1efeb50": { 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:33:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "20a26b1f-c873-47dd-bed6-5463c1efeb50", 2023-12-14T18:33:30.244 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:33:30.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:33:30.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:33:30.244 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:33:30.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6b6c0a1340 msgr2=0x7f6b6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6b6c0a1340 0x7f6b6c0a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009990 tx=0x7f6b7401a040 comp rx=0 tx=0).stop 2023-12-14T18:33:30.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 msgr2=0x7f6b84196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:30.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 0x7f6b84196990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800d950 tx=0x7f6b7800dc60 comp rx=0 tx=0).stop 2023-12-14T18:33:30.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 shutdown_connections 2023-12-14T18:33:30.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f6b6c0a1340 0x7f6b6c0a37f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b841054f0 0x7f6b8419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b841004b0 0x7f6b84196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 --2- 172.21.15.31:0/154563531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b84100e50 0x7f6b84196990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 >> 172.21.15.31:0/154563531 conn(0x7f6b840fbd40 msgr2=0x7f6b840fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 shutdown_connections 2023-12-14T18:33:30.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:33:29.916+0000 7f6b891d9700 1 -- 172.21.15.31:0/154563531 wait complete. 2023-12-14T18:33:30.246 INFO:tasks.cephfs.filesystem:scrub status for tag:20a26b1f-c873-47dd-bed6-5463c1efeb50 - {'path': '/', 'tag': '20a26b1f-c873-47dd-bed6-5463c1efeb50', 'options': 'recursive,force'} 2023-12-14T18:33:30.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:30 smithi031 ceph-mon[42085]: pgmap v173: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 538 op/s 2023-12-14T18:33:30.246 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:30 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:33:30.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:30 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:30.247 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:30 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:33:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:30 smithi204 ceph-mon[43866]: pgmap v173: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 538 op/s 2023-12-14T18:33:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:30 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:33:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:30 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:30 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:33:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:30 smithi176 ceph-mon[43506]: pgmap v173: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 538 op/s 2023-12-14T18:33:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:30 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:33:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:30 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:30 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:33:31.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:31 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:33:31.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:31 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:33:31.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:31 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:33:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:32 smithi204 ceph-mon[43866]: pgmap v174: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2023-12-14T18:33:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:32 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:32 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:33:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:32 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:32 smithi031 ceph-mon[42085]: pgmap v174: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2023-12-14T18:33:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:32 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:32 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:33:32.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:32 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:32.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:32 smithi176 ceph-mon[43506]: pgmap v174: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2023-12-14T18:33:32.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:32 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:32.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:32 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:33:32.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:32 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:33.242 INFO:tasks.workunit.client.0.smithi031.stdout:cleaned up hierarchy 2023-12-14T18:33:33.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:33 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:33:33.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:33 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:33.341 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:33 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:33.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:33 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:33:33.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:33 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:33.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:33 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:33.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:33 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:33:33.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:33 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:33.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:33 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:33:34.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:34 smithi204 ceph-mon[43866]: pgmap v175: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2023-12-14T18:33:34.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:34 smithi031 ceph-mon[42085]: pgmap v175: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2023-12-14T18:33:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:34 smithi176 ceph-mon[43506]: pgmap v175: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 400 op/s 2023-12-14T18:33:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:36 smithi204 ceph-mon[43866]: pgmap v176: 129 pgs: 129 active+clean; 553 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 615 op/s 2023-12-14T18:33:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:36 smithi031 ceph-mon[42085]: pgmap v176: 129 pgs: 129 active+clean; 553 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 615 op/s 2023-12-14T18:33:36.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:36 smithi176 ceph-mon[43506]: pgmap v176: 129 pgs: 129 active+clean; 553 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 615 op/s 2023-12-14T18:33:37.157 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 -- 172.21.15.31:0/2757255651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de810a7a0 msgr2=0x7f0de810cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 --2- 172.21.15.31:0/2757255651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de810a7a0 0x7f0de810cc30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0de000b0a0 tx=0x7f0de000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 -- 172.21.15.31:0/2757255651 shutdown_connections 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 --2- 172.21.15.31:0/2757255651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de810a7a0 0x7f0de810cc30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 --2- 172.21.15.31:0/2757255651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0de8101460 0x7f0de8101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 --2- 172.21.15.31:0/2757255651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de8101d70 0x7f0de810a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 -- 172.21.15.31:0/2757255651 >> 172.21.15.31:0/2757255651 conn(0x7f0de80763a0 msgr2=0x7f0de80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 -- 172.21.15.31:0/2757255651 shutdown_connections 2023-12-14T18:33:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.152+0000 7f0deead5700 1 -- 172.21.15.31:0/2757255651 wait complete. 2023-12-14T18:33:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 Processor -- start 2023-12-14T18:33:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 -- start start 2023-12-14T18:33:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 0x7f0de80ff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0de8101d70 0x7f0de80ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de810a7a0 0x7f0de81a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80ffe20 con 0x7f0de8101460 2023-12-14T18:33:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81a4b20 con 0x7f0de8101d70 2023-12-14T18:33:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0deead5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81a4c60 con 0x7f0de810a7a0 2023-12-14T18:33:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.153+0000 7f0dec871700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 0x7f0de80ff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0ddffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0de8101d70 0x7f0de80ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 0x7f0de80ff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41680/0 (socket says 172.21.15.31:41680) 2023-12-14T18:33:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 -- 172.21.15.31:0/507391430 learned_addr learned my addr 172.21.15.31:0/507391430 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0ded072700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de810a7a0 0x7f0de81a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 -- 172.21.15.31:0/507391430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de810a7a0 msgr2=0x7f0de81a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de810a7a0 0x7f0de81a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 -- 172.21.15.31:0/507391430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0de8101d70 msgr2=0x7f0de80ff5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0de8101d70 0x7f0de80ff5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 -- 172.21.15.31:0/507391430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de0009d00 con 0x7f0de8101460 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dec871700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 0x7f0de80ff0b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0de400d950 tx=0x7f0de400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dddffb700 1 -- 172.21.15.31:0/507391430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de400f890 con 0x7f0de8101460 2023-12-14T18:33:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de8110260 con 0x7f0de8101460 2023-12-14T18:33:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dddffb700 1 -- 172.21.15.31:0/507391430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de4010460 con 0x7f0de8101460 2023-12-14T18:33:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.154+0000 7f0dddffb700 1 -- 172.21.15.31:0/507391430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de401b650 con 0x7f0de8101460 2023-12-14T18:33:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.155+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81100d0 con 0x7f0de8101460 2023-12-14T18:33:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.157+0000 7f0dddffb700 1 -- 172.21.15.31:0/507391430 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0de40108d0 con 0x7f0de8101460 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.157+0000 7f0dddffb700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0dd00a12c0 0x7f0dd00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.157+0000 7f0dddffb700 1 -- 172.21.15.31:0/507391430 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0de40cd1a0 con 0x7f0de8101460 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.157+0000 7f0ddffff700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0dd00a12c0 0x7f0dd00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.158+0000 7f0ddffff700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0dd00a12c0 0x7f0dd00a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0de81008a0 tx=0x7f0dd8009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.158+0000 7f0dfac63b80 1 Processor -- start 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.158+0000 7f0dfac63b80 1 -- start start 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 0x556f9fe07e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fcf2580 con 0x556f9fc65430 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fd61810 con 0x556f9fe130f0 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fd5aca0 con 0x556f9fe08370 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 0x556f9fe07e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 0x556f9fe07e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41688/0 (socket says 172.21.15.31:41688) 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 -- 172.21.15.31:0/435356027 learned_addr learned my addr 172.21.15.31:0/435356027 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0ddffff700 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0ded072700 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 -- 172.21.15.31:0/435356027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 msgr2=0x556f9fe087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe087e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 -- 172.21.15.31:0/435356027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 msgr2=0x556f9fe154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 -- 172.21.15.31:0/435356027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f9fbdc350 con 0x556f9fc65430 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dec871700 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 0x556f9fe07e30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0de40a1e00 tx=0x7f0de4099fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a74345c1cce8a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.159+0000 7f0dceffd700 1 -- 172.21.15.31:0/435356027 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de40a2330 con 0x556f9fc65430 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dceffd700 1 -- 172.21.15.31:0/435356027 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de40ca7e0 con 0x556f9fc65430 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dceffd700 1 -- 172.21.15.31:0/435356027 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4099610 con 0x556f9fc65430 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 -- 172.21.15.31:0/435356027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 msgr2=0x556f9fe07e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 0x556f9fe07e30 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f0de40a1e00 tx=0x7f0de4099fd0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 -- 172.21.15.31:0/435356027 shutdown_connections 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fc65430 0x556f9fe07e30 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/435356027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 -- 172.21.15.31:0/435356027 >> 172.21.15.31:0/435356027 conn(0x556f9fdfe890 msgr2=0x556f9fe04200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 -- 172.21.15.31:0/435356027 shutdown_connections 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.160+0000 7f0dfac63b80 1 -- 172.21.15.31:0/435356027 wait complete. 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.161+0000 7f0dfac63b80 1 -- start start 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.161+0000 7f0dfac63b80 1 Processor -- start 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe12a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe0fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe10350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fcf6f30 con 0x556f9fe08370 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fcf6bd0 con 0x556f9fe130f0 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fbde640 con 0x556f9fc65430 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 10 client.0 _ll_register_callbacks cb 0x556f9fd1a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe12a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe12a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:38400/0 (socket says 172.21.15.31:38400) 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 -- 172.21.15.31:0/2781591049 learned_addr learned my addr 172.21.15.31:0/2781591049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0ddffff700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe0fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0ded072700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe10350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 -- 172.21.15.31:0/2781591049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 msgr2=0x556f9fe10350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe10350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 -- 172.21.15.31:0/2781591049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 msgr2=0x556f9fe0fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe0fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 -- 172.21.15.31:0/2781591049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81100d0 con 0x556f9fc65430 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dec871700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe12a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0de40ca610 tx=0x7f0de40ca500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=208f868adb58b72 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dc7fff700 1 -- 172.21.15.31:0/2781591049 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de40a2330 con 0x556f9fc65430 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f9fdfbea0 con 0x556f9fc65430 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.162+0000 7f0dc7fff700 1 -- 172.21.15.31:0/2781591049 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0de40addf0 con 0x556f9fc65430 2023-12-14T18:33:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dc7fff700 1 -- 172.21.15.31:0/2781591049 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0de40b8120 con 0x556f9fc65430 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 10 client.26698 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556f9fd21110 con 0x556f9fc65430 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dc7fff700 1 -- 172.21.15.31:0/2781591049 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de4063080 con 0x556f9fc65430 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dc7fff700 1 -- 172.21.15.31:0/2781591049 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0de409a140 con 0x556f9fc65430 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 10 client.26698 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 20 client.26698 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe226d0 0x556f9fe24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 4 client.26698 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f9fd53c80 con 0x556f9fe226d0 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.163+0000 7f0ddffff700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe226d0 0x556f9fe24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dc7fff700 10 client.26698 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0ddffff700 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe226d0 0x556f9fe24ae0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfb4510bcf9bf703 server_cookie=c8529e5faef00413 in_seq=0 out_seq=0 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dc7fff700 1 -- 172.21.15.31:0/2781591049 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556f9fd53c80 con 0x556f9fe226d0 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dc7fff700 10 client.26698 handle_command_reply: tid=0 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 client.26698 shutdown 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 msgr2=0x556f9fe12a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe12a50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0de40ca610 tx=0x7f0de40ca500 comp rx=0 tx=0).stop 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 shutdown_connections 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe226d0 0x556f9fe24ae0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe10350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe12a50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2781591049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe0fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dc7fff700 0 client.26698 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 >> 172.21.15.31:0/2781591049 conn(0x556f9fdfe890 msgr2=0x556f9fe03d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.164+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 shutdown_connections 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.165+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2781591049 wait complete. 2023-12-14T18:33:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.165+0000 7f0dfac63b80 20 client.26698 trim_cache size 0 max 16384 2023-12-14T18:33:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 Processor -- start 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 -- start start 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fc65430 0x556f9fe19c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe03cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 0x556f9fe1edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fe15a10 con 0x556f9fe130f0 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fe11350 con 0x556f9fc65430 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.180+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fe098c0 con 0x556f9fe08370 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 0x556f9fe1edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0dec871700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fc65430 0x556f9fe19c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 0x556f9fe1edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41708/0 (socket says 172.21.15.31:41708) 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 -- 172.21.15.31:0/2611320682 learned_addr learned my addr 172.21.15.31:0/2611320682 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ddffff700 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe03cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 -- 172.21.15.31:0/2611320682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 msgr2=0x556f9fe03cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe03cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 -- 172.21.15.31:0/2611320682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fc65430 msgr2=0x556f9fe19c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fc65430 0x556f9fe19c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 -- 172.21.15.31:0/2611320682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f9fe000d0 con 0x556f9fe130f0 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0dec871700 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fc65430 0x556f9fe19c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ddffff700 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe03cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0ded072700 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 0x556f9fe1edd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009560 tx=0x7f0de0009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51602789ba082450 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0dcf7fe700 1 -- 172.21.15.31:0/2611320682 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de000e040 con 0x556f9fe130f0 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0dcf7fe700 1 -- 172.21.15.31:0/2611320682 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de000bde0 con 0x556f9fe130f0 2023-12-14T18:33:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.181+0000 7f0dcf7fe700 1 -- 172.21.15.31:0/2611320682 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de00054a0 con 0x556f9fe130f0 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2611320682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 msgr2=0x556f9fe1edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 0x556f9fe1edd0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009560 tx=0x7f0de0009590 comp rx=0 tx=0).stop 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2611320682 shutdown_connections 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe130f0 0x556f9fe1edd0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fc65430 0x556f9fe19c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/2611320682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fe08370 0x556f9fe03cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2611320682 >> 172.21.15.31:0/2611320682 conn(0x556f9fdfe890 msgr2=0x556f9fe02930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2611320682 shutdown_connections 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 -- 172.21.15.31:0/2611320682 wait complete. 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 -- start start 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.182+0000 7f0dfac63b80 1 Processor -- start 2023-12-14T18:33:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe17b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe18080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe6f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fe6fad0 con 0x556f9fe08370 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fe6fc40 con 0x556f9fe130f0 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f9fe6fe00 con 0x556f9fc65430 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dfac63b80 10 client.0 _ll_register_callbacks cb 0x7f0dd8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe18080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe18080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41714/0 (socket says 172.21.15.31:41714) 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 -- 172.21.15.31:0/899214261 learned_addr learned my addr 172.21.15.31:0/899214261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dec871700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe17b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.191 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ded072700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe6f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 -- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 msgr2=0x556f9fe17b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe17b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 -- 172.21.15.31:0/899214261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 msgr2=0x556f9fe6f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe6f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 -- 172.21.15.31:0/899214261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f9fd21110 con 0x556f9fe08370 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0ddffff700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe18080 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009fd0 tx=0x7f0dd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c860790d4b180d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.183+0000 7f0dc7fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dc7fff700 1 -- 172.21.15.31:0/899214261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8024040 con 0x556f9fe08370 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dc7fff700 1 -- 172.21.15.31:0/899214261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0dd8004050 con 0x556f9fe08370 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0de40ca7e0 con 0x556f9fe08370 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dc7fff700 1 -- 172.21.15.31:0/899214261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dd8004210 con 0x556f9fe08370 2023-12-14T18:33:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 10 client.16863 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f0de810d8f0 con 0x556f9fe08370 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dc7fff700 1 -- 172.21.15.31:0/899214261 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0de40ca7e0 con 0x556f9fe08370 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dc7fff700 1 -- 172.21.15.31:0/899214261 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0dd801e4a0 con 0x556f9fe08370 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 10 client.16863 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 20 client.16863 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe776e0 0x556f9fe79af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 4 client.16863 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f9fd539a0 con 0x556f9fe776e0 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.184+0000 7f0dec871700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe776e0 0x556f9fe79af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.185+0000 7f0dec871700 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe776e0 0x556f9fe79af0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41012f276579df8c server_cookie=51a8b49b6f3121cf in_seq=0 out_seq=0 2023-12-14T18:33:37.193 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.185+0000 7f0dc7fff700 10 client.16863 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.316+0000 7f0dc7fff700 1 -- 172.21.15.31:0/899214261 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f9fd539a0 con 0x556f9fe776e0 2023-12-14T18:33:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.316+0000 7f0dc7fff700 10 client.16863 handle_command_reply: tid=0 2023-12-14T18:33:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 client.16863 shutdown 2023-12-14T18:33:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 msgr2=0x556f9fe18080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe18080 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009fd0 tx=0x7f0dd80047c0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 shutdown_connections 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f9fe776e0 0x556f9fe79af0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f9fe130f0 0x556f9fe6f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f9fc65430 0x556f9fe17b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 --2- 172.21.15.31:0/899214261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f9fe08370 0x556f9fe18080 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dc7fff700 0 client.16863 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 >> 172.21.15.31:0/899214261 conn(0x556f9fdfe890 msgr2=0x556f9fe033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 shutdown_connections 2023-12-14T18:33:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 1 -- 172.21.15.31:0/899214261 wait complete. 2023-12-14T18:33:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.317+0000 7f0dfac63b80 20 client.16863 trim_cache size 0 max 16384 2023-12-14T18:33:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:33:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:33:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:33:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:33:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0dd00a12c0 msgr2=0x7f0dd00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0dd00a12c0 0x7f0dd00a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0de81008a0 tx=0x7f0dd8009260 comp rx=0 tx=0).stop 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 msgr2=0x7f0de80ff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 0x7f0de80ff0b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0de400d950 tx=0x7f0de400dc60 comp rx=0 tx=0).stop 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 shutdown_connections 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0dd00a12c0 0x7f0dd00a3770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0de810a7a0 0x7f0de81a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0de8101460 0x7f0de80ff0b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 --2- 172.21.15.31:0/507391430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0de8101d70 0x7f0de80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 >> 172.21.15.31:0/507391430 conn(0x7f0de80763a0 msgr2=0x7f0de81042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 shutdown_connections 2023-12-14T18:33:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:37.318+0000 7f0deead5700 1 -- 172.21.15.31:0/507391430 wait complete. 2023-12-14T18:33:38.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:38 smithi204 ceph-mon[43866]: pgmap v177: 129 pgs: 129 active+clean; 553 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 472 op/s 2023-12-14T18:33:38.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:38 smithi031 ceph-mon[42085]: pgmap v177: 129 pgs: 129 active+clean; 553 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 472 op/s 2023-12-14T18:33:38.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:38 smithi176 ceph-mon[43506]: pgmap v177: 129 pgs: 129 active+clean; 553 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 472 op/s 2023-12-14T18:33:39.488 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:39.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:39.535 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-12-14T18:33:39.535 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-12-14T18:33:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:40 smithi204 ceph-mon[43866]: pgmap v178: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 628 op/s 2023-12-14T18:33:40.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:40 smithi031 ceph-mon[42085]: pgmap v178: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 628 op/s 2023-12-14T18:33:40.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:40 smithi176 ceph-mon[43506]: pgmap v178: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 628 op/s 2023-12-14T18:33:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:42 smithi204 ceph-mon[43866]: pgmap v179: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 981 KiB/s wr, 371 op/s 2023-12-14T18:33:42.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:42 smithi031 ceph-mon[42085]: pgmap v179: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 981 KiB/s wr, 371 op/s 2023-12-14T18:33:42.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:42 smithi176 ceph-mon[43506]: pgmap v179: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 981 KiB/s wr, 371 op/s 2023-12-14T18:33:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:44 smithi204 ceph-mon[43866]: pgmap v180: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 981 KiB/s wr, 371 op/s 2023-12-14T18:33:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:44 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:33:44.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:44 smithi031 ceph-mon[42085]: pgmap v180: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 981 KiB/s wr, 371 op/s 2023-12-14T18:33:44.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:44 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:33:44.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:44 smithi176 ceph-mon[43506]: pgmap v180: 129 pgs: 129 active+clean; 546 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 981 KiB/s wr, 371 op/s 2023-12-14T18:33:44.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:44 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:33:45.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:45 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:33:45.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:45 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:33:45.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:45 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:33:45.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:45 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:33:45.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:45 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:33:45.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:45 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:33:46.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:46 smithi031 ceph-mon[42085]: pgmap v181: 129 pgs: 129 active+clean; 542 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 376 op/s 2023-12-14T18:33:46.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:46 smithi176 ceph-mon[43506]: pgmap v181: 129 pgs: 129 active+clean; 542 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 376 op/s 2023-12-14T18:33:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:46 smithi204 ceph-mon[43866]: pgmap v181: 129 pgs: 129 active+clean; 542 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 376 op/s 2023-12-14T18:33:48.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:48 smithi031 ceph-mon[42085]: pgmap v182: 129 pgs: 129 active+clean; 542 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 197 KiB/s wr, 161 op/s 2023-12-14T18:33:48.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:48 smithi176 ceph-mon[43506]: pgmap v182: 129 pgs: 129 active+clean; 542 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 197 KiB/s wr, 161 op/s 2023-12-14T18:33:48.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:48 smithi204 ceph-mon[43866]: pgmap v182: 129 pgs: 129 active+clean; 542 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 197 KiB/s wr, 161 op/s 2023-12-14T18:33:49.630 INFO:tasks.workunit.client.0.smithi031.stdout:ok 2023-12-14T18:33:49.635 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:49.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:49.705 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-12-14T18:33:49.706 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-12-14T18:33:50.448 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.445+0000 7f5df9070700 1 -- 172.21.15.31:0/3334211314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df4104460 msgr2=0x7f5df410c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.445+0000 7f5df9070700 1 --2- 172.21.15.31:0/3334211314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df4104460 0x7f5df410c030 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00b0d0 tx=0x7f5dec00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 -- 172.21.15.31:0/3334211314 shutdown_connections 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 --2- 172.21.15.31:0/3334211314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df4104460 0x7f5df410c030 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 --2- 172.21.15.31:0/3334211314 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df40ff420 0x7f5df40ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 --2- 172.21.15.31:0/3334211314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df40ffdc0 0x7f5df4103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 -- 172.21.15.31:0/3334211314 >> 172.21.15.31:0/3334211314 conn(0x7f5df4075fd0 msgr2=0x7f5df40763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 -- 172.21.15.31:0/3334211314 shutdown_connections 2023-12-14T18:33:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.446+0000 7f5df9070700 1 -- 172.21.15.31:0/3334211314 wait complete. 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 Processor -- start 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 -- start start 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df40ff420 0x7f5df41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 0x7f5df4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df4104460 0x7f5df41a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df41981a0 con 0x7f5df40ffdc0 2023-12-14T18:33:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df41a29b0 con 0x7f5df40ff420 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df9070700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df41a2af0 con 0x7f5df4104460 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.447+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 0x7f5df4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 0x7f5df4197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39132/0 (socket says 172.21.15.31:39132) 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 -- 172.21.15.31:0/1915679645 learned_addr learned my addr 172.21.15.31:0/1915679645 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df359e700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df4104460 0x7f5df41a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df40ff420 0x7f5df41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 -- 172.21.15.31:0/1915679645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df4104460 msgr2=0x7f5df41a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df4104460 0x7f5df41a2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 -- 172.21.15.31:0/1915679645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df40ff420 msgr2=0x7f5df41973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df40ff420 0x7f5df41973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 -- 172.21.15.31:0/1915679645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec009d00 con 0x7f5df40ffdc0 2023-12-14T18:33:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df259c700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 0x7f5df4197930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5de800d950 tx=0x7f5de800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5ddbfff700 1 -- 172.21.15.31:0/1915679645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8004220 con 0x7f5df40ffdc0 2023-12-14T18:33:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df410f230 con 0x7f5df40ffdc0 2023-12-14T18:33:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5ddbfff700 1 -- 172.21.15.31:0/1915679645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de8004cf0 con 0x7f5df40ffdc0 2023-12-14T18:33:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df410f0a0 con 0x7f5df40ffdc0 2023-12-14T18:33:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.448+0000 7f5ddbfff700 1 -- 172.21.15.31:0/1915679645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80056a0 con 0x7f5df40ffdc0 2023-12-14T18:33:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.451+0000 7f5ddbfff700 1 -- 172.21.15.31:0/1915679645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5de8004380 con 0x7f5df40ffdc0 2023-12-14T18:33:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.452+0000 7f5ddbfff700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5ddc0a12c0 0x7f5ddc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.452+0000 7f5ddbfff700 1 -- 172.21.15.31:0/1915679645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f5de80ccd90 con 0x7f5df40ffdc0 2023-12-14T18:33:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.452+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5ddc0a12c0 0x7f5ddc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 Processor -- start 2023-12-14T18:33:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 -- start start 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5ddc0a12c0 0x7f5ddc0a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009c60 tx=0x7f5de4009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e381d8430 0x559e3837f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e3837fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3838a500 0x559e3838c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38265580 con 0x559e3837f780 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e382d4810 con 0x559e3838a500 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5e051feb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e382cdca0 con 0x559e381d8430 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e3837fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e3837fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39148/0 (socket says 172.21.15.31:39148) 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 -- 172.21.15.31:0/2780676716 learned_addr learned my addr 172.21.15.31:0/2780676716 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df359e700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3838a500 0x559e3838c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e381d8430 0x559e3837f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 -- 172.21.15.31:0/2780676716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e381d8430 msgr2=0x559e3837f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e381d8430 0x559e3837f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 -- 172.21.15.31:0/2780676716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3838a500 msgr2=0x559e3838c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3838a500 0x559e3838c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df259c700 1 -- 172.21.15.31:0/2780676716 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e3814f350 con 0x559e3837f780 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df359e700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3838a500 0x559e3838c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.453+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e381d8430 0x559e3837f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5df259c700 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e3837fbf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5de8005060 tx=0x7f5de80996e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c488ba58c6fd611 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5dd97fa700 1 -- 172.21.15.31:0/2780676716 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80a1ec0 con 0x559e3837f780 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5dd97fa700 1 -- 172.21.15.31:0/2780676716 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de8099cd0 con 0x559e3837f780 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5dd97fa700 1 -- 172.21.15.31:0/2780676716 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80a1c50 con 0x559e3837f780 2023-12-14T18:33:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 -- 172.21.15.31:0/2780676716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 msgr2=0x559e3837fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e3837fbf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5de8005060 tx=0x7f5de80996e0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 -- 172.21.15.31:0/2780676716 shutdown_connections 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3838a500 0x559e3838c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e381d8430 0x559e3837f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 --2- 172.21.15.31:0/2780676716 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e3837fbf0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.454+0000 7f5e051feb80 1 -- 172.21.15.31:0/2780676716 >> 172.21.15.31:0/2780676716 conn(0x559e38371890 msgr2=0x559e38377200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 -- 172.21.15.31:0/2780676716 shutdown_connections 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 -- 172.21.15.31:0/2780676716 wait complete. 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 -- start start 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 Processor -- start 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38389e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.465 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e383871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38387750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38269f30 con 0x559e3837f780 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38269bd0 con 0x559e381d8430 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38151640 con 0x559e3838a500 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5e051feb80 10 client.0 _ll_register_callbacks cb 0x559e3828d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e383871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e383871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39160/0 (socket says 172.21.15.31:39160) 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.455+0000 7f5df259c700 1 -- 172.21.15.31:0/3391579074 learned_addr learned my addr 172.21.15.31:0/3391579074 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df359e700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38387750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38389e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.466 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df259c700 1 -- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 msgr2=0x559e38387750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df259c700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38387750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df259c700 1 -- 172.21.15.31:0/3391579074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 msgr2=0x559e38389e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df259c700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38389e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df259c700 1 -- 172.21.15.31:0/3391579074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df410f0a0 con 0x559e3837f780 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5df259c700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e383871b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5de8099c30 tx=0x7f5de8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a9a02a0e01cca server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5dd6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/3391579074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b2290 con 0x559e3837f780 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/3391579074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f5de80a1ec0 con 0x559e3837f780 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e3836eea0 con 0x559e3837f780 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/3391579074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5de8099170 con 0x559e3837f780 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5e051feb80 10 client.16875 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.456+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559e38294110 con 0x559e3837f780 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/3391579074 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e3836eea0 con 0x559e3837f780 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/3391579074 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5de8004750 con 0x559e3837f780 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5e051feb80 10 client.16875 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5e051feb80 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5e051feb80 20 client.16875 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5e051feb80 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e38399ae0 0x559e3839bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5e051feb80 4 client.16875 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e382c6c80 con 0x559e38399ae0 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.457+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e38399ae0 0x559e3839bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.458+0000 7f5dd6ff5700 10 client.16875 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.458+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e38399ae0 0x559e3839bef0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb5c3012354e68f server_cookie=646f2613689e69c1 in_seq=0 out_seq=0 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.458+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/3391579074 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559e382c6c80 con 0x559e38399ae0 2023-12-14T18:33:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.458+0000 7f5dd6ff5700 10 client.16875 handle_command_reply: tid=0 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.458+0000 7f5e051feb80 1 client.16875 shutdown 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 msgr2=0x559e383871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e383871b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5de8099c30 tx=0x7f5de8005840 comp rx=0 tx=0).stop 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 shutdown_connections 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e38399ae0 0x559e3839bef0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38387750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38389e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 --2- 172.21.15.31:0/3391579074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e383871b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 >> 172.21.15.31:0/3391579074 conn(0x559e38371890 msgr2=0x559e38376d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5dd6ff5700 0 client.16875 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 shutdown_connections 2023-12-14T18:33:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 1 -- 172.21.15.31:0/3391579074 wait complete. 2023-12-14T18:33:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.459+0000 7f5e051feb80 20 client.16875 trim_cache size 0 max 16384 2023-12-14T18:33:50.470 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:50 smithi031 ceph-mon[42085]: pgmap v183: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 227 KiB/s wr, 165 op/s 2023-12-14T18:33:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.474+0000 7f5e051feb80 1 Processor -- start 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.474+0000 7f5e051feb80 1 -- start start 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38387160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e38390fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38389950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5e051feb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38381110 con 0x559e3837f780 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5e051feb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38383000 con 0x559e381d8430 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5e051feb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e38396790 con 0x559e3838a500 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e38390fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e38390fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39176/0 (socket says 172.21.15.31:39176) 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 -- 172.21.15.31:0/4063980901 learned_addr learned my addr 172.21.15.31:0/4063980901 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 -- 172.21.15.31:0/4063980901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 msgr2=0x559e38389950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df359e700 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38389950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38387160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38389950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 -- 172.21.15.31:0/4063980901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 msgr2=0x559e38387160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38387160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 -- 172.21.15.31:0/4063980901 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e38293e60 con 0x559e3837f780 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38387160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5df259c700 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e38390fd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5de8099770 tx=0x7f5de80996e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aff9621b7b9540 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5dd9ffb700 1 -- 172.21.15.31:0/4063980901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b2290 con 0x559e3837f780 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5dd9ffb700 1 -- 172.21.15.31:0/4063980901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de80a1ec0 con 0x559e3837f780 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.475+0000 7f5dd9ffb700 1 -- 172.21.15.31:0/4063980901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8098b70 con 0x559e3837f780 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 -- 172.21.15.31:0/4063980901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 msgr2=0x559e38390fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e38390fd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5de8099770 tx=0x7f5de80996e0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 -- 172.21.15.31:0/4063980901 shutdown_connections 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e38389950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e381d8430 0x559e38387160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 --2- 172.21.15.31:0/4063980901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e3837f780 0x559e38390fd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 -- 172.21.15.31:0/4063980901 >> 172.21.15.31:0/4063980901 conn(0x559e38371890 msgr2=0x559e38386a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 -- 172.21.15.31:0/4063980901 shutdown_connections 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 -- 172.21.15.31:0/4063980901 wait complete. 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.476+0000 7f5e051feb80 1 -- start start 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 Processor -- start 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 0x559e3838ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3837f780 0x559e3838ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e383ec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e3838f850 con 0x559e381d8430 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e383ecea0 con 0x559e3837f780 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e383ecfe0 con 0x559e3838a500 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5e051feb80 10 client.0 _ll_register_callbacks cb 0x7f5de80995b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 0x559e3838ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 0x559e3838ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39180/0 (socket says 172.21.15.31:39180) 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 -- 172.21.15.31:0/1681281897 learned_addr learned my addr 172.21.15.31:0/1681281897 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df359e700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e383ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df259c700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3837f780 0x559e3838ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 -- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 msgr2=0x559e383ec960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e383ec960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 -- 172.21.15.31:0/1681281897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3837f780 msgr2=0x559e3838ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3837f780 0x559e3838ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.477+0000 7f5df2d9d700 1 -- 172.21.15.31:0/1681281897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e38294110 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5df2d9d700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 0x559e3838ea00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5de4013010 tx=0x7f5de4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abda994c09a4f170 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5dd6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/1681281897 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4023040 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5de8099cd0 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/1681281897 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f5de4003d10 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/1681281897 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5de4003ed0 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 10 client.16881 fetch_fsmap learned FSMap version 28 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559e3827a940 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/1681281897 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5de8099cd0 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/1681281897 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5de401d810 con 0x559e381d8430 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 10 client.16881 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:33:50.486 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 20 client.16881 populate_metadata read hostname 'smithi031' 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e383ee670 0x559e383f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 4 client.16881 mds_command: new command op to 24538 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.478+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e382c69a0 con 0x559e383ee670 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.479+0000 7f5df259c700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e383ee670 0x559e383f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.479+0000 7f5dd6ff5700 10 client.16881 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.479+0000 7f5df259c700 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e383ee670 0x559e383f0a80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371f6b87031be5dd server_cookie=ea4a44293a5c714f in_seq=0 out_seq=0 2023-12-14T18:33:50.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:50 smithi176 ceph-mon[43506]: pgmap v183: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 227 KiB/s wr, 165 op/s 2023-12-14T18:33:50.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:50 smithi204 ceph-mon[43866]: pgmap v183: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 227 KiB/s wr, 165 op/s 2023-12-14T18:33:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5dd6ff5700 1 -- 172.21.15.31:0/1681281897 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e382c69a0 con 0x559e383ee670 2023-12-14T18:33:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5dd6ff5700 10 client.16881 handle_command_reply: tid=0 2023-12-14T18:33:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 client.16881 shutdown 2023-12-14T18:33:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 msgr2=0x559e3838ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.599 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 0x559e3838ea00 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5de4013010 tx=0x7f5de4013040 comp rx=0 tx=0).stop 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 shutdown_connections 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559e383ee670 0x559e383f0a80 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559e3838a500 0x559e383ec960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559e381d8430 0x559e3838ea00 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5dd6ff5700 0 client.16881 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 --2- 172.21.15.31:0/1681281897 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559e3837f780 0x559e3838ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.596+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 >> 172.21.15.31:0/1681281897 conn(0x559e38371890 msgr2=0x559e38381330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 shutdown_connections 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5e051feb80 1 -- 172.21.15.31:0/1681281897 wait complete. 2023-12-14T18:33:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5e051feb80 20 client.16881 trim_cache size 0 max 16384 2023-12-14T18:33:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:{ 2023-12-14T18:33:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "message": "", 2023-12-14T18:33:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout: "return_code": 0 2023-12-14T18:33:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stdout:} 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5ddc0a12c0 msgr2=0x7f5ddc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5ddc0a12c0 0x7f5ddc0a3770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009c60 tx=0x7f5de4009260 comp rx=0 tx=0).stop 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 msgr2=0x7f5df4197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 0x7f5df4197930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5de800d950 tx=0x7f5de800dc60 comp rx=0 tx=0).stop 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 shutdown_connections 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5ddc0a12c0 0x7f5ddc0a3770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df4104460 0x7f5df41a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5df40ff420 0x7f5df41973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 --2- 172.21.15.31:0/1915679645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5df40ffdc0 0x7f5df4197930 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.597+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 >> 172.21.15.31:0/1915679645 conn(0x7f5df4075fd0 msgr2=0x7f5df40fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:50.602 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.598+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 shutdown_connections 2023-12-14T18:33:50.603 INFO:teuthology.task.background_exec.ubuntu@smithi031.front.sepia.ceph.com.smithi031.stderr:2023-12-14T18:33:50.598+0000 7f5df9070700 1 -- 172.21.15.31:0/1915679645 wait complete. 2023-12-14T18:33:52.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:52 smithi031 ceph-mon[42085]: pgmap v184: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 8 op/s 2023-12-14T18:33:52.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:52 smithi176 ceph-mon[43506]: pgmap v184: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 8 op/s 2023-12-14T18:33:52.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:52 smithi204 ceph-mon[43866]: pgmap v184: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 8 op/s 2023-12-14T18:33:54.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:54 smithi031 ceph-mon[42085]: pgmap v185: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 8 op/s 2023-12-14T18:33:54.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:54 smithi176 ceph-mon[43506]: pgmap v185: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 8 op/s 2023-12-14T18:33:54.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:54 smithi204 ceph-mon[43866]: pgmap v185: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 90 KiB/s wr, 8 op/s 2023-12-14T18:33:56.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:56 smithi031 ceph-mon[42085]: pgmap v186: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 122 KiB/s wr, 106 op/s 2023-12-14T18:33:56.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:56 smithi176 ceph-mon[43506]: pgmap v186: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 122 KiB/s wr, 106 op/s 2023-12-14T18:33:56.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:56 smithi204 ceph-mon[43866]: pgmap v186: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 122 KiB/s wr, 106 op/s 2023-12-14T18:33:58.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:33:58 smithi031 ceph-mon[42085]: pgmap v187: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 100 op/s 2023-12-14T18:33:58.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:33:58 smithi176 ceph-mon[43506]: pgmap v187: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 100 op/s 2023-12-14T18:33:58.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:33:58 smithi204 ceph-mon[43866]: pgmap v187: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 62 KiB/s wr, 100 op/s 2023-12-14T18:33:59.765 INFO:tasks.workunit.client.0.smithi031.stdout:ok 2023-12-14T18:33:59.769 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:59.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:59.817 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-12-14T18:33:59.817 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-12-14T18:33:59.858 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p dir 2023-12-14T18:33:59.861 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-14T18:33:59.861 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'ceph.dir.pin="-1"' 2023-12-14T18:33:59.863 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.dir.pin="-1" 2023-12-14T18:33:59.863 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.pin dir 2023-12-14T18:33:59.863 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:59.864 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:59.864 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-12-14T18:33:59.866 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-14T18:33:59.866 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'ceph.dir.pin="1"' 2023-12-14T18:33:59.868 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.dir.pin="1" 2023-12-14T18:33:59.868 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.pin dir 2023-12-14T18:33:59.870 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-14T18:33:59.870 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'ceph.dir.pin="-1"' 2023-12-14T18:33:59.871 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.dir.pin="-1" 2023-12-14T18:33:59.872 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-12-14T18:33:59.872 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:59.873 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:59.873 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-12-14T18:33:59.875 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-12-14T18:33:59.877 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-12-14T18:33:59.877 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:59.877 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:59.878 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-12-14T18:33:59.879 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-12-14T18:33:59.881 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.quota.max_bytes dir 2023-12-14T18:33:59.881 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:59.882 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:59.882 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.quota.max_bytes -v 100000000 dir 2023-12-14T18:33:59.893 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.quota.max_bytes dir 2023-12-14T18:33:59.897 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.quota.max_files dir 2023-12-14T18:33:59.897 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:59.898 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:59.898 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.quota.max_files -v 10000 dir 2023-12-14T18:33:59.901 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.quota.max_files dir 2023-12-14T18:33:59.905 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir dir 2023-12-14T18:33:59.909 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo OK 2023-12-14T18:33:59.909 INFO:tasks.workunit.client.0.smithi031.stdout:OK 2023-12-14T18:33:59.910 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:59.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:59.993 INFO:tasks.workunit:Running workunit fs/misc/i_complete_vs_rename.sh... 2023-12-14T18:33:59.994 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/i_complete_vs_rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/i_complete_vs_rename.sh 2023-12-14T18:34:00.081 INFO:tasks.workunit.client.0.smithi031.stdout:a 2023-12-14T18:34:00.081 INFO:tasks.workunit.client.0.smithi031.stdout:b 2023-12-14T18:34:00.081 INFO:tasks.workunit.client.0.smithi031.stdout:c 2023-12-14T18:34:00.081 INFO:tasks.workunit.client.0.smithi031.stdout:d 2023-12-14T18:34:00.092 INFO:tasks.workunit.client.0.smithi031.stderr:stat: cannot statx 'e': No such file or directory 2023-12-14T18:34:00.096 INFO:tasks.workunit.client.0.smithi031.stdout:attempting rename over existing file... 2023-12-14T18:34:00.119 INFO:tasks.workunit.client.0.smithi031.stdout:f 2023-12-14T18:34:00.119 INFO:tasks.workunit.client.0.smithi031.stdout:rename over existing file is okay 2023-12-14T18:34:00.119 INFO:tasks.workunit.client.0.smithi031.stdout:attempting rename over negative dentry... 2023-12-14T18:34:00.129 INFO:tasks.workunit.client.0.smithi031.stdout:e 2023-12-14T18:34:00.130 INFO:tasks.workunit.client.0.smithi031.stdout:rename over negative dentry is ok 2023-12-14T18:34:00.130 INFO:tasks.workunit.client.0.smithi031.stdout:OK 2023-12-14T18:34:00.130 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:34:00.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:34:00.193 INFO:tasks.workunit:Running workunit fs/misc/layout_vxattrs.sh... 2023-12-14T18:34:00.194 DEBUG:teuthology.orchestra.run.smithi031:workunit test fs/misc/layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh 2023-12-14T18:34:00.223 INFO:tasks.workunit.client.0.smithi031.stderr:+ datapool= 2023-12-14T18:34:00.224 INFO:tasks.workunit.client.0.smithi031.stderr:+ dir=. 2023-12-14T18:34:00.224 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.224 INFO:tasks.workunit.client.0.smithi031.stdout:. 2023-12-14T18:34:00.224 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo . 2023-12-14T18:34:00.224 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfattr -n ceph.dir.layout.pool . --only-values 2023-12-14T18:34:00.225 INFO:tasks.workunit.client.0.smithi031.stderr:.: ceph.dir.layout.pool: No such attribute 2023-12-14T18:34:00.225 INFO:tasks.workunit.client.0.smithi031.stderr:+ datapool= 2023-12-14T18:34:00.225 INFO:tasks.workunit.client.0.smithi031.stderr:+ dir=./.. 2023-12-14T18:34:00.225 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.225 INFO:tasks.workunit.client.0.smithi031.stdout:./.. 2023-12-14T18:34:00.226 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo ./.. 2023-12-14T18:34:00.226 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfattr -n ceph.dir.layout.pool ./.. --only-values 2023-12-14T18:34:00.226 INFO:tasks.workunit.client.0.smithi031.stderr:./..: ceph.dir.layout.pool: No such attribute 2023-12-14T18:34:00.226 INFO:tasks.workunit.client.0.smithi031.stderr:+ datapool= 2023-12-14T18:34:00.227 INFO:tasks.workunit.client.0.smithi031.stderr:+ dir=./../.. 2023-12-14T18:34:00.227 INFO:tasks.workunit.client.0.smithi031.stdout:./../.. 2023-12-14T18:34:00.227 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.227 INFO:tasks.workunit.client.0.smithi031.stderr:+ echo ./../.. 2023-12-14T18:34:00.227 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfattr -n ceph.dir.layout.pool ./../.. --only-values 2023-12-14T18:34:00.228 INFO:tasks.workunit.client.0.smithi031.stderr:+ datapool=cephfs_data_ec 2023-12-14T18:34:00.228 INFO:tasks.workunit.client.0.smithi031.stderr:+ break 2023-12-14T18:34:00.228 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -f file file2 2023-12-14T18:34:00.229 INFO:tasks.workunit.client.0.smithi031.stderr:+ touch file file2 2023-12-14T18:34:00.232 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:34:00.233 INFO:tasks.workunit.client.0.smithi031.stdout:# file: file 2023-12-14T18:34:00.233 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-14T18:34:00.233 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.234 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:34:00.234 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q object_size= 2023-12-14T18:34:00.235 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:34:00.235 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q stripe_count= 2023-12-14T18:34:00.237 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:34:00.237 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q stripe_unit= 2023-12-14T18:34:00.238 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:34:00.239 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q pool= 2023-12-14T18:34:00.239 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool file 2023-12-14T18:34:00.241 INFO:tasks.workunit.client.0.smithi031.stdout:# file: file 2023-12-14T18:34:00.241 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.file.layout.pool="cephfs_data_ec" 2023-12-14T18:34:00.241 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.241 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace file 2023-12-14T18:34:00.242 INFO:tasks.workunit.client.0.smithi031.stdout:# file: file 2023-12-14T18:34:00.243 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.file.layout.pool_namespace="" 2023-12-14T18:34:00.243 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.243 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_unit file 2023-12-14T18:34:00.244 INFO:tasks.workunit.client.0.smithi031.stdout:# file: file 2023-12-14T18:34:00.244 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.file.layout.stripe_unit="4194304" 2023-12-14T18:34:00.244 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.244 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file 2023-12-14T18:34:00.246 INFO:tasks.workunit.client.0.smithi031.stdout:# file: file 2023-12-14T18:34:00.246 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.file.layout.stripe_count="1" 2023-12-14T18:34:00.246 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.246 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.object_size file 2023-12-14T18:34:00.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:34:00.248 INFO:tasks.workunit.client.0.smithi031.stdout:# file: file 2023-12-14T18:34:00.248 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.file.layout.object_size="4194304" 2023-12-14T18:34:00.248 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.249 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.bogus file 2023-12-14T18:34:00.249 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 'No such attribute' 2023-12-14T18:34:00.249 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout file 2023-12-14T18:34:00.249 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 'No such attribute' 2023-12-14T18:34:00.251 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2 2023-12-14T18:34:00.257 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout.stripe_count -v 8 file2 2023-12-14T18:34:00.261 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout.object_size -v 10485760 file2 2023-12-14T18:34:00.267 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout.pool -v cephfs_data_ec file2 2023-12-14T18:34:00.274 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:34:00.274 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:34:00.275 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout.pool_namespace -v foons file2 2023-12-14T18:34:00.279 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:34:00.279 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q foons 2023-12-14T18:34:00.281 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.pool_namespace file2 2023-12-14T18:34:00.285 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:34:00.285 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q -v foons 2023-12-14T18:34:00.287 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:34:00.287 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 1048576 2023-12-14T18:34:00.288 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:34:00.289 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 8 2023-12-14T18:34:00.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:34:00.290 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 10485760 2023-12-14T18:34:00.291 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=4194304 stripe_count=16 object_size=41943040 pool=cephfs_data_ec pool_namespace=foons' file2 2023-12-14T18:34:00.296 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:34:00.296 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 4194304 2023-12-14T18:34:00.297 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:34:00.297 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 16 2023-12-14T18:34:00.299 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:34:00.299 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 41943040 2023-12-14T18:34:00.300 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:34:00.300 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:34:00.302 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:34:00.302 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q foons 2023-12-14T18:34:00.303 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout -v stripe_unit=1048576 file2 2023-12-14T18:34:00.310 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:34:00.310 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 1048576 2023-12-14T18:34:00.312 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:34:00.312 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 16 2023-12-14T18:34:00.313 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:34:00.313 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 41943040 2023-12-14T18:34:00.315 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:34:00.315 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:34:00.316 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:34:00.316 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q foons 2023-12-14T18:34:00.317 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=2097152 stripe_count=4 object_size=2097152 pool=cephfs_data_ec pool_namespace=barns' file2 2023-12-14T18:34:00.322 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:34:00.322 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 2097152 2023-12-14T18:34:00.324 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:34:00.324 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 4 2023-12-14T18:34:00.326 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:34:00.326 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 2097152 2023-12-14T18:34:00.328 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:34:00.328 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:34:00.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:34:00.330 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q barns 2023-12-14T18:34:00.331 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -f dir/file 2023-12-14T18:34:00.332 INFO:tasks.workunit.client.0.smithi031.stderr:+ rmdir dir 2023-12-14T18:34:00.332 INFO:tasks.workunit.client.0.smithi031.stderr:rmdir: failed to remove 'dir': No such file or directory 2023-12-14T18:34:00.333 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.333 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p dir 2023-12-14T18:34:00.335 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -d -m - dir 2023-12-14T18:34:00.335 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q ceph.dir.layout 2023-12-14T18:34:00.336 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.336 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -d -m - dir 2023-12-14T18:34:00.336 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q ceph.file.layout 2023-12-14T18:34:00.337 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.337 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.338 INFO:tasks.workunit.client.0.smithi031.stderr:dir: ceph.dir.layout: No such attribute 2023-12-14T18:34:00.338 INFO:tasks.workunit.client.0.smithi031.stderr:+ true 2023-12-14T18:34:00.338 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.layout.stripe_unit -v 1048576 dir 2023-12-14T18:34:00.343 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.layout.stripe_count -v 8 dir 2023-12-14T18:34:00.347 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.layout.object_size -v 10485760 dir 2023-12-14T18:34:00.350 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.layout.pool -v cephfs_data_ec dir 2023-12-14T18:34:00.354 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.layout.pool_namespace -v dirns dir 2023-12-14T18:34:00.358 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.360 INFO:tasks.workunit.client.0.smithi031.stdout:# file: dir 2023-12-14T18:34:00.360 INFO:tasks.workunit.client.0.smithi031.stdout:ceph.dir.layout="stripe_unit=1048576 stripe_count=8 object_size=10485760 pool=cephfs_data_ec pool_namespace=dirns" 2023-12-14T18:34:00.360 INFO:tasks.workunit.client.0.smithi031.stdout: 2023-12-14T18:34:00.360 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.360 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q object_size=10485760 2023-12-14T18:34:00.362 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.362 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q stripe_count=8 2023-12-14T18:34:00.363 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.363 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q stripe_unit=1048576 2023-12-14T18:34:00.365 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.365 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q pool=cephfs_data_ec 2023-12-14T18:34:00.366 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.366 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q pool_namespace=dirns 2023-12-14T18:34:00.367 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout.pool dir 2023-12-14T18:34:00.367 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:34:00.369 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout.stripe_unit dir 2023-12-14T18:34:00.369 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 1048576 2023-12-14T18:34:00.370 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout.stripe_count dir 2023-12-14T18:34:00.371 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 8 2023-12-14T18:34:00.372 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout.object_size dir 2023-12-14T18:34:00.372 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 10485760 2023-12-14T18:34:00.373 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout.pool_namespace dir 2023-12-14T18:34:00.374 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q dirns 2023-12-14T18:34:00.375 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout -v stripe_count=16 file2 2023-12-14T18:34:00.376 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:34:00.376 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 16 2023-12-14T18:34:00.378 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.file.layout -v 'object_size=10485760 stripe_count=8 stripe_unit=1048576 pool=cephfs_data_ec pool_namespace=dirns' file2 2023-12-14T18:34:00.382 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:34:00.382 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 8 2023-12-14T18:34:00.384 INFO:tasks.workunit.client.0.smithi031.stderr:+ touch dir/file 2023-12-14T18:34:00.386 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool dir/file 2023-12-14T18:34:00.386 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:34:00.387 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_unit dir/file 2023-12-14T18:34:00.387 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 1048576 2023-12-14T18:34:00.389 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.stripe_count dir/file 2023-12-14T18:34:00.389 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 8 2023-12-14T18:34:00.391 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.object_size dir/file 2023-12-14T18:34:00.391 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 10485760 2023-12-14T18:34:00.392 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.file.layout.pool_namespace dir/file 2023-12-14T18:34:00.392 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q dirns 2023-12-14T18:34:00.397 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.pool dir 2023-12-14T18:34:00.398 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:34:00.402 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:34:00.403 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.pool_id dir 2023-12-14T18:34:00.403 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:34:00.404 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:34:00.405 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.pool_name dir 2023-12-14T18:34:00.405 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 -- 172.21.15.31:0/3625640615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 msgr2=0x7f0d8810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 --2- 172.21.15.31:0/3625640615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 0x7f0d8810bfc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009fc0 tx=0x7f0d8000b330 comp rx=0 tx=0).stop 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 -- 172.21.15.31:0/3625640615 shutdown_connections 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 --2- 172.21.15.31:0/3625640615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 0x7f0d8810bfc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 --2- 172.21.15.31:0/3625640615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d880ff7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 --2- 172.21.15.31:0/3625640615 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d880ffda0 0x7f0d88103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 -- 172.21.15.31:0/3625640615 >> 172.21.15.31:0/3625640615 conn(0x7f0d880761c0 msgr2=0x7f0d880765c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 -- 172.21.15.31:0/3625640615 shutdown_connections 2023-12-14T18:34:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.404+0000 7f0d8e536700 1 -- 172.21.15.31:0/3625640615 wait complete. 2023-12-14T18:34:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.405+0000 7f0d8e536700 1 Processor -- start 2023-12-14T18:34:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.405+0000 7f0d8e536700 1 -- start start 2023-12-14T18:34:00.407 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8e536700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d88196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8e536700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d880ffda0 0x7f0d881969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8e536700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 0x7f0d8819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8e536700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819b470 con 0x7f0d880ff400 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8e536700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819b5e0 con 0x7f0d880ffda0 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8e536700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8819b7a0 con 0x7f0d881043f0 2023-12-14T18:34:00.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d88196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.408 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.stripe_unit dir 2023-12-14T18:34:00.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d88196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38332/0 (socket says 172.21.15.31:38332) 2023-12-14T18:34:00.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 -- 172.21.15.31:0/2132901289 learned_addr learned my addr 172.21.15.31:0/2132901289 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:00.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 0x7f0d8819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.409 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d880ffda0 0x7f0d881969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 -- 172.21.15.31:0/2132901289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 msgr2=0x7f0d8819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 0x7f0d8819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 -- 172.21.15.31:0/2132901289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d880ffda0 msgr2=0x7f0d881969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d880ffda0 0x7f0d881969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.406+0000 7f0d87fff700 1 -- 172.21.15.31:0/2132901289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80009c40 con 0x7f0d880ff400 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.407+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d88196490 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e9c0 tx=0x7f0d7800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.407+0000 7f0d857fa700 1 -- 172.21.15.31:0/2132901289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78004a30 con 0x7f0d880ff400 2023-12-14T18:34:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.407+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d88110380 con 0x7f0d880ff400 2023-12-14T18:34:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.407+0000 7f0d857fa700 1 -- 172.21.15.31:0/2132901289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d7800cc80 con 0x7f0d880ff400 2023-12-14T18:34:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.407+0000 7f0d857fa700 1 -- 172.21.15.31:0/2132901289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780054e0 con 0x7f0d880ff400 2023-12-14T18:34:00.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.407+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d881101f0 con 0x7f0d880ff400 2023-12-14T18:34:00.411 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:34:00.411 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.stripe_count dir 2023-12-14T18:34:00.411 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:34:00.412 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:34:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.410+0000 7f0d857fa700 1 -- 172.21.15.31:0/2132901289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0d78004b90 con 0x7f0d880ff400 2023-12-14T18:34:00.412 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.object_size dir 2023-12-14T18:34:00.412 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:34:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.411+0000 7f0d857fa700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0d700a1340 0x7f0d700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.411+0000 7f0d857fa700 1 -- 172.21.15.31:0/2132901289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0d78014070 con 0x7f0d880ff400 2023-12-14T18:34:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.411+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0d700a1340 0x7f0d700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.411+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0d700a1340 0x7f0d700a37f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d88197aa0 tx=0x7f0d7c0073d0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.413 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir: Invalid argument 2023-12-14T18:34:00.413 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.pool dir/file 2023-12-14T18:34:00.414 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:34:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 Processor -- start 2023-12-14T18:34:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 -- start start 2023-12-14T18:34:00.414 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:34:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 0x56182a53f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a54a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 0x56182a54cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a4295e0 con 0x56182a53fc90 2023-12-14T18:34:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a498870 con 0x56182a53f380 2023-12-14T18:34:00.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a491d00 con 0x56182a54ab50 2023-12-14T18:34:00.416 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.pool_id dir/file 2023-12-14T18:34:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d8cad3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 0x56182a54cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 0x56182a53f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.416 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d8cad3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 0x56182a54cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:59240/0 (socket says 172.21.15.31:59240) 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d8cad3700 1 -- 172.21.15.31:0/2303048845 learned_addr learned my addr 172.21.15.31:0/2303048845 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.413+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 0x56182a53f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:34684/0 (socket says 172.21.15.31:34684) 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a54a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d8cad3700 1 -- 172.21.15.31:0/2303048845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 msgr2=0x56182a53f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 0x56182a53f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d8cad3700 1 -- 172.21.15.31:0/2303048845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 msgr2=0x56182a54a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a54a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d8cad3700 1 -- 172.21.15.31:0/2303048845 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182a3133b0 con 0x56182a54ab50 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a54a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 0x56182a53f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:00.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 0x56182a54cf30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009420 tx=0x7f0d80009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=879adec872c75619 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d6e7fc700 1 -- 172.21.15.31:0/2303048845 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000e040 con 0x56182a54ab50 2023-12-14T18:34:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d6e7fc700 1 -- 172.21.15.31:0/2303048845 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d80007b50 con 0x56182a54ab50 2023-12-14T18:34:00.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.414+0000 7f0d6e7fc700 1 -- 172.21.15.31:0/2303048845 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8000bc50 con 0x56182a54ab50 2023-12-14T18:34:00.419 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:34:00.419 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.pool_name dir/file 2023-12-14T18:34:00.419 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:34:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2303048845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 msgr2=0x56182a54cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 0x56182a54cf30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009420 tx=0x7f0d80009450 comp rx=0 tx=0).stop 2023-12-14T18:34:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2303048845 shutdown_connections 2023-12-14T18:34:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a54ab50 0x56182a54cf30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53f380 0x56182a53f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2303048845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a54a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2303048845 >> 172.21.15.31:0/2303048845 conn(0x56182a535ca0 msgr2=0x56182a53b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2303048845 shutdown_connections 2023-12-14T18:34:00.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.415+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2303048845 wait complete. 2023-12-14T18:34:00.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 -- start start 2023-12-14T18:34:00.421 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:34:00.421 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.stripe_unit dir/file 2023-12-14T18:34:00.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 Processor -- start 2023-12-14T18:34:00.422 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a54a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a547820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a547e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a42df90 con 0x56182a53fc90 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a42dc30 con 0x56182a54ab50 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a3156a0 con 0x56182a53f380 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.416+0000 7f0d9a6c4b80 10 client.0 _ll_register_callbacks cb 0x56182a451910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:00.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a54a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a547820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a54a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:59256/0 (socket says 172.21.15.31:59256) 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a547820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38352/0 (socket says 172.21.15.31:38352) 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 -- 172.21.15.31:0/2770915703 learned_addr learned my addr 172.21.15.31:0/2770915703 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a547e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 -- 172.21.15.31:0/2770915703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 msgr2=0x56182a547e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a547e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 -- 172.21.15.31:0/2770915703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 msgr2=0x56182a547820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a547820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 -- 172.21.15.31:0/2770915703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88068d20 con 0x56182a53f380 2023-12-14T18:34:00.424 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:34:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a547820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d87fff700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a54a570 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d78099da0 tx=0x7f0d7800ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7ae6cacba46abf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d677fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:34:00.425 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.stripe_count dir/file 2023-12-14T18:34:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.417+0000 7f0d677fe700 1 -- 172.21.15.31:0/2770915703 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780cade0 con 0x56182a53f380 2023-12-14T18:34:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d677fe700 1 -- 172.21.15.31:0/2770915703 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0d780a2260 con 0x56182a53f380 2023-12-14T18:34:00.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56182a5332b0 con 0x56182a53f380 2023-12-14T18:34:00.426 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d677fe700 1 -- 172.21.15.31:0/2770915703 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d7809a070 con 0x56182a53f380 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 10 client.26731 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56182a458170 con 0x56182a53f380 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d677fe700 1 -- 172.21.15.31:0/2770915703 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56182a5332b0 con 0x56182a53f380 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d677fe700 1 -- 172.21.15.31:0/2770915703 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0d780b7fb0 con 0x56182a53f380 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 10 client.26731 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 20 client.26731 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:00.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a55a510 0x56182a55c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 4 client.26731 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56182a48ace0 con 0x56182a55a510 2023-12-14T18:34:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.418+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a55a510 0x56182a55c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.428 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:34:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.419+0000 7f0d677fe700 10 client.26731 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:00.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.419+0000 7f0d877fe700 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a55a510 0x56182a55c920 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da591c8bb76154ee server_cookie=dd086f16354c1338 in_seq=0 out_seq=0 2023-12-14T18:34:00.429 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.object_size dir/file 2023-12-14T18:34:00.429 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.419+0000 7f0d677fe700 1 -- 172.21.15.31:0/2770915703 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56182a48ace0 con 0x56182a55a510 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.419+0000 7f0d677fe700 10 client.26731 handle_command_reply: tid=0 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.419+0000 7f0d9a6c4b80 1 client.26731 shutdown 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 msgr2=0x56182a54a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a54a570 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d78099da0 tx=0x7f0d7800ce60 comp rx=0 tx=0).stop 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 shutdown_connections 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a55a510 0x56182a55c920 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a547e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a54a570 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/2770915703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53fc90 0x56182a547820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d677fe700 0 client.26731 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:00.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 >> 172.21.15.31:0/2770915703 conn(0x56182a535ca0 msgr2=0x56182a53b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 shutdown_connections 2023-12-14T18:34:00.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/2770915703 wait complete. 2023-12-14T18:34:00.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.420+0000 7f0d9a6c4b80 20 client.26731 trim_cache size 0 max 16384 2023-12-14T18:34:00.431 INFO:tasks.workunit.client.0.smithi031.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:34:00.432 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.file.layout.pool_namespace dir/file 2023-12-14T18:34:00.432 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout.pool_namespace dir 2023-12-14T18:34:00.432 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.432 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q -v pool_namespace=dirns 2023-12-14T18:34:00.432 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -x ceph.dir.layout dir 2023-12-14T18:34:00.435 INFO:tasks.workunit.client.0.smithi031.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:34:00.435 INFO:tasks.workunit.client.0.smithi031.stderr:+ grep -q 'No such attribute' 2023-12-14T18:34:00.436 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfattr -n ceph.dir.layout . 2023-12-14T18:34:00.436 INFO:tasks.workunit.client.0.smithi031.stderr:++ sed -n 2,2p 2023-12-14T18:34:00.437 INFO:tasks.workunit.client.0.smithi031.stderr:.: ceph.dir.layout: No such attribute 2023-12-14T18:34:00.438 INFO:tasks.workunit.client.0.smithi031.stderr:+ DEFAULT_LAYOUT= 2023-12-14T18:34:00.438 INFO:tasks.workunit.client.0.smithi031.stderr:+ setfattr -n ceph.dir.layout -v 'stripe_unit=131072 stripe_count=2 object_size=131072' . 2023-12-14T18:34:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 Processor -- start 2023-12-14T18:34:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 -- start start 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a551630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53fc90 0x56182a54a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 0x56182a54a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a540330 con 0x56182a54ab50 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a545d80 con 0x56182a53fc90 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a53a5b0 con 0x56182a53f380 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d8cad3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 0x56182a54a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a551630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d877fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53fc90 0x56182a54a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a551630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:59268/0 (socket says 172.21.15.31:59268) 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d8cad3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 0x56182a54a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38356/0 (socket says 172.21.15.31:38356) 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 -- 172.21.15.31:0/1112763131 learned_addr learned my addr 172.21.15.31:0/1112763131 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 -- 172.21.15.31:0/1112763131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53fc90 msgr2=0x56182a54a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53fc90 0x56182a54a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 -- 172.21.15.31:0/1112763131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 msgr2=0x56182a54a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 0x56182a54a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d87fff700 1 -- 172.21.15.31:0/1112763131 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56182a457ec0 con 0x56182a53f380 2023-12-14T18:34:00.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.442+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 0x56182a54a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:00.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.443+0000 7f0d87fff700 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a551630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005950 tx=0x7f0d78099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=864d2c414ff66317 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.443+0000 7f0d6effd700 1 -- 172.21.15.31:0/1112763131 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780a20a0 con 0x56182a53f380 2023-12-14T18:34:00.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.443+0000 7f0d6effd700 1 -- 172.21.15.31:0/1112763131 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d780a2260 con 0x56182a53f380 2023-12-14T18:34:00.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.443+0000 7f0d6effd700 1 -- 172.21.15.31:0/1112763131 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7809a070 con 0x56182a53f380 2023-12-14T18:34:00.445 INFO:tasks.workunit.client.0.smithi031.stderr:++ sed -n 2,2p 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/1112763131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 msgr2=0x56182a551630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a551630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0d78005950 tx=0x7f0d78099d10 comp rx=0 tx=0).stop 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/1112763131 shutdown_connections 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a54ab50 0x56182a54a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53f380 0x56182a551630 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/1112763131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a53fc90 0x56182a54a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/1112763131 >> 172.21.15.31:0/1112763131 conn(0x56182a535ca0 msgr2=0x56182a5567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.447 INFO:tasks.workunit.client.0.smithi031.stderr:++ getfattr -n ceph.dir.layout . 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/1112763131 shutdown_connections 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/1112763131 wait complete. 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.444+0000 7f0d9a6c4b80 1 -- start start 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 Processor -- start 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 0x56182a54f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53fc90 0x56182a54fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a5ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a5ad930 con 0x56182a53f380 2023-12-14T18:34:00.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a5ada70 con 0x56182a54ab50 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56182a5bebc0 con 0x56182a53fc90 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d9a6c4b80 10 client.0 _ll_register_callbacks cb 0x7f0d78099a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 0x56182a54f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 0x56182a54f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38366/0 (socket says 172.21.15.31:38366) 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d87fff700 1 -- 172.21.15.31:0/3944437215 learned_addr learned my addr 172.21.15.31:0/3944437215 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.445+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a5ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d877fe700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53fc90 0x56182a54fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d87fff700 1 -- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53fc90 msgr2=0x56182a54fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d87fff700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53fc90 0x56182a54fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d87fff700 1 -- 172.21.15.31:0/3944437215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 msgr2=0x56182a5ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d87fff700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a5ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d87fff700 1 -- 172.21.15.31:0/3944437215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56182a3133b0 con 0x56182a53f380 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d8cad3700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a5ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d87fff700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 0x56182a54f800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0d78099b10 tx=0x7f0d78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fffd5c18a6a212 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d677fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d677fe700 1 -- 172.21.15.31:0/3944437215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7809a270 con 0x56182a53f380 2023-12-14T18:34:00.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d677fe700 1 -- 172.21.15.31:0/3944437215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0d78099300 con 0x56182a53f380 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d80007b50 con 0x56182a53f380 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d677fe700 1 -- 172.21.15.31:0/3944437215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d780b8170 con 0x56182a53f380 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d9a6c4b80 10 client.16899 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.446+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56182a458170 con 0x56182a53f380 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d677fe700 1 -- 172.21.15.31:0/3944437215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d80007b50 con 0x56182a53f380 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d677fe700 1 -- 172.21.15.31:0/3944437215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0d78083250 con 0x56182a53f380 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d9a6c4b80 10 client.16899 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d9a6c4b80 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d9a6c4b80 20 client.16899 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a5bfea0 0x56182a5c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d9a6c4b80 4 client.16899 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56182a48aa00 con 0x56182a5bfea0 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d877fe700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a5bfea0 0x56182a5c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d877fe700 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a5bfea0 0x56182a5c22b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f26fae913aa7604 server_cookie=4dbae9991d92278e in_seq=0 out_seq=0 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.447+0000 7f0d677fe700 10 client.16899 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:00.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d677fe700 1 -- 172.21.15.31:0/3944437215 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56182a48aa00 con 0x56182a5bfea0 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d677fe700 10 client.16899 handle_command_reply: tid=0 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 client.16899 shutdown 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 msgr2=0x56182a54f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 0x56182a54f800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0d78099b10 tx=0x7f0d78004770 comp rx=0 tx=0).stop 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 shutdown_connections 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56182a5bfea0 0x56182a5c22b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56182a54ab50 0x56182a5ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56182a53f380 0x56182a54f800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 --2- 172.21.15.31:0/3944437215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56182a53fc90 0x56182a54fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 >> 172.21.15.31:0/3944437215 conn(0x56182a535ca0 msgr2=0x56182a542b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d677fe700 0 client.16899 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 shutdown_connections 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 1 -- 172.21.15.31:0/3944437215 wait complete. 2023-12-14T18:34:00.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.448+0000 7f0d9a6c4b80 20 client.16899 trim_cache size 0 max 16384 2023-12-14T18:34:00.454 INFO:tasks.workunit.client.0.smithi031.stderr:+ CHANGED_LAYOUT='ceph.dir.layout="stripe_unit=131072 stripe_count=2 object_size=131072 pool=cephfs_data_ec"' 2023-12-14T18:34:00.454 INFO:tasks.workunit.client.0.smithi031.stderr:+ '[' 'ceph.dir.layout="stripe_unit=131072 stripe_count=2 object_size=131072 pool=cephfs_data_ec"' == 'ceph.dir.layout="stripe_unit=131072 stripe_count=2 object_size=131072 pool=cephfs.a.data"' ']' 2023-12-14T18:34:00.454 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:34:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:34:00.454 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:34:00.454 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:34:00.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:34:00.455 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-12-14T18:34:00.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0d700a1340 msgr2=0x7f0d700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0d700a1340 0x7f0d700a37f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0d88197aa0 tx=0x7f0d7c0073d0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 msgr2=0x7f0d88196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d88196490 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800e9c0 tx=0x7f0d7800ecd0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 shutdown_connections 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0d700a1340 0x7f0d700a37f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0d881043f0 0x7f0d8819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d880ff400 0x7f0d88196490 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 --2- 172.21.15.31:0/2132901289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d880ffda0 0x7f0d881969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.449+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 >> 172.21.15.31:0/2132901289 conn(0x7f0d880761c0 msgr2=0x7f0d880fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.450+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 shutdown_connections 2023-12-14T18:34:00.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.450+0000 7f0d8e536700 1 -- 172.21.15.31:0/2132901289 wait complete. 2023-12-14T18:34:00.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:00 smithi031 ceph-mon[42085]: pgmap v188: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 64 KiB/s wr, 104 op/s 2023-12-14T18:34:00.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:34:00.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:34:00.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:00 smithi176 ceph-mon[43506]: pgmap v188: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 64 KiB/s wr, 104 op/s 2023-12-14T18:34:00.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:00 smithi204 ceph-mon[43866]: pgmap v188: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 64 KiB/s wr, 104 op/s 2023-12-14T18:34:00.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/2948756293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069100 msgr2=0x7fe2e40694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/2948756293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069100 0x7fe2e40694d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40099c0 tx=0x7fe2d4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/2948756293 shutdown_connections 2023-12-14T18:34:00.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/2948756293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2e4105510 0x7fe2e410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/2948756293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069100 0x7fe2e40694d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/2948756293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/2948756293 >> 172.21.15.31:0/2948756293 conn(0x7fe2e4075fa0 msgr2=0x7fe2e40763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/2948756293 shutdown_connections 2023-12-14T18:34:00.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.648+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/2948756293 wait complete. 2023-12-14T18:34:00.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 Processor -- start 2023-12-14T18:34:00.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 -- start start 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2e4069100 0x7fe2e4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2e4105510 0x7fe2e419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e419b320 con 0x7fe2e4069aa0 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e419b490 con 0x7fe2e4069100 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2ea7b2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e419b650 con 0x7fe2e4105510 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38388/0 (socket says 172.21.15.31:38388) 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e37fe700 1 -- 172.21.15.31:0/3922927785 learned_addr learned my addr 172.21.15.31:0/3922927785 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e3fff700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2e4069100 0x7fe2e4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e8d4f700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2e4105510 0x7fe2e419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e37fe700 1 -- 172.21.15.31:0/3922927785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2e4105510 msgr2=0x7fe2e419ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e37fe700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2e4105510 0x7fe2e419ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.649+0000 7fe2e37fe700 1 -- 172.21.15.31:0/3922927785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2e4069100 msgr2=0x7fe2e4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2e37fe700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2e4069100 0x7fe2e4196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2e37fe700 1 -- 172.21.15.31:0/3922927785 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40095f0 con 0x7fe2e4069aa0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2e37fe700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4196880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e940 tx=0x7fe2d800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8009730 con 0x7fe2e4069aa0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e41102d0 con 0x7fe2e4069aa0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2d800ee90 con 0x7fe2e4069aa0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8010ea0 con 0x7fe2e4069aa0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.650+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e4110140 con 0x7fe2e4069aa0 2023-12-14T18:34:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.651+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e4110460 con 0x7fe2e4069aa0 2023-12-14T18:34:00.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.653+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe2d800cbd0 con 0x7fe2e4069aa0 2023-12-14T18:34:00.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.654+0000 7fe2e17fa700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe2cc0a12c0 0x7fe2cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.654+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7fe2d8014070 con 0x7fe2e4069aa0 2023-12-14T18:34:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.654+0000 7fe2e3fff700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe2cc0a12c0 0x7fe2cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:00.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.654+0000 7fe2e3fff700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe2cc0a12c0 0x7fe2cc0a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009990 tx=0x7fe2d4005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:00.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.657+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe2d80967c0 con 0x7fe2e4069aa0 2023-12-14T18:34:00.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.917+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2e419bb20 con 0x7fe2e4069aa0 2023-12-14T18:34:00.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.918+0000 7fe2e17fa700 1 -- 172.21.15.31:0/3922927785 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10822 (secure 0 0 0) 0x7fe2d8099de0 con 0x7fe2e4069aa0 2023-12-14T18:34:00.919 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:34:00.919 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:31:40.963020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:34:00.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe2cc0a12c0 msgr2=0x7fe2cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe2cc0a12c0 0x7fe2cc0a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009990 tx=0x7fe2d4005fb0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 msgr2=0x7fe2e4196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4196880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe2d800e940 tx=0x7fe2d800ec50 comp rx=0 tx=0).stop 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 shutdown_connections 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe2cc0a12c0 0x7fe2cc0a3770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2e4105510 0x7fe2e419ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe2e4069100 0x7fe2e4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 --2- 172.21.15.31:0/3922927785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2e4069aa0 0x7fe2e4196880 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 >> 172.21.15.31:0/3922927785 conn(0x7fe2e4075fa0 msgr2=0x7fe2e4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 shutdown_connections 2023-12-14T18:34:00.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:00.920+0000 7fe2ea7b2700 1 -- 172.21.15.31:0/3922927785 wait complete. 2023-12-14T18:34:00.923 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-12-14T18:34:00.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 -- 172.21.15.31:0/3719378298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 msgr2=0x7f0074104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 --2- 172.21.15.31:0/3719378298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074104e10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f00640099c0 tx=0x7f0064009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 -- 172.21.15.31:0/3719378298 shutdown_connections 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 --2- 172.21.15.31:0/3719378298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00741054f0 0x7f007410d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 --2- 172.21.15.31:0/3719378298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00741004b0 0x7f0074100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 --2- 172.21.15.31:0/3719378298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074104e10 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.075+0000 7f007b3b7700 1 -- 172.21.15.31:0/3719378298 >> 172.21.15.31:0/3719378298 conn(0x7f00740fbd40 msgr2=0x7f00740fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.076+0000 7f007b3b7700 1 -- 172.21.15.31:0/3719378298 shutdown_connections 2023-12-14T18:34:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.076+0000 7f007b3b7700 1 -- 172.21.15.31:0/3719378298 wait complete. 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.076+0000 7f007b3b7700 1 Processor -- start 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.076+0000 7f007b3b7700 1 -- start start 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00741004b0 0x7f00741963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00741054f0 0x7f007419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419b3d0 con 0x7f0074100e50 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419b540 con 0x7f00741004b0 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007419b700 con 0x7f00741054f0 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38400/0 (socket says 172.21.15.31:38400) 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 -- 172.21.15.31:0/436117470 learned_addr learned my addr 172.21.15.31:0/436117470 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0079954700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00741054f0 0x7f007419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0079153700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00741004b0 0x7f00741963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 -- 172.21.15.31:0/436117470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00741054f0 msgr2=0x7f007419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00741054f0 0x7f007419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 -- 172.21.15.31:0/436117470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00741004b0 msgr2=0x7f00741963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00741004b0 0x7f00741963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 -- 172.21.15.31:0/436117470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00640095f0 con 0x7f0074100e50 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f0078952700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074196930 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0064009fd0 tx=0x7f0064009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f006a7fc700 1 -- 172.21.15.31:0/436117470 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006401d070 con 0x7f0074100e50 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f006a7fc700 1 -- 172.21.15.31:0/436117470 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00640054e0 con 0x7f0074100e50 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f006a7fc700 1 -- 172.21.15.31:0/436117470 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064004b10 con 0x7f0074100e50 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.077+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0074110270 con 0x7f0074100e50 2023-12-14T18:34:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.078+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741100e0 con 0x7f0074100e50 2023-12-14T18:34:01.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.080+0000 7f006a7fc700 1 -- 172.21.15.31:0/436117470 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f006400fbc0 con 0x7f0074100e50 2023-12-14T18:34:01.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.081+0000 7f006a7fc700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f00600a12c0 0x7f00600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.081+0000 7f006a7fc700 1 -- 172.21.15.31:0/436117470 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f00640cdcc0 con 0x7f0074100e50 2023-12-14T18:34:01.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.081+0000 7f0079153700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f00600a12c0 0x7f00600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.081+0000 7f0079153700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f00600a12c0 0x7f00600a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0070005950 tx=0x7f00700058e0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 Processor -- start 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 -- start start 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4dab750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4c955e0 con 0x5649a4dabc90 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4d04870 con 0x5649a4db6b50 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0087545b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4cfdd00 con 0x5649a4dab380 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0078952700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0079153700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4dab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0078952700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38408/0 (socket says 172.21.15.31:38408) 2023-12-14T18:34:01.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.082+0000 7f0078952700 1 -- 172.21.15.31:0/2346054964 learned_addr learned my addr 172.21.15.31:0/2346054964 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0079954700 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0078952700 1 -- 172.21.15.31:0/2346054964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 msgr2=0x5649a4dab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0078952700 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4dab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0078952700 1 -- 172.21.15.31:0/2346054964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 msgr2=0x5649a4db8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0078952700 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0078952700 1 -- 172.21.15.31:0/2346054964 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a4b7f3b0 con 0x5649a4dabc90 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0079954700 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0079153700 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4dab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f0078952700 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db6610 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0064096f90 tx=0x7f0064096f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc20349c1c3ebd06 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f005f7fe700 1 -- 172.21.15.31:0/2346054964 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006409eac0 con 0x5649a4dabc90 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f005f7fe700 1 -- 172.21.15.31:0/2346054964 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00640a2e50 con 0x5649a4dabc90 2023-12-14T18:34:01.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.083+0000 7f005f7fe700 1 -- 172.21.15.31:0/2346054964 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064099f80 con 0x5649a4dabc90 2023-12-14T18:34:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 -- 172.21.15.31:0/2346054964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 msgr2=0x5649a4db6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db6610 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0064096f90 tx=0x7f0064096f00 comp rx=0 tx=0).stop 2023-12-14T18:34:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 -- 172.21.15.31:0/2346054964 shutdown_connections 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4dab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 --2- 172.21.15.31:0/2346054964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db6610 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 -- 172.21.15.31:0/2346054964 >> 172.21.15.31:0/2346054964 conn(0x5649a4da1ca0 msgr2=0x5649a4da77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 -- 172.21.15.31:0/2346054964 shutdown_connections 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 -- 172.21.15.31:0/2346054964 wait complete. 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.084+0000 7f0087545b80 1 -- start start 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 Processor -- start 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4db7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 0x5649a4db79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4c99f90 con 0x5649a4db6b50 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4c99c30 con 0x5649a4dabc90 2023-12-14T18:34:01.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4b816a0 con 0x5649a4dab380 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0087545b80 10 client.0 _ll_register_callbacks cb 0x5649a4cbd910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0079954700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 0x5649a4db79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0079954700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 0x5649a4db79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38422/0 (socket says 172.21.15.31:38422) 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0079954700 1 -- 172.21.15.31:0/3492661504 learned_addr learned my addr 172.21.15.31:0/3492661504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0079153700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.085+0000 7f0078952700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4db7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079954700 1 -- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 msgr2=0x5649a4db6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079954700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079954700 1 -- 172.21.15.31:0/3492661504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 msgr2=0x5649a4db7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079954700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4db7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079954700 1 -- 172.21.15.31:0/3492661504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00741100e0 con 0x5649a4db6b50 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079153700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0079954700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 0x5649a4db79a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f006c009550 tx=0x7f006c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f01f1f3b5609076 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f005cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f005cff9700 1 -- 172.21.15.31:0/3492661504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00e070 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f005cff9700 1 -- 172.21.15.31:0/3492661504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f006c012ac0 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649a4d9f2b0 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f005cff9700 1 -- 172.21.15.31:0/3492661504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006c012c80 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0087545b80 10 client.16917 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5649a4cc4170 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.086+0000 7f005cff9700 1 -- 172.21.15.31:0/3492661504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649a4d9f2b0 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f005cff9700 1 -- 172.21.15.31:0/3492661504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f006c014040 con 0x5649a4db6b50 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0087545b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0087545b80 10 client.16917 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0087545b80 20 client.16917 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0087545b80 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4dc61e0 0x5649a4dc85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0087545b80 4 client.16917 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649a4cf6ce0 con 0x5649a4dc61e0 2023-12-14T18:34:01.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.087+0000 7f0079153700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4dc61e0 0x5649a4dc85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.088+0000 7f005cff9700 10 client.16917 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.088+0000 7f0079153700 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4dc61e0 0x5649a4dc85f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7bd004a73cb9268 server_cookie=6c607b8b331d8e97 in_seq=0 out_seq=0 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.088+0000 7f005cff9700 1 -- 172.21.15.31:0/3492661504 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649a4cf6ce0 con 0x5649a4dc61e0 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.088+0000 7f005cff9700 10 client.16917 handle_command_reply: tid=0 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.088+0000 7f0087545b80 1 client.16917 shutdown 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 msgr2=0x5649a4db79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 0x5649a4db79a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f006c009550 tx=0x7f006c007e30 comp rx=0 tx=0).stop 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 shutdown_connections 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4dc61e0 0x5649a4dc85f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4db6b50 0x5649a4db79a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f005cff9700 0 client.16917 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 --2- 172.21.15.31:0/3492661504 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4db7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 >> 172.21.15.31:0/3492661504 conn(0x5649a4da1ca0 msgr2=0x5649a4da7370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 shutdown_connections 2023-12-14T18:34:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 1 -- 172.21.15.31:0/3492661504 wait complete. 2023-12-14T18:34:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.089+0000 7f0087545b80 20 client.16917 trim_cache size 0 max 16384 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 Processor -- start 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 -- start start 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 0x5649a4db0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4dbaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4db6b50 0x5649a4db5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4dae270 con 0x5649a4dab380 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4db6990 con 0x5649a4dabc90 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0087545b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4dc37b0 con 0x5649a4db6b50 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.103+0000 7f0079153700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 0x5649a4db0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 0x5649a4db0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38434/0 (socket says 172.21.15.31:38434) 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 -- 172.21.15.31:0/1533366683 learned_addr learned my addr 172.21.15.31:0/1533366683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079954700 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4db6b50 0x5649a4db5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0078952700 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4dbaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 -- 172.21.15.31:0/1533366683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4db6b50 msgr2=0x5649a4db5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4db6b50 0x5649a4db5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 -- 172.21.15.31:0/1533366683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 msgr2=0x5649a4dbaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4dbaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 -- 172.21.15.31:0/1533366683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a4db6740 con 0x5649a4dab380 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079954700 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4db6b50 0x5649a4db5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f0079153700 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 0x5649a4db0560 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f007000ba40 tx=0x7f0070012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d84fb6a951adb0d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f005ffff700 1 -- 172.21.15.31:0/1533366683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070024040 con 0x5649a4dab380 2023-12-14T18:34:01.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f005ffff700 1 -- 172.21.15.31:0/1533366683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0070011900 con 0x5649a4dab380 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.104+0000 7f005ffff700 1 -- 172.21.15.31:0/1533366683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007001f8d0 con 0x5649a4dab380 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 -- 172.21.15.31:0/1533366683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 msgr2=0x5649a4db0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 0x5649a4db0560 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f007000ba40 tx=0x7f0070012000 comp rx=0 tx=0).stop 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 -- 172.21.15.31:0/1533366683 shutdown_connections 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4db6b50 0x5649a4db5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dab380 0x5649a4db0560 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 --2- 172.21.15.31:0/1533366683 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4dabc90 0x5649a4dbaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 -- 172.21.15.31:0/1533366683 >> 172.21.15.31:0/1533366683 conn(0x5649a4da1ca0 msgr2=0x5649a4db0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 -- 172.21.15.31:0/1533366683 shutdown_connections 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 -- 172.21.15.31:0/1533366683 wait complete. 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.105+0000 7f0087545b80 1 -- start start 2023-12-14T18:34:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 Processor -- start 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4e19190 con 0x5649a4dabc90 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4e19300 con 0x5649a4db6b50 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a4e194c0 con 0x5649a4dab380 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0087545b80 10 client.0 _ll_register_callbacks cb 0x7f006c007500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0078952700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0078952700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38440/0 (socket says 172.21.15.31:38440) 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0078952700 1 -- 172.21.15.31:0/1612537920 learned_addr learned my addr 172.21.15.31:0/1612537920 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0079954700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.106+0000 7f0079153700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0078952700 1 -- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 msgr2=0x5649a4db87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0078952700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0078952700 1 -- 172.21.15.31:0/1612537920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 msgr2=0x5649a4db9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0078952700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0078952700 1 -- 172.21.15.31:0/1612537920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649a4cc4170 con 0x5649a4dabc90 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0078952700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db8d00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f00640cb130 tx=0x7f006409a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562b53fc9481db79 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f005cff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f005cff9700 1 -- 172.21.15.31:0/1612537920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0064099f80 con 0x5649a4dabc90 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f005cff9700 1 -- 172.21.15.31:0/1612537920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f006409a0e0 con 0x5649a4dabc90 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00640a2e50 con 0x5649a4dabc90 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f005cff9700 1 -- 172.21.15.31:0/1612537920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006409acf0 con 0x5649a4dabc90 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0087545b80 10 client.16923 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f007000f980 con 0x5649a4dabc90 2023-12-14T18:34:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.107+0000 7f005cff9700 1 -- 172.21.15.31:0/1612537920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00640a2e50 con 0x5649a4dabc90 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f005cff9700 1 -- 172.21.15.31:0/1612537920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f00640999f0 con 0x5649a4dabc90 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 10 client.16923 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 10 client.16923 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 20 client.16923 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4e1a890 0x5649a4e1cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 4 client.16923 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5649a4cf6a00 con 0x5649a4e1a890 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0079153700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4e1a890 0x5649a4e1cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0079153700 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4e1a890 0x5649a4e1cca0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab150dc7ba25ced8 server_cookie=64d62de33f661c4b in_seq=0 out_seq=0 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f005cff9700 10 client.16923 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f005cff9700 1 -- 172.21.15.31:0/1612537920 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5649a4cf6a00 con 0x5649a4e1a890 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f005cff9700 10 client.16923 handle_command_reply: tid=0 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.108+0000 7f0087545b80 1 client.16923 shutdown 2023-12-14T18:34:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 msgr2=0x5649a4db8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db8d00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f00640cb130 tx=0x7f006409a4f0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 shutdown_connections 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5649a4e1a890 0x5649a4e1cca0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5649a4db6b50 0x5649a4db9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5649a4dab380 0x5649a4db87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f005cff9700 0 client.16923 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 --2- 172.21.15.31:0/1612537920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5649a4dabc90 0x5649a4db8d00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 >> 172.21.15.31:0/1612537920 conn(0x5649a4da1ca0 msgr2=0x5649a4dad260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 shutdown_connections 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 1 -- 172.21.15.31:0/1612537920 wait complete. 2023-12-14T18:34:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f0087545b80 20 client.16923 trim_cache size 0 max 16384 2023-12-14T18:34:01.117 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:34:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f00600a12c0 msgr2=0x7f00600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f00600a12c0 0x7f00600a3770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0070005950 tx=0x7f00700058e0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 msgr2=0x7f0074196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074196930 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0064009fd0 tx=0x7f0064009580 comp rx=0 tx=0).stop 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 shutdown_connections 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f00600a12c0 0x7f00600a3770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00741054f0 0x7f007419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00741004b0 0x7f00741963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 --2- 172.21.15.31:0/436117470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0074100e50 0x7f0074196930 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.109+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 >> 172.21.15.31:0/436117470 conn(0x7f00740fbd40 msgr2=0x7f00740fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.110+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 shutdown_connections 2023-12-14T18:34:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.110+0000 7f007b3b7700 1 -- 172.21.15.31:0/436117470 wait complete. 2023-12-14T18:34:01.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:34:01.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:01 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3922927785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:34:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 -- 172.21.15.31:0/4218676150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c01043d0 msgr2=0x7f89c010bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 --2- 172.21.15.31:0/4218676150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c01043d0 0x7f89c010bfa0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f89b800b0d0 tx=0x7f89b800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 -- 172.21.15.31:0/4218676150 shutdown_connections 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 --2- 172.21.15.31:0/4218676150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c01043d0 0x7f89c010bfa0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 --2- 172.21.15.31:0/4218676150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 0x7f89c00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 --2- 172.21.15.31:0/4218676150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c00ffd30 0x7f89c0103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 -- 172.21.15.31:0/4218676150 >> 172.21.15.31:0/4218676150 conn(0x7f89c0075fd0 msgr2=0x7f89c00763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 -- 172.21.15.31:0/4218676150 shutdown_connections 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 -- 172.21.15.31:0/4218676150 wait complete. 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.266+0000 7f89c648c700 1 Processor -- start 2023-12-14T18:34:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 -- start start 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 0x7f89c01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 0x7f89c0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c01043d0 0x7f89c019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019c590 con 0x7f89c00ffd30 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019c700 con 0x7f89c01043d0 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c648c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c019c8c0 con 0x7f89c00ff390 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 0x7f89c0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 0x7f89c0198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38472/0 (socket says 172.21.15.31:38472) 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1148540903 learned_addr learned my addr 172.21.15.31:0/1148540903 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bffff700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 0x7f89c01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89c4a29700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c01043d0 0x7f89c019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1148540903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 msgr2=0x7f89c01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 0x7f89c01986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1148540903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c01043d0 msgr2=0x7f89c019bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c01043d0 0x7f89c019bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1148540903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b8009d00 con 0x7f89c00ffd30 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.267+0000 7f89bffff700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 0x7f89c01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.268+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 0x7f89c0198be0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f89b400e940 tx=0x7f89b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.268+0000 7f89bd7fa700 1 -- 172.21.15.31:0/1148540903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4009730 con 0x7f89c00ffd30 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.268+0000 7f89bd7fa700 1 -- 172.21.15.31:0/1148540903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b400cbd0 con 0x7f89c00ffd30 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.268+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c010f1a0 con 0x7f89c00ffd30 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.268+0000 7f89bd7fa700 1 -- 172.21.15.31:0/1148540903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4018680 con 0x7f89c00ffd30 2023-12-14T18:34:01.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.268+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c010f010 con 0x7f89c00ffd30 2023-12-14T18:34:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.270+0000 7f89bd7fa700 1 -- 172.21.15.31:0/1148540903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f89b400cd40 con 0x7f89c00ffd30 2023-12-14T18:34:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.271+0000 7f89bd7fa700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89a80a12c0 0x7f89a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.271+0000 7f89bffff700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89a80a12c0 0x7f89a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.271+0000 7f89bd7fa700 1 -- 172.21.15.31:0/1148540903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f89b4014070 con 0x7f89c00ffd30 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.271+0000 7f89bffff700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89a80a12c0 0x7f89a80a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89b000bc10 tx=0x7f89b000b5f0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 Processor -- start 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 -- start start 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360160b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x563601616610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x563601618f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636014f55e0 con 0x56360160bc90 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563601564870 con 0x56360160b380 2023-12-14T18:34:01.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89d261ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360155dd00 con 0x563601616b50 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x563601616610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x563601616610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38488/0 (socket says 172.21.15.31:38488) 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1661200743 learned_addr learned my addr 172.21.15.31:0/1661200743 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bffff700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360160b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89c4a29700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x563601618f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1661200743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 msgr2=0x563601618f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x563601618f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1661200743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 msgr2=0x56360160b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360160b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.272+0000 7f89bf7fe700 1 -- 172.21.15.31:0/1661200743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636013df3b0 con 0x56360160bc90 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.273+0000 7f89c4a29700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x563601618f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.273+0000 7f89bffff700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360160b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.273+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x563601616610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f89b4096b00 tx=0x7f89b40999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd0a776b8051f58 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.273+0000 7f89a67fc700 1 -- 172.21.15.31:0/1661200743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b409e0c0 con 0x56360160bc90 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.273+0000 7f89a67fc700 1 -- 172.21.15.31:0/1661200743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b40a24e0 con 0x56360160bc90 2023-12-14T18:34:01.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.273+0000 7f89a67fc700 1 -- 172.21.15.31:0/1661200743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40a2050 con 0x56360160bc90 2023-12-14T18:34:01.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 -- 172.21.15.31:0/1661200743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 msgr2=0x563601616610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x563601616610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f89b4096b00 tx=0x7f89b40999f0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 -- 172.21.15.31:0/1661200743 shutdown_connections 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x563601618f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360160b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 --2- 172.21.15.31:0/1661200743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x563601616610 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 -- 172.21.15.31:0/1661200743 >> 172.21.15.31:0/1661200743 conn(0x563601601ca0 msgr2=0x5636016077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 -- 172.21.15.31:0/1661200743 shutdown_connections 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 -- 172.21.15.31:0/1661200743 wait complete. 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.274+0000 7f89d261ab80 1 -- start start 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 Processor -- start 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 0x563601616580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160bc90 0x563601613830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563601616b50 0x563601613f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636014f9f90 con 0x56360160b380 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636014f9c30 con 0x563601616b50 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636013e16a0 con 0x56360160bc90 2023-12-14T18:34:01.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89d261ab80 10 client.0 _ll_register_callbacks cb 0x56360151d910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 0x563601616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 0x563601616580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38490/0 (socket says 172.21.15.31:38490) 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 -- 172.21.15.31:0/812452391 learned_addr learned my addr 172.21.15.31:0/812452391 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89c4a29700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563601616b50 0x563601613f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160bc90 0x563601613830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 -- 172.21.15.31:0/812452391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160bc90 msgr2=0x563601613830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160bc90 0x563601613830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 -- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563601616b50 msgr2=0x563601613f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563601616b50 0x563601613f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.275+0000 7f89bffff700 1 -- 172.21.15.31:0/812452391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c010f010 con 0x56360160b380 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f89bffff700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 0x563601616580 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f89b0005e10 tx=0x7f89b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d7e6e3198f9c04 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f899f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f899f7fe700 1 -- 172.21.15.31:0/812452391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b001f630 con 0x56360160b380 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f899f7fe700 1 -- 172.21.15.31:0/812452391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f89b001f790 con 0x56360160b380 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636015ff2b0 con 0x56360160b380 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f899f7fe700 1 -- 172.21.15.31:0/812452391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b000fec0 con 0x56360160b380 2023-12-14T18:34:01.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f89d261ab80 10 client.16935 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563601524170 con 0x56360160b380 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f899f7fe700 1 -- 172.21.15.31:0/812452391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636015ff2b0 con 0x56360160b380 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f899f7fe700 1 -- 172.21.15.31:0/812452391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f89b0025020 con 0x56360160b380 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.276+0000 7f89d261ab80 10 client.16935 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89d261ab80 10 client.16935 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89d261ab80 20 client.16935 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89d261ab80 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563601626650 0x563601628a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89d261ab80 4 client.16935 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563601556ce0 con 0x563601626650 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563601626650 0x563601628a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563601626650 0x563601628a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b241582ff706d1e9 server_cookie=a507438febaeac72 in_seq=0 out_seq=0 2023-12-14T18:34:01.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.277+0000 7f899f7fe700 10 client.16935 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f899f7fe700 1 -- 172.21.15.31:0/812452391 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563601556ce0 con 0x563601626650 2023-12-14T18:34:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f899f7fe700 10 client.16935 handle_command_reply: tid=0 2023-12-14T18:34:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 client.16935 shutdown 2023-12-14T18:34:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 msgr2=0x563601616580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 0x563601616580 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f89b0005e10 tx=0x7f89b0004750 comp rx=0 tx=0).stop 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 shutdown_connections 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x563601616b50 0x563601613f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160b380 0x563601616580 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x563601626650 0x563601628a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 --2- 172.21.15.31:0/812452391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160bc90 0x563601613830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 >> 172.21.15.31:0/812452391 conn(0x563601601ca0 msgr2=0x5636016073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f899f7fe700 0 client.16935 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 shutdown_connections 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 1 -- 172.21.15.31:0/812452391 wait complete. 2023-12-14T18:34:01.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.278+0000 7f89d261ab80 20 client.16935 trim_cache size 0 max 16384 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.292+0000 7f89d261ab80 1 Processor -- start 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.292+0000 7f89d261ab80 1 -- start start 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160b380 0x563601616040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160bc90 0x563601616580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 0x56360160eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89d261ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x563601614550 con 0x563601616b50 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89d261ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x563601622540 con 0x56360160bc90 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89d261ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360160f680 con 0x56360160b380 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 0x56360160eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 0x56360160eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38500/0 (socket says 172.21.15.31:38500) 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 -- 172.21.15.31:0/768845467 learned_addr learned my addr 172.21.15.31:0/768845467 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89bffff700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160b380 0x563601616040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160bc90 0x563601616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 -- 172.21.15.31:0/768845467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160b380 msgr2=0x563601616040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160b380 0x563601616040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 -- 172.21.15.31:0/768845467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160bc90 msgr2=0x563601616580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160bc90 0x563601616580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 -- 172.21.15.31:0/768845467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563601523ec0 con 0x563601616b50 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89bffff700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160b380 0x563601616040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160bc90 0x563601616580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.293+0000 7f89c4a29700 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 0x56360160eb10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f89b80093c0 tx=0x7f89b80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8526f0b86a72114f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89a6ffd700 1 -- 172.21.15.31:0/768845467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b800e040 con 0x563601616b50 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89a6ffd700 1 -- 172.21.15.31:0/768845467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b800bca0 con 0x563601616b50 2023-12-14T18:34:01.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89a6ffd700 1 -- 172.21.15.31:0/768845467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b80266e0 con 0x563601616b50 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89d261ab80 1 -- 172.21.15.31:0/768845467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 msgr2=0x56360160eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89d261ab80 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 0x56360160eb10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f89b80093c0 tx=0x7f89b80093f0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89d261ab80 1 -- 172.21.15.31:0/768845467 shutdown_connections 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89d261ab80 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x563601616b50 0x56360160eb10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.294+0000 7f89d261ab80 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56360160b380 0x563601616040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.295+0000 7f89d261ab80 1 --2- 172.21.15.31:0/768845467 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160bc90 0x563601616580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.295+0000 7f89d261ab80 1 -- 172.21.15.31:0/768845467 >> 172.21.15.31:0/768845467 conn(0x563601601ca0 msgr2=0x563601621e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.295+0000 7f89d261ab80 1 -- 172.21.15.31:0/768845467 shutdown_connections 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.295+0000 7f89d261ab80 1 -- 172.21.15.31:0/768845467 wait complete. 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.295+0000 7f89d261ab80 1 -- start start 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.295+0000 7f89d261ab80 1 Processor -- start 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360161d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x56360161d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x56360161b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360161bb40 con 0x56360160bc90 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360161bc80 con 0x56360160b380 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56360161be40 con 0x563601616b50 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89d261ab80 10 client.0 _ll_register_callbacks cb 0x7f89b000fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x56360161d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x56360161d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38512/0 (socket says 172.21.15.31:38512) 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 -- 172.21.15.31:0/118370302 learned_addr learned my addr 172.21.15.31:0/118370302 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89c4a29700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x56360161b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bffff700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360161d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 -- 172.21.15.31:0/118370302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 msgr2=0x56360161b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x56360161b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 -- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 msgr2=0x56360161d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360161d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 -- 172.21.15.31:0/118370302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563601524170 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89c4a29700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x56360161b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f89bf7fe700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x56360161d5d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f89b4099d90 tx=0x7f89b409a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106907a8fd480456 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f899f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f899f7fe700 1 -- 172.21.15.31:0/118370302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b4099780 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.296+0000 7f899f7fe700 1 -- 172.21.15.31:0/118370302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f89b409a2f0 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89b40a24e0 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f899f7fe700 1 -- 172.21.15.31:0/118370302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89b409e140 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 10 client.16941 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f89c010c7e0 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f899f7fe700 1 -- 172.21.15.31:0/118370302 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89b40a24e0 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f899f7fe700 1 -- 172.21.15.31:0/118370302 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f89b4098ff0 con 0x56360160bc90 2023-12-14T18:34:01.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 10 client.16941 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 10 client.16941 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 20 client.16941 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56360167b3d0 0x56360167d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 4 client.16941 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563601556a00 con 0x56360167b3d0 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.297+0000 7f89bffff700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56360167b3d0 0x56360167d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89bffff700 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56360167b3d0 0x56360167d7e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50718f69dd380bc1 server_cookie=1fb223f251e3ddc3 in_seq=0 out_seq=0 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f899f7fe700 10 client.16941 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f899f7fe700 1 -- 172.21.15.31:0/118370302 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563601556a00 con 0x56360167b3d0 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f899f7fe700 10 client.16941 handle_command_reply: tid=0 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 client.16941 shutdown 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 msgr2=0x56360161d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x56360161d5d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f89b4099d90 tx=0x7f89b409a110 comp rx=0 tx=0).stop 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 shutdown_connections 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x56360167b3d0 0x56360167d7e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563601616b50 0x56360161b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56360160b380 0x56360161d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f899f7fe700 0 client.16941 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 --2- 172.21.15.31:0/118370302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56360160bc90 0x56360161d5d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 >> 172.21.15.31:0/118370302 conn(0x563601601ca0 msgr2=0x56360160eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 shutdown_connections 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 1 -- 172.21.15.31:0/118370302 wait complete. 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.298+0000 7f89d261ab80 20 client.16941 trim_cache size 0 max 16384 2023-12-14T18:34:01.304 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89a80a12c0 msgr2=0x7f89a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89a80a12c0 0x7f89a80a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89b000bc10 tx=0x7f89b000b5f0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 msgr2=0x7f89c0198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 0x7f89c0198be0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f89b400e940 tx=0x7f89b400ed00 comp rx=0 tx=0).stop 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 shutdown_connections 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f89c01043d0 0x7f89c019bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89a80a12c0 0x7f89a80a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89c00ff390 0x7f89c01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 --2- 172.21.15.31:0/1148540903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89c00ffd30 0x7f89c0198be0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 >> 172.21.15.31:0/1148540903 conn(0x7f89c0075fd0 msgr2=0x7f89c0067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 shutdown_connections 2023-12-14T18:34:01.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.299+0000 7f89c648c700 1 -- 172.21.15.31:0/1148540903 wait complete. 2023-12-14T18:34:01.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:34:01.437 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/layout_vxattrs.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh' 2023-12-14T18:34:01.534 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=08caab1e99f04e72a3cdaaf5804e0c70 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/layout_vxattrs.sh) on smithi031 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh' 2023-12-14T18:34:01.536 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-12-14T18:34:01.546 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:34:01.550 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:01 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3922927785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 -- 172.21.15.31:0/38484634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ffd30 msgr2=0x7f15a0103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 --2- 172.21.15.31:0/38484634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ffd30 0x7f15a0103cf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f15880099c0 tx=0x7f1588009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 -- 172.21.15.31:0/38484634 shutdown_connections 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 --2- 172.21.15.31:0/38484634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 0x7f15a010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 --2- 172.21.15.31:0/38484634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a00ff390 0x7f15a00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 --2- 172.21.15.31:0/38484634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ffd30 0x7f15a0103cf0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 -- 172.21.15.31:0/38484634 >> 172.21.15.31:0/38484634 conn(0x7f15a0075fd0 msgr2=0x7f15a00763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.452+0000 7f15a6417700 1 -- 172.21.15.31:0/38484634 shutdown_connections 2023-12-14T18:34:01.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.453+0000 7f15a6417700 1 -- 172.21.15.31:0/38484634 wait complete. 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.453+0000 7f15a6417700 1 Processor -- start 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.453+0000 7f15a6417700 1 -- start start 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a6417700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 0x7f15a0100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a6417700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a00ffd30 0x7f15a01012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a6417700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 0x7f15a01a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a6417700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a0101b00 con 0x7f15a00ff390 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a6417700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a01a4b60 con 0x7f15a01043d0 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a6417700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a01a4ca0 con 0x7f15a00ffd30 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 0x7f15a0100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 0x7f15a0100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38522/0 (socket says 172.21.15.31:38522) 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 -- 172.21.15.31:0/3162556299 learned_addr learned my addr 172.21.15.31:0/3162556299 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159f7fe700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a00ffd30 0x7f15a01012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a49b4700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 0x7f15a01a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 -- 172.21.15.31:0/3162556299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a00ffd30 msgr2=0x7f15a01012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a00ffd30 0x7f15a01012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 -- 172.21.15.31:0/3162556299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 msgr2=0x7f15a01a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 0x7f15a01a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 -- 172.21.15.31:0/3162556299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880095f0 con 0x7f15a00ff390 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f15a49b4700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 0x7f15a01a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.454+0000 7f159ffff700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 0x7f15a0100d90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f159000e9a0 tx=0x7f159000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.455+0000 7f159d7fa700 1 -- 172.21.15.31:0/3162556299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590009790 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.455+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a010f150 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.455+0000 7f159d7fa700 1 -- 172.21.15.31:0/3162556299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159000cbd0 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.455+0000 7f159d7fa700 1 -- 172.21.15.31:0/3162556299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590018680 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.455+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a010efc0 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.457+0000 7f159d7fa700 1 -- 172.21.15.31:0/3162556299 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f159000cd40 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.457+0000 7f159d7fa700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f158c0a12c0 0x7f158c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.458+0000 7f159d7fa700 1 -- 172.21.15.31:0/3162556299 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1590014070 con 0x7f15a00ff390 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.458+0000 7f159f7fe700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f158c0a12c0 0x7f158c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.458+0000 7f159f7fe700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f158c0a12c0 0x7f158c0a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f15880098e0 tx=0x7f1588000bc0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 Processor -- start 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 -- start start 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e81610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e83f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870d605e0 con 0x55b870e76c90 2023-12-14T18:34:01.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870dcf870 con 0x55b870e76380 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870dc8d00 con 0x55b870e81b50 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e81610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e81610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38538/0 (socket says 172.21.15.31:38538) 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f159f7fe700 1 -- 172.21.15.31:0/2787050198 learned_addr learned my addr 172.21.15.31:0/2787050198 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f159ffff700 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.459+0000 7f15a49b4700 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e83f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f159f7fe700 1 -- 172.21.15.31:0/2787050198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 msgr2=0x55b870e83f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f159f7fe700 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e83f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f159f7fe700 1 -- 172.21.15.31:0/2787050198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 msgr2=0x55b870e76750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f159f7fe700 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f159f7fe700 1 -- 172.21.15.31:0/2787050198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b870c4a3b0 con 0x55b870e76c90 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f159f7fe700 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e81610 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f15880099c0 tx=0x7f158801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1b1413791d8f89 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f15967fc700 1 -- 172.21.15.31:0/2787050198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800b990 con 0x55b870e76c90 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f15967fc700 1 -- 172.21.15.31:0/2787050198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158801e040 con 0x55b870e76c90 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.460+0000 7f15967fc700 1 -- 172.21.15.31:0/2787050198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15880304c0 con 0x55b870e76c90 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2787050198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 msgr2=0x55b870e81610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e81610 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f15880099c0 tx=0x7f158801f000 comp rx=0 tx=0).stop 2023-12-14T18:34:01.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2787050198 shutdown_connections 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e83f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2787050198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e81610 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2787050198 >> 172.21.15.31:0/2787050198 conn(0x55b870e6cca0 msgr2=0x55b870e727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2787050198 shutdown_connections 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2787050198 wait complete. 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 -- start start 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.461+0000 7f15b25a5b80 1 Processor -- start 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e7e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e7ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870d64f90 con 0x55b870e76c90 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870d64c30 con 0x55b870e76380 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870c4c6a0 con 0x55b870e81b50 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15b25a5b80 10 client.0 _ll_register_callbacks cb 0x55b870d88910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e7e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e7e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38550/0 (socket says 172.21.15.31:38550) 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 -- 172.21.15.31:0/2695192146 learned_addr learned my addr 172.21.15.31:0/2695192146 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f15a49b4700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e7ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159ffff700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 -- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 msgr2=0x55b870e7ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e7ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 -- 172.21.15.31:0/2695192146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 msgr2=0x55b870e814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 -- 172.21.15.31:0/2695192146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a010c7e0 con 0x55b870e76c90 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f159f7fe700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e7e810 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1588009d10 tx=0x7f1588004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3341dc6b04d52fce server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.462+0000 7f157f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f157f7fe700 1 -- 172.21.15.31:0/2695192146 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588005610 con 0x55b870e76c90 2023-12-14T18:34:01.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f157f7fe700 1 -- 172.21.15.31:0/2695192146 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f1588030780 con 0x55b870e76c90 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b870e6a2b0 con 0x55b870e76c90 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f157f7fe700 1 -- 172.21.15.31:0/2695192146 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1588030940 con 0x55b870e76c90 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 10 client.16953 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b870d8f170 con 0x55b870e76c90 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f157f7fe700 1 -- 172.21.15.31:0/2695192146 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b870e6a2b0 con 0x55b870e76c90 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f157f7fe700 1 -- 172.21.15.31:0/2695192146 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1588004640 con 0x55b870e76c90 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 10 client.16953 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 10 client.16953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 20 client.16953 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870e910a0 0x55b870e93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 4 client.16953 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.463+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b870dc1ce0 con 0x55b870e910a0 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.464+0000 7f159ffff700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870e910a0 0x55b870e93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.464+0000 7f157f7fe700 10 client.16953 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.464+0000 7f159ffff700 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870e910a0 0x55b870e93480 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3efc05b12e0370 server_cookie=10ece1c81c02eed3 in_seq=0 out_seq=0 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f157f7fe700 1 -- 172.21.15.31:0/2695192146 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b870dc1ce0 con 0x55b870e910a0 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f157f7fe700 10 client.16953 handle_command_reply: tid=0 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 client.16953 shutdown 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 msgr2=0x55b870e7e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e7e810 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1588009d10 tx=0x7f1588004470 comp rx=0 tx=0).stop 2023-12-14T18:34:01.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 shutdown_connections 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870e910a0 0x55b870e93480 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870e7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76380 0x55b870e814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f157f7fe700 0 client.16953 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/2695192146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e7e810 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 >> 172.21.15.31:0/2695192146 conn(0x55b870e6cca0 msgr2=0x55b870e723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 shutdown_connections 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 1 -- 172.21.15.31:0/2695192146 wait complete. 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.465+0000 7f15b25a5b80 20 client.16953 trim_cache size 0 max 16384 2023-12-14T18:34:01.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 Processor -- start 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 -- start start 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e88630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e81b50 0x55b870e81020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870e77580 con 0x55b870e76c90 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870e79270 con 0x55b870e81b50 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870e88b70 con 0x55b870e76380 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e88630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f159f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e88630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38560/0 (socket says 172.21.15.31:38560) 2023-12-14T18:34:01.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.479+0000 7f159f7fe700 1 -- 172.21.15.31:0/3316132425 learned_addr learned my addr 172.21.15.31:0/3316132425 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159ffff700 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f15a49b4700 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e81b50 0x55b870e81020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159f7fe700 1 -- 172.21.15.31:0/3316132425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e76380 msgr2=0x55b870e76750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159f7fe700 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159f7fe700 1 -- 172.21.15.31:0/3316132425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e81b50 msgr2=0x55b870e81020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159f7fe700 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e81b50 0x55b870e81020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159f7fe700 1 -- 172.21.15.31:0/3316132425 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b870d8eec0 con 0x55b870e76c90 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159ffff700 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f159f7fe700 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e88630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1588004aa0 tx=0x7f1588004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8380771140127058 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f1596ffd700 1 -- 172.21.15.31:0/3316132425 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588004d90 con 0x55b870e76c90 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f1596ffd700 1 -- 172.21.15.31:0/3316132425 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1588004f50 con 0x55b870e76c90 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.480+0000 7f1596ffd700 1 -- 172.21.15.31:0/3316132425 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15880308f0 con 0x55b870e76c90 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 -- 172.21.15.31:0/3316132425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 msgr2=0x55b870e88630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e88630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1588004aa0 tx=0x7f1588004c40 comp rx=0 tx=0).stop 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 -- 172.21.15.31:0/3316132425 shutdown_connections 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e81b50 0x55b870e81020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e76380 0x55b870e76750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/3316132425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76c90 0x55b870e88630 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 -- 172.21.15.31:0/3316132425 >> 172.21.15.31:0/3316132425 conn(0x55b870e6cca0 msgr2=0x55b870e8d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 -- 172.21.15.31:0/3316132425 shutdown_connections 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 -- 172.21.15.31:0/3316132425 wait complete. 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 -- start start 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.481+0000 7f15b25a5b80 1 Processor -- start 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 0x55b870ed8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76c90 0x55b870ed93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870ed9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870ee3fe0 con 0x55b870e76380 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870ee4120 con 0x55b870e76c90 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b870ee42e0 con 0x55b870e81b50 2023-12-14T18:34:01.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15b25a5b80 10 client.0 _ll_register_callbacks cb 0x7f1590010aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 0x55b870ed8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 0x55b870ed8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38566/0 (socket says 172.21.15.31:38566) 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 -- 172.21.15.31:0/1884349563 learned_addr learned my addr 172.21.15.31:0/1884349563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f15a49b4700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870ed9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159f7fe700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76c90 0x55b870ed93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 -- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 msgr2=0x55b870ed9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870ed9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 -- 172.21.15.31:0/1884349563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76c90 msgr2=0x55b870ed93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76c90 0x55b870ed93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 -- 172.21.15.31:0/1884349563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b870d8f170 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f159ffff700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 0x55b870ed8e80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f15900cacc0 tx=0x7f15900cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6dda658c834697e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.482+0000 7f157f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f157f7fe700 1 -- 172.21.15.31:0/1884349563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159009a2a0 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f157f7fe700 1 -- 172.21.15.31:0/1884349563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f159009a400 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f158801e040 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f157f7fe700 1 -- 172.21.15.31:0/1884349563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15900b82b0 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 10 client.16959 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b870d759a0 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f157f7fe700 1 -- 172.21.15.31:0/1884349563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15900ca080 con 0x55b870e76380 2023-12-14T18:34:01.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f157f7fe700 1 -- 172.21.15.31:0/1884349563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1590099530 con 0x55b870e76380 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 20 client.16959 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870ee5a20 0x55b870ee7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 4 client.16959 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b870dc1a00 con 0x55b870ee5a20 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.483+0000 7f159f7fe700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870ee5a20 0x55b870ee7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f159f7fe700 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870ee5a20 0x55b870ee7e30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870aa1d5f9e59fec server_cookie=a39b8bcbe3824055 in_seq=0 out_seq=0 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f157f7fe700 10 client.16959 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:01.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f157f7fe700 1 -- 172.21.15.31:0/1884349563 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b870dc1a00 con 0x55b870ee5a20 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f157f7fe700 10 client.16959 handle_command_reply: tid=0 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 client.16959 shutdown 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 msgr2=0x55b870ed8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 0x55b870ed8e80 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f15900cacc0 tx=0x7f15900cc040 comp rx=0 tx=0).stop 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 shutdown_connections 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b870ee5a20 0x55b870ee7e30 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b870e81b50 0x55b870ed9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b870e76380 0x55b870ed8e80 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 --2- 172.21.15.31:0/1884349563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b870e76c90 0x55b870ed93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 >> 172.21.15.31:0/1884349563 conn(0x55b870e6cca0 msgr2=0x55b870e79ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f157f7fe700 0 client.16959 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 shutdown_connections 2023-12-14T18:34:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 1 -- 172.21.15.31:0/1884349563 wait complete. 2023-12-14T18:34:01.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.484+0000 7f15b25a5b80 20 client.16959 trim_cache size 0 max 16384 2023-12-14T18:34:01.591 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:34:01.594 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:01 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3922927785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:34:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f158c0a12c0 msgr2=0x7f158c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f158c0a12c0 0x7f158c0a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f15880098e0 tx=0x7f1588000bc0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 msgr2=0x7f15a0100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:01.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 0x7f15a0100d90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f159000e9a0 tx=0x7f159000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 shutdown_connections 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15a01043d0 0x7f15a01a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a00ff390 0x7f15a0100d90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f158c0a12c0 0x7f158c0a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 --2- 172.21.15.31:0/3162556299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a00ffd30 0x7f15a01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 >> 172.21.15.31:0/3162556299 conn(0x7f15a0075fd0 msgr2=0x7f15a0067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 shutdown_connections 2023-12-14T18:34:01.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:01.485+0000 7f15a6417700 1 -- 172.21.15.31:0/3162556299 wait complete. 2023-12-14T18:34:01.597 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-12-14T18:34:01.606 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-12-14T18:34:01.606 INFO:teuthology.task.exec:Executing custom commands... 2023-12-14T18:34:01.607 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:34:01.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-12-14T18:34:01.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-12-14T18:34:02.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:02 smithi031 ceph-mon[42085]: pgmap v189: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 101 op/s 2023-12-14T18:34:02.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:02 smithi176 ceph-mon[43506]: pgmap v189: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 101 op/s 2023-12-14T18:34:02.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:02 smithi204 ceph-mon[43866]: pgmap v189: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 101 op/s 2023-12-14T18:34:02.597 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:34:02.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:34:02.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 -- 172.21.15.31:0/1029928725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ffd30 msgr2=0x7f387c103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 --2- 172.21.15.31:0/1029928725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ffd30 0x7f387c103cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f38700099c0 tx=0x7f3870009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 -- 172.21.15.31:0/1029928725 shutdown_connections 2023-12-14T18:34:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 --2- 172.21.15.31:0/1029928725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f387c1043d0 0x7f387c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 --2- 172.21.15.31:0/1029928725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f387c0ff390 0x7f387c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 --2- 172.21.15.31:0/1029928725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ffd30 0x7f387c103cf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 -- 172.21.15.31:0/1029928725 >> 172.21.15.31:0/1029928725 conn(0x7f387c075fd0 msgr2=0x7f387c0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 -- 172.21.15.31:0/1029928725 shutdown_connections 2023-12-14T18:34:02.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.749+0000 7f3881327700 1 -- 172.21.15.31:0/1029928725 wait complete. 2023-12-14T18:34:02.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 Processor -- start 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 -- start start 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 0x7f387c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f387c0ffd30 0x7f387c196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f387c1043d0 0x7f387c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c19b410 con 0x7f387c0ff390 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c19b580 con 0x7f387c1043d0 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.750+0000 7f3881327700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c19b740 con 0x7f387c0ffd30 2023-12-14T18:34:02.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 0x7f387c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 0x7f387c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38588/0 (socket says 172.21.15.31:38588) 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 -- 172.21.15.31:0/3822276573 learned_addr learned my addr 172.21.15.31:0/3822276573 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387b7fe700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f387c1043d0 0x7f387c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387a7fc700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f387c0ffd30 0x7f387c196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 -- 172.21.15.31:0/3822276573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f387c0ffd30 msgr2=0x7f387c196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f387c0ffd30 0x7f387c196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 -- 172.21.15.31:0/3822276573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f387c1043d0 msgr2=0x7f387c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f387c1043d0 0x7f387c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 -- 172.21.15.31:0/3822276573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700095f0 con 0x7f387c0ff390 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f387affd700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 0x7f387c196430 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f386c00d950 tx=0x7f386c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f3863fff700 1 -- 172.21.15.31:0/3822276573 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c00f890 con 0x7f387c0ff390 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f3863fff700 1 -- 172.21.15.31:0/3822276573 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f386c010460 con 0x7f387c0ff390 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c10f150 con 0x7f387c0ff390 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f3863fff700 1 -- 172.21.15.31:0/3822276573 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c00e610 con 0x7f387c0ff390 2023-12-14T18:34:02.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.751+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c10efc0 con 0x7f387c0ff390 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.754+0000 7f3863fff700 1 -- 172.21.15.31:0/3822276573 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f386c0108d0 con 0x7f387c0ff390 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.754+0000 7f3863fff700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38640a12c0 0x7f38640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.755+0000 7f3863fff700 1 -- 172.21.15.31:0/3822276573 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f386c0cce90 con 0x7f387c0ff390 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.755+0000 7f387a7fc700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38640a12c0 0x7f38640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.755+0000 7f387a7fc700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38640a12c0 0x7f38640a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38700058e0 tx=0x7f3870005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.755+0000 7f388d4b5b80 1 Processor -- start 2023-12-14T18:34:02.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.755+0000 7f388d4b5b80 1 -- start start 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f3657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f370690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f372fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f24f660 con 0x55d02f365400 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f2be8f0 con 0x55d02f365d10 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f2b7d80 con 0x55d02f370bd0 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f3657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f3657d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38594/0 (socket says 172.21.15.31:38594) 2023-12-14T18:34:02.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 -- 172.21.15.31:0/1693928480 learned_addr learned my addr 172.21.15.31:0/1693928480 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387b7fe700 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f372fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387a7fc700 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f370690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 -- 172.21.15.31:0/1693928480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 msgr2=0x55d02f372fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f372fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 -- 172.21.15.31:0/1693928480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 msgr2=0x55d02f370690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f370690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 -- 172.21.15.31:0/1693928480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02f139430 con 0x55d02f365400 2023-12-14T18:34:02.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387b7fe700 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f372fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f387affd700 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f3657d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f386c0960d0 tx=0x7f386c099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd25bb1cbbab60b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.756+0000 7f38617fa700 1 -- 172.21.15.31:0/1693928480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0962e0 con 0x55d02f365400 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f38617fa700 1 -- 172.21.15.31:0/1693928480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f386c096440 con 0x55d02f365400 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f38617fa700 1 -- 172.21.15.31:0/1693928480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0a1d90 con 0x55d02f365400 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1693928480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 msgr2=0x55d02f3657d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f3657d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f386c0960d0 tx=0x7f386c099c50 comp rx=0 tx=0).stop 2023-12-14T18:34:02.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1693928480 shutdown_connections 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f372fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f3657d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1693928480 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f370690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1693928480 >> 172.21.15.31:0/1693928480 conn(0x55d02f35bd20 msgr2=0x55d02f361860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1693928480 shutdown_connections 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.757+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1693928480 wait complete. 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 -- start start 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 Processor -- start 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f370570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 0x55d02f36d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f370bd0 0x55d02f36dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f254010 con 0x55d02f365d10 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f253cb0 con 0x55d02f370bd0 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f13b720 con 0x55d02f365400 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f388d4b5b80 10 client.0 _ll_register_callbacks cb 0x55d02f277990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f387a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 0x55d02f36d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.758+0000 7f387a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 0x55d02f36d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38602/0 (socket says 172.21.15.31:38602) 2023-12-14T18:34:02.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 -- 172.21.15.31:0/1644847235 learned_addr learned my addr 172.21.15.31:0/1644847235 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387affd700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f370570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387b7fe700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f370bd0 0x55d02f36dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 -- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 msgr2=0x55d02f370570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f370570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 -- 172.21.15.31:0/1644847235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f370bd0 msgr2=0x55d02f36dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f370bd0 0x55d02f36dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 -- 172.21.15.31:0/1644847235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c10efc0 con 0x55d02f365d10 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387affd700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f370570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:02.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f387a7fc700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 0x55d02f36d890 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f38700099c0 tx=0x7f3870009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd12d7668be9e3e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f385eff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f385eff5700 1 -- 172.21.15.31:0/1644847235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387002b070 con 0x55d02f365d10 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f385eff5700 1 -- 172.21.15.31:0/1644847235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3870004050 con 0x55d02f365d10 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d02f359330 con 0x55d02f365d10 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.759+0000 7f385eff5700 1 -- 172.21.15.31:0/1644847235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3870004210 con 0x55d02f365d10 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 10 client.16971 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d02f27e1f0 con 0x55d02f365d10 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f385eff5700 1 -- 172.21.15.31:0/1644847235 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d02f359330 con 0x55d02f365d10 2023-12-14T18:34:02.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f385eff5700 1 -- 172.21.15.31:0/1644847235 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f387000fac0 con 0x55d02f365d10 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 10 client.16971 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 20 client.16971 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f380470 0x55d02f382880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 4 client.16971 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d02f2b0d60 con 0x55d02f380470 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.760+0000 7f387affd700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f380470 0x55d02f382880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.761+0000 7f387affd700 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f380470 0x55d02f382880 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720d3f3ebf334272 server_cookie=edbcdb08dcb0b199 in_seq=0 out_seq=0 2023-12-14T18:34:02.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.761+0000 7f385eff5700 10 client.16971 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.761+0000 7f385eff5700 1 -- 172.21.15.31:0/1644847235 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d02f2b0d60 con 0x55d02f380470 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.761+0000 7f385eff5700 10 client.16971 handle_command_reply: tid=0 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.761+0000 7f388d4b5b80 1 client.16971 shutdown 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 msgr2=0x55d02f36d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 0x55d02f36d890 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f38700099c0 tx=0x7f3870009310 comp rx=0 tx=0).stop 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 shutdown_connections 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f380470 0x55d02f382880 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f370bd0 0x55d02f36dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f385eff5700 0 client.16971 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f370570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/1644847235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365d10 0x55d02f36d890 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 >> 172.21.15.31:0/1644847235 conn(0x55d02f35bd20 msgr2=0x55d02f3613f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 shutdown_connections 2023-12-14T18:34:02.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 1 -- 172.21.15.31:0/1644847235 wait complete. 2023-12-14T18:34:02.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.762+0000 7f388d4b5b80 20 client.16971 trim_cache size 0 max 16384 2023-12-14T18:34:02.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.779+0000 7f388d4b5b80 1 Processor -- start 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.779+0000 7f388d4b5b80 1 -- start start 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f377650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f361360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f3618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f361e20 con 0x55d02f365400 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f36be20 con 0x55d02f365d10 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f3696e0 con 0x55d02f370bd0 2023-12-14T18:34:02.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f377650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f377650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38608/0 (socket says 172.21.15.31:38608) 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 -- 172.21.15.31:0/2450097051 learned_addr learned my addr 172.21.15.31:0/2450097051 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 -- 172.21.15.31:0/2450097051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 msgr2=0x55d02f3618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f3618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 -- 172.21.15.31:0/2450097051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 msgr2=0x55d02f361360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f361360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 -- 172.21.15.31:0/2450097051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02f27df40 con 0x55d02f365400 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f387affd700 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f377650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f386c099b60 tx=0x7f386c010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b9338d3aecbfcf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f3861ffb700 1 -- 172.21.15.31:0/2450097051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c09dc00 con 0x55d02f365400 2023-12-14T18:34:02.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f3861ffb700 1 -- 172.21.15.31:0/2450097051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f386c098050 con 0x55d02f365400 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.780+0000 7f3861ffb700 1 -- 172.21.15.31:0/2450097051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0a1b90 con 0x55d02f365400 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 -- 172.21.15.31:0/2450097051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 msgr2=0x55d02f377650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f377650 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f386c099b60 tx=0x7f386c010cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 -- 172.21.15.31:0/2450097051 shutdown_connections 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f370bd0 0x55d02f3618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f365400 0x55d02f377650 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/2450097051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f361360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 -- 172.21.15.31:0/2450097051 >> 172.21.15.31:0/2450097051 conn(0x55d02f35bd20 msgr2=0x55d02f368b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 -- 172.21.15.31:0/2450097051 shutdown_connections 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 -- 172.21.15.31:0/2450097051 wait complete. 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 -- start start 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.781+0000 7f388d4b5b80 1 Processor -- start 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f375d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f376270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 0x55d02f3767b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f3d4340 con 0x55d02f370bd0 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f3d4480 con 0x55d02f365d10 2023-12-14T18:34:02.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02f3d45c0 con 0x55d02f365400 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f388d4b5b80 10 client.0 _ll_register_callbacks cb 0x7f3870003cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 0x55d02f3767b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 0x55d02f3767b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38616/0 (socket says 172.21.15.31:38616) 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 -- 172.21.15.31:0/779836776 learned_addr learned my addr 172.21.15.31:0/779836776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387affd700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f375d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387a7fc700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f376270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 -- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 msgr2=0x55d02f375d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f375d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 -- 172.21.15.31:0/779836776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 msgr2=0x55d02f376270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f376270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 -- 172.21.15.31:0/779836776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d02f139430 con 0x55d02f370bd0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f387b7fe700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 0x55d02f3767b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3874007f40 tx=0x7f38740093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c8bf979c133572 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f385eff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.782+0000 7f385eff5700 1 -- 172.21.15.31:0/779836776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387400e040 con 0x55d02f370bd0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f385eff5700 1 -- 172.21.15.31:0/779836776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f3874003e10 con 0x55d02f370bd0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f386c096440 con 0x55d02f370bd0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f385eff5700 1 -- 172.21.15.31:0/779836776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3874004bb0 con 0x55d02f370bd0 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 10 client.16977 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:02.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d02f264a20 con 0x55d02f370bd0 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f385eff5700 1 -- 172.21.15.31:0/779836776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f386c096440 con 0x55d02f370bd0 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f385eff5700 1 -- 172.21.15.31:0/779836776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f387401d030 con 0x55d02f370bd0 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 10 client.16977 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 20 client.16977 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f3d5e40 0x55d02f3d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 4 client.16977 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.783+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d02f2b0a80 con 0x55d02f3d5e40 2023-12-14T18:34:02.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.784+0000 7f387affd700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f3d5e40 0x55d02f3d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.784+0000 7f387affd700 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f3d5e40 0x55d02f3d8250 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21baaa66c5285b06 server_cookie=c0edc658f13df577 in_seq=0 out_seq=0 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.784+0000 7f385eff5700 10 client.16977 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f385eff5700 1 -- 172.21.15.31:0/779836776 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d02f2b0a80 con 0x55d02f3d5e40 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f385eff5700 10 client.16977 handle_command_reply: tid=0 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 client.16977 shutdown 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 msgr2=0x55d02f3767b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 0x55d02f3767b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3874007f40 tx=0x7f38740093a0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 shutdown_connections 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d02f3d5e40 0x55d02f3d8250 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d02f370bd0 0x55d02f3767b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d02f365400 0x55d02f375d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 --2- 172.21.15.31:0/779836776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d02f365d10 0x55d02f376270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 >> 172.21.15.31:0/779836776 conn(0x55d02f35bd20 msgr2=0x55d02f368b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f385eff5700 0 client.16977 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 shutdown_connections 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.785+0000 7f388d4b5b80 1 -- 172.21.15.31:0/779836776 wait complete. 2023-12-14T18:34:02.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f388d4b5b80 20 client.16977 trim_cache size 0 max 16384 2023-12-14T18:34:02.795 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:34:02.795 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:34:02.795 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "cccb130d-99c1-4419-8dcf-6be3eeed4924", 2023-12-14T18:34:02.795 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:34:02.795 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38640a12c0 msgr2=0x7f38640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38640a12c0 0x7f38640a3770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38700058e0 tx=0x7f3870005fb0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 msgr2=0x7f387c196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 0x7f387c196430 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f386c00d950 tx=0x7f386c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 shutdown_connections 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f387c1043d0 0x7f387c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f387c0ff390 0x7f387c196430 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38640a12c0 0x7f38640a3770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 --2- 172.21.15.31:0/3822276573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f387c0ffd30 0x7f387c196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 >> 172.21.15.31:0/3822276573 conn(0x7f387c075fd0 msgr2=0x7f387c067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 shutdown_connections 2023-12-14T18:34:02.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.786+0000 7f3881327700 1 -- 172.21.15.31:0/3822276573 wait complete. 2023-12-14T18:34:02.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:34:02.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 -- 172.21.15.31:0/3078937691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05940ff420 msgr2=0x7f05940ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 --2- 172.21.15.31:0/3078937691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05940ff420 0x7f05940ff7f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f05880099c0 tx=0x7f0588009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 -- 172.21.15.31:0/3078937691 shutdown_connections 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 --2- 172.21.15.31:0/3078937691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0594104410 0x7f059410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 --2- 172.21.15.31:0/3078937691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05940ff420 0x7f05940ff7f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 --2- 172.21.15.31:0/3078937691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05940ffdc0 0x7f0594103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 -- 172.21.15.31:0/3078937691 >> 172.21.15.31:0/3078937691 conn(0x7f0594075fc0 msgr2=0x7f05940763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 -- 172.21.15.31:0/3078937691 shutdown_connections 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.943+0000 7f059ae31700 1 -- 172.21.15.31:0/3078937691 wait complete. 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 Processor -- start 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 -- start start 2023-12-14T18:34:02.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05940ff420 0x7f0594198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05940ffdc0 0x7f0594198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 0x7f059419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059419d620 con 0x7f0594104410 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059419d790 con 0x7f05940ffdc0 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f059ae31700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059419d950 con 0x7f05940ff420 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 0x7f059419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 0x7f059419cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38634/0 (socket says 172.21.15.31:38634) 2023-12-14T18:34:02.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f0593fff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05940ffdc0 0x7f0594198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 -- 172.21.15.31:0/3254030464 learned_addr learned my addr 172.21.15.31:0/3254030464 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f0598bcd700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05940ff420 0x7f0594198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 -- 172.21.15.31:0/3254030464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05940ff420 msgr2=0x7f0594198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05940ff420 0x7f0594198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 -- 172.21.15.31:0/3254030464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05940ffdc0 msgr2=0x7f0594198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05940ffdc0 0x7f0594198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.944+0000 7f05993ce700 1 -- 172.21.15.31:0/3254030464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05880095f0 con 0x7f0594104410 2023-12-14T18:34:02.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.945+0000 7f05993ce700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 0x7f059419cfe0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f058c00ef30 tx=0x7f058c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.945+0000 7f0591ffb700 1 -- 172.21.15.31:0/3254030464 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c018070 con 0x7f0594104410 2023-12-14T18:34:02.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.945+0000 7f0591ffb700 1 -- 172.21.15.31:0/3254030464 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f058c00f040 con 0x7f0594104410 2023-12-14T18:34:02.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.945+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059410e190 con 0x7f0594104410 2023-12-14T18:34:02.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.945+0000 7f0591ffb700 1 -- 172.21.15.31:0/3254030464 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c00cd00 con 0x7f0594104410 2023-12-14T18:34:02.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.945+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059410e000 con 0x7f0594104410 2023-12-14T18:34:02.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.948+0000 7f0591ffb700 1 -- 172.21.15.31:0/3254030464 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f058c003a00 con 0x7f0594104410 2023-12-14T18:34:02.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.949+0000 7f0591ffb700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f05840a1340 0x7f05840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.949+0000 7f0591ffb700 1 -- 172.21.15.31:0/3254030464 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f058c0ccb60 con 0x7f0594104410 2023-12-14T18:34:02.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.949+0000 7f0598bcd700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f05840a1340 0x7f05840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.949+0000 7f0598bcd700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f05840a1340 0x7f05840a37f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0588009990 tx=0x7f0588005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 Processor -- start 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 -- start start 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf360750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf36b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf36df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf24a5e0 con 0x562bcf36bb50 2023-12-14T18:34:02.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf2b9870 con 0x562bcf360380 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf2b2d00 con 0x562bcf360c90 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf36df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf36df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38648/0 (socket says 172.21.15.31:38648) 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 -- 172.21.15.31:0/42043276 learned_addr learned my addr 172.21.15.31:0/42043276 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f0598bcd700 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf360750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f0593fff700 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf36b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 -- 172.21.15.31:0/42043276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 msgr2=0x562bcf36b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf36b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 -- 172.21.15.31:0/42043276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 msgr2=0x562bcf360750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf360750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.950+0000 7f05993ce700 1 -- 172.21.15.31:0/42043276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcf1343b0 con 0x562bcf36bb50 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.951+0000 7f05993ce700 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf36df30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f058c0a1760 tx=0x7f058c099410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a890d6f357df3556 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.951+0000 7f057effd700 1 -- 172.21.15.31:0/42043276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c004ad0 con 0x562bcf36bb50 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.951+0000 7f057effd700 1 -- 172.21.15.31:0/42043276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f058c0998d0 con 0x562bcf36bb50 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.951+0000 7f057effd700 1 -- 172.21.15.31:0/42043276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c099d90 con 0x562bcf36bb50 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/42043276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 msgr2=0x562bcf36df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf36df30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f058c0a1760 tx=0x7f058c099410 comp rx=0 tx=0).stop 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/42043276 shutdown_connections 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf36df30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf360750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/42043276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf36b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/42043276 >> 172.21.15.31:0/42043276 conn(0x562bcf356ca0 msgr2=0x562bcf35c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/42043276 shutdown_connections 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/42043276 wait complete. 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 -- start start 2023-12-14T18:34:02.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.952+0000 7f05a6fbfb80 1 Processor -- start 2023-12-14T18:34:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf36b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 0x562bcf3688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf36bb50 0x562bcf368ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf24ef90 con 0x562bcf360c90 2023-12-14T18:34:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf24ec30 con 0x562bcf360380 2023-12-14T18:34:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf1366a0 con 0x562bcf36bb50 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f05a6fbfb80 10 client.0 _ll_register_callbacks cb 0x562bcf272910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 0x562bcf3688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 0x562bcf3688b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38660/0 (socket says 172.21.15.31:38660) 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 -- 172.21.15.31:0/3333026671 learned_addr learned my addr 172.21.15.31:0/3333026671 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0598bcd700 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf36b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 -- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf36bb50 msgr2=0x562bcf368ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf36bb50 0x562bcf368ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 -- 172.21.15.31:0/3333026671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 msgr2=0x562bcf36b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf36b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 -- 172.21.15.31:0/3333026671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059410e000 con 0x562bcf360c90 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0593fff700 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 0x562bcf3688b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f05800078a0 tx=0x7f0580007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7255b5c4e8cc1778 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.953+0000 7f0573fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f0573fff700 1 -- 172.21.15.31:0/3333026671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580010040 con 0x562bcf360c90 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f0573fff700 1 -- 172.21.15.31:0/3333026671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f0580004230 con 0x562bcf360c90 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bcf3542b0 con 0x562bcf360c90 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f0573fff700 1 -- 172.21.15.31:0/3333026671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05800043f0 con 0x562bcf360c90 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 10 client.16989 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562bcf279170 con 0x562bcf360c90 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f0573fff700 1 -- 172.21.15.31:0/3333026671 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562bcf3542b0 con 0x562bcf360c90 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f0573fff700 1 -- 172.21.15.31:0/3333026671 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f058001e060 con 0x562bcf360c90 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 10 client.16989 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 20 client.16989 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf37b750 0x562bcf37db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 4 client.16989 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.954+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bcf2abce0 con 0x562bcf37b750 2023-12-14T18:34:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.955+0000 7f0598bcd700 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf37b750 0x562bcf37db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.955+0000 7f0598bcd700 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf37b750 0x562bcf37db60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7fe9caa59c700a6 server_cookie=3971a47f42078994 in_seq=0 out_seq=0 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.955+0000 7f0573fff700 10 client.16989 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f0573fff700 1 -- 172.21.15.31:0/3333026671 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bcf2abce0 con 0x562bcf37b750 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f0573fff700 10 client.16989 handle_command_reply: tid=0 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 client.16989 shutdown 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 msgr2=0x562bcf3688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 0x562bcf3688b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f05800078a0 tx=0x7f0580007bb0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 shutdown_connections 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf37b750 0x562bcf37db60 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf36bb50 0x562bcf368ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360380 0x562bcf36b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/3333026671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360c90 0x562bcf3688b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f0573fff700 0 client.16989 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 >> 172.21.15.31:0/3333026671 conn(0x562bcf356ca0 msgr2=0x562bcf35c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 shutdown_connections 2023-12-14T18:34:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/3333026671 wait complete. 2023-12-14T18:34:02.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.956+0000 7f05a6fbfb80 20 client.16989 trim_cache size 0 max 16384 2023-12-14T18:34:02.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 Processor -- start 2023-12-14T18:34:02.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 -- start start 2023-12-14T18:34:02.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 0x562bcf363270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf363c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf36bb50 0x562bcf36b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf378010 con 0x562bcf360380 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf356ab0 con 0x562bcf36bb50 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf361390 con 0x562bcf360c90 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 0x562bcf363270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 0x562bcf363270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38676/0 (socket says 172.21.15.31:38676) 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 -- 172.21.15.31:0/1866827775 learned_addr learned my addr 172.21.15.31:0/1866827775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f05993ce700 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf36bb50 0x562bcf36b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0593fff700 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf363c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 -- 172.21.15.31:0/1866827775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 msgr2=0x562bcf363c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf363c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 -- 172.21.15.31:0/1866827775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf36bb50 msgr2=0x562bcf36b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.974+0000 7f0598bcd700 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf36bb50 0x562bcf36b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.975+0000 7f0598bcd700 1 -- 172.21.15.31:0/1866827775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcf368420 con 0x562bcf360380 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.975+0000 7f0598bcd700 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 0x562bcf363270 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f05880036c0 tx=0x7f0588022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9334544a546ce0c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.975+0000 7f057f7fe700 1 -- 172.21.15.31:0/1866827775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058802c040 con 0x562bcf360380 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.975+0000 7f057f7fe700 1 -- 172.21.15.31:0/1866827775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f058802d040 con 0x562bcf360380 2023-12-14T18:34:02.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.975+0000 7f057f7fe700 1 -- 172.21.15.31:0/1866827775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588004260 con 0x562bcf360380 2023-12-14T18:34:02.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1866827775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 msgr2=0x562bcf363270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 0x562bcf363270 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f05880036c0 tx=0x7f0588022000 comp rx=0 tx=0).stop 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1866827775 shutdown_connections 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf36bb50 0x562bcf36b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf360380 0x562bcf363270 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1866827775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360c90 0x562bcf363c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1866827775 >> 172.21.15.31:0/1866827775 conn(0x562bcf356ca0 msgr2=0x562bcf377900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1866827775 shutdown_connections 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1866827775 wait complete. 2023-12-14T18:34:02.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 -- start start 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.976+0000 7f05a6fbfb80 1 Processor -- start 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360380 0x562bcf371f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360c90 0x562bcf3724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf3703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf3709c0 con 0x562bcf36bb50 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf370b30 con 0x562bcf360c90 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcf370cf0 con 0x562bcf360380 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05a6fbfb80 10 client.0 _ll_register_callbacks cb 0x7f058000bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:02.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf3703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf3703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38678/0 (socket says 172.21.15.31:38678) 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 -- 172.21.15.31:0/1837083742 learned_addr learned my addr 172.21.15.31:0/1837083742 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f0593fff700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360c90 0x562bcf3724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f0598bcd700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360380 0x562bcf371f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 -- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360380 msgr2=0x562bcf371f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360380 0x562bcf371f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 -- 172.21.15.31:0/1837083742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360c90 msgr2=0x562bcf3724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360c90 0x562bcf3724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 -- 172.21.15.31:0/1837083742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059410c820 con 0x562bcf36bb50 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f05993ce700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf3703a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f058c0995d0 tx=0x7f058c099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96817ec745512218 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f0573fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.977+0000 7f0573fff700 1 -- 172.21.15.31:0/1837083742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c004ad0 con 0x562bcf36bb50 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f0573fff700 1 -- 172.21.15.31:0/1837083742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9437+0+0 (secure 0 0 0) 0x7f058c0a1c90 con 0x562bcf36bb50 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f058c0998d0 con 0x562bcf36bb50 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f0573fff700 1 -- 172.21.15.31:0/1837083742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058c09d900 con 0x562bcf36bb50 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 10 client.16995 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:02.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562bcf0e0b60 con 0x562bcf36bb50 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f0573fff700 1 -- 172.21.15.31:0/1837083742 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f058c0998d0 con 0x562bcf36bb50 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f0573fff700 1 -- 172.21.15.31:0/1837083742 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f058c0a1800 con 0x562bcf36bb50 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 10 client.16995 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 10 client.16995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 20 client.16995 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf3d8b80 0x562bcf3daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 4 client.16995 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562bcf2aba00 con 0x562bcf3d8b80 2023-12-14T18:34:02.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.978+0000 7f0598bcd700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf3d8b80 0x562bcf3daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f0598bcd700 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf3d8b80 0x562bcf3daf90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcd246062ed46ef server_cookie=bf1df05290a4c239 in_seq=0 out_seq=0 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f0573fff700 10 client.16995 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f0573fff700 1 -- 172.21.15.31:0/1837083742 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562bcf2aba00 con 0x562bcf3d8b80 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f0573fff700 10 client.16995 handle_command_reply: tid=0 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 client.16995 shutdown 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 msgr2=0x562bcf3703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf3703a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f058c0995d0 tx=0x7f058c099a40 comp rx=0 tx=0).stop 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 shutdown_connections 2023-12-14T18:34:02.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x562bcf3d8b80 0x562bcf3daf90 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x562bcf36bb50 0x562bcf3703a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562bcf360380 0x562bcf371f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 --2- 172.21.15.31:0/1837083742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x562bcf360c90 0x562bcf3724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f0573fff700 0 client.16995 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 >> 172.21.15.31:0/1837083742 conn(0x562bcf356ca0 msgr2=0x562bcf363c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 shutdown_connections 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.979+0000 7f05a6fbfb80 1 -- 172.21.15.31:0/1837083742 wait complete. 2023-12-14T18:34:02.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f05a6fbfb80 20 client.16995 trim_cache size 0 max 16384 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: "cccb130d-99c1-4419-8dcf-6be3eeed4924": { 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "cccb130d-99c1-4419-8dcf-6be3eeed4924", 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:34:02.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:34:02.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:34:02.991 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:34:02.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f05840a1340 msgr2=0x7f05840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f05840a1340 0x7f05840a37f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0588009990 tx=0x7f0588005fb0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 msgr2=0x7f059419cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 0x7f059419cfe0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f058c00ef30 tx=0x7f058c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 shutdown_connections 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0594104410 0x7f059419cfe0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f05840a1340 0x7f05840a37f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05940ff420 0x7f0594198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 --2- 172.21.15.31:0/3254030464 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05940ffdc0 0x7f0594198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:02.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 >> 172.21.15.31:0/3254030464 conn(0x7f0594075fc0 msgr2=0x7f05940fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:02.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 shutdown_connections 2023-12-14T18:34:02.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:02.980+0000 7f059ae31700 1 -- 172.21.15.31:0/3254030464 wait complete. 2023-12-14T18:34:02.995 INFO:tasks.cephfs.filesystem:scrub status for tag:cccb130d-99c1-4419-8dcf-6be3eeed4924 - {'path': '/', 'tag': 'cccb130d-99c1-4419-8dcf-6be3eeed4924', 'options': 'recursive,force'} 2023-12-14T18:34:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:03 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:34:03.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:03 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:03.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:03 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:34:03.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:03 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:03.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:03 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:34:03.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:03 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:03.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:03 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:34:03.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:03 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:03.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:03 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:34:03.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:03 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:03.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:03 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:34:03.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:03 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:04 smithi031 ceph-mon[42085]: pgmap v190: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 101 op/s 2023-12-14T18:34:04.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:04 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:34:04.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:04 smithi176 ceph-mon[43506]: pgmap v190: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 101 op/s 2023-12-14T18:34:04.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:04 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:34:04.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:04 smithi204 ceph-mon[43866]: pgmap v190: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 34 KiB/s wr, 101 op/s 2023-12-14T18:34:04.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:04 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:34:06.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:06 smithi031 ceph-mon[42085]: pgmap v191: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 123 op/s 2023-12-14T18:34:06.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:06 smithi176 ceph-mon[43506]: pgmap v191: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 123 op/s 2023-12-14T18:34:06.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:06 smithi204 ceph-mon[43866]: pgmap v191: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 102 KiB/s wr, 123 op/s 2023-12-14T18:34:08.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:08 smithi031 ceph-mon[42085]: pgmap v192: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 26 op/s 2023-12-14T18:34:08.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:08 smithi176 ceph-mon[43506]: pgmap v192: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 26 op/s 2023-12-14T18:34:08.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:08 smithi204 ceph-mon[43866]: pgmap v192: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 69 KiB/s wr, 26 op/s 2023-12-14T18:34:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:10 smithi031 ceph-mon[42085]: pgmap v193: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-12-14T18:34:10.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:10 smithi031 ceph-mon[42085]: osdmap e91: 12 total, 12 up, 12 in 2023-12-14T18:34:10.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:10 smithi176 ceph-mon[43506]: pgmap v193: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-12-14T18:34:10.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:10 smithi176 ceph-mon[43506]: osdmap e91: 12 total, 12 up, 12 in 2023-12-14T18:34:10.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:10 smithi204 ceph-mon[43866]: pgmap v193: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 72 KiB/s wr, 29 op/s 2023-12-14T18:34:10.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:10 smithi204 ceph-mon[43866]: osdmap e91: 12 total, 12 up, 12 in 2023-12-14T18:34:12.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:12 smithi031 ceph-mon[42085]: pgmap v195: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-12-14T18:34:12.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:12 smithi176 ceph-mon[43506]: pgmap v195: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-12-14T18:34:12.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:12 smithi204 ceph-mon[43866]: pgmap v195: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-12-14T18:34:14.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:14 smithi031 ceph-mon[42085]: pgmap v196: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-12-14T18:34:14.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:14 smithi176 ceph-mon[43506]: pgmap v196: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-12-14T18:34:14.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:14 smithi204 ceph-mon[43866]: pgmap v196: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 30 op/s 2023-12-14T18:34:15.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:15 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:34:15.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:15 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:34:15.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:15 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:34:15.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:15 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:34:15.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:15 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:34:15.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:15 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:34:15.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:15 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:34:15.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:15 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:34:15.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:15 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:34:15.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:15 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:34:15.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:15 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:34:15.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:15 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:34:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:16 smithi031 ceph-mon[42085]: pgmap v197: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-12-14T18:34:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:16 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:34:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:16 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:34:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:16 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:34:16.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:16 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:34:16.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:16 smithi176 ceph-mon[43506]: pgmap v197: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-12-14T18:34:16.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:16 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:34:16.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:16 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:34:16.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:16 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:34:16.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:16 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:34:16.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:16 smithi204 ceph-mon[43866]: pgmap v197: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-12-14T18:34:16.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:16 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:34:16.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:16 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:34:16.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:16 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:34:16.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:16 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:34:18.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:18 smithi031 ceph-mon[42085]: pgmap v198: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-12-14T18:34:18.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:18 smithi176 ceph-mon[43506]: pgmap v198: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-12-14T18:34:18.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:18 smithi204 ceph-mon[43866]: pgmap v198: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2023-12-14T18:34:19.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:19 smithi031 ceph-mon[42085]: pgmap v199: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:19.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:19 smithi031 ceph-mon[42085]: osdmap e92: 12 total, 12 up, 12 in 2023-12-14T18:34:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:19 smithi176 ceph-mon[43506]: pgmap v199: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:19 smithi176 ceph-mon[43506]: osdmap e92: 12 total, 12 up, 12 in 2023-12-14T18:34:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:19 smithi204 ceph-mon[43866]: pgmap v199: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:19 smithi204 ceph-mon[43866]: osdmap e92: 12 total, 12 up, 12 in 2023-12-14T18:34:21.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:21 smithi031 ceph-mon[42085]: pgmap v201: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:21.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:21 smithi176 ceph-mon[43506]: pgmap v201: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:21 smithi204 ceph-mon[43866]: pgmap v201: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:23.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:23 smithi031 ceph-mon[42085]: pgmap v202: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:23.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:23 smithi176 ceph-mon[43506]: pgmap v202: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:23 smithi204 ceph-mon[43866]: pgmap v202: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.8 KiB/s wr, 9 op/s 2023-12-14T18:34:25.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:25 smithi176 ceph-mon[43506]: pgmap v203: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2023-12-14T18:34:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:25 smithi204 ceph-mon[43866]: pgmap v203: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2023-12-14T18:34:25.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:25 smithi031 ceph-mon[42085]: pgmap v203: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2023-12-14T18:34:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:27 smithi176 ceph-mon[43506]: pgmap v204: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2023-12-14T18:34:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:27 smithi204 ceph-mon[43866]: pgmap v204: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2023-12-14T18:34:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:27 smithi031 ceph-mon[42085]: pgmap v204: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2023-12-14T18:34:29.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:29 smithi176 ceph-mon[43506]: pgmap v205: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-12-14T18:34:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:29 smithi204 ceph-mon[43866]: pgmap v205: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-12-14T18:34:29.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:29 smithi031 ceph-mon[42085]: pgmap v205: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2023-12-14T18:34:31.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:31 smithi176 ceph-mon[43506]: pgmap v206: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:34:31.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:31 smithi204 ceph-mon[43866]: pgmap v206: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:34:31.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:31 smithi031 ceph-mon[42085]: pgmap v206: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:34:32.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:34:33.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 -- 172.21.15.31:0/437864684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 msgr2=0x7f0b38103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 --2- 172.21.15.31:0/437864684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38103d30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0b200099c0 tx=0x7f0b20009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 -- 172.21.15.31:0/437864684 shutdown_connections 2023-12-14T18:34:33.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 --2- 172.21.15.31:0/437864684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b38104410 0x7f0b3810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 --2- 172.21.15.31:0/437864684 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b380ff3d0 0x7f0b380ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 --2- 172.21.15.31:0/437864684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38103d30 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 -- 172.21.15.31:0/437864684 >> 172.21.15.31:0/437864684 conn(0x7f0b38076230 msgr2=0x7f0b38076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 -- 172.21.15.31:0/437864684 shutdown_connections 2023-12-14T18:34:33.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 -- 172.21.15.31:0/437864684 wait complete. 2023-12-14T18:34:33.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 Processor -- start 2023-12-14T18:34:33.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.143+0000 7f0b3e489700 1 -- start start 2023-12-14T18:34:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3e489700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b380ff3d0 0x7f0b38196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3e489700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3e489700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b38104410 0x7f0b3819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3e489700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3819b410 con 0x7f0b380ffd70 2023-12-14T18:34:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3e489700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3819b580 con 0x7f0b38104410 2023-12-14T18:34:33.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3e489700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3819b740 con 0x7f0b380ff3d0 2023-12-14T18:34:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52334/0 (socket says 172.21.15.31:52334) 2023-12-14T18:34:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 -- 172.21.15.31:0/162335137 learned_addr learned my addr 172.21.15.31:0/162335137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b37fff700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b380ff3d0 0x7f0b38196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b38104410 0x7f0b3819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 -- 172.21.15.31:0/162335137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b380ff3d0 msgr2=0x7f0b38196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b380ff3d0 0x7f0b38196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 -- 172.21.15.31:0/162335137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b38104410 msgr2=0x7f0b3819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b38104410 0x7f0b3819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 -- 172.21.15.31:0/162335137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200095f0 con 0x7f0b380ffd70 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.144+0000 7f0b377fe700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38196970 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009990 tx=0x7f0b2000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.145+0000 7f0b357fa700 1 -- 172.21.15.31:0/162335137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2001d070 con 0x7f0b380ffd70 2023-12-14T18:34:33.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.145+0000 7f0b357fa700 1 -- 172.21.15.31:0/162335137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b20005020 con 0x7f0b380ffd70 2023-12-14T18:34:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.145+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3810f1e0 con 0x7f0b380ffd70 2023-12-14T18:34:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.145+0000 7f0b357fa700 1 -- 172.21.15.31:0/162335137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20004a70 con 0x7f0b380ffd70 2023-12-14T18:34:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.145+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3810f050 con 0x7f0b380ffd70 2023-12-14T18:34:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.147+0000 7f0b357fa700 1 -- 172.21.15.31:0/162335137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0b20005400 con 0x7f0b380ffd70 2023-12-14T18:34:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.147+0000 7f0b357fa700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0b240a1340 0x7f0b240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.148+0000 7f0b357fa700 1 -- 172.21.15.31:0/162335137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f0b200cdac0 con 0x7f0b380ffd70 2023-12-14T18:34:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.148+0000 7f0b37fff700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0b240a1340 0x7f0b240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.148+0000 7f0b37fff700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0b240a1340 0x7f0b240a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009c60 tx=0x7f0b28009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 Processor -- start 2023-12-14T18:34:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 -- start start 2023-12-14T18:34:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baab2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baabd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baabdb50 0x5606baabff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ba99c5e0 con 0x5606baab2380 2023-12-14T18:34:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baa0b870 con 0x5606baab2c90 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baa04d00 con 0x5606baabdb50 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baab2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baab2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52348/0 (socket says 172.21.15.31:52348) 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 -- 172.21.15.31:0/3026163300 learned_addr learned my addr 172.21.15.31:0/3026163300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baabdb50 0x5606baabff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b377fe700 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baabd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 -- 172.21.15.31:0/3026163300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baabdb50 msgr2=0x5606baabff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baabdb50 0x5606baabff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 -- 172.21.15.31:0/3026163300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 msgr2=0x5606baabd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.149+0000 7f0b37fff700 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baabd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.150+0000 7f0b37fff700 1 -- 172.21.15.31:0/3026163300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ba8863b0 con 0x5606baab2380 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.150+0000 7f0b37fff700 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baab2750 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e4e0 tx=0x7f0b28014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b271832dab07bd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.150+0000 7f0b2e7fc700 1 -- 172.21.15.31:0/3026163300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28026430 con 0x5606baab2380 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.150+0000 7f0b2e7fc700 1 -- 172.21.15.31:0/3026163300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b28021070 con 0x5606baab2380 2023-12-14T18:34:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.150+0000 7f0b2e7fc700 1 -- 172.21.15.31:0/3026163300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2801d740 con 0x5606baab2380 2023-12-14T18:34:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 -- 172.21.15.31:0/3026163300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 msgr2=0x5606baab2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baab2750 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e4e0 tx=0x7f0b28014000 comp rx=0 tx=0).stop 2023-12-14T18:34:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 -- 172.21.15.31:0/3026163300 shutdown_connections 2023-12-14T18:34:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baabdb50 0x5606baabff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baab2750 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/3026163300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baabd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 -- 172.21.15.31:0/3026163300 >> 172.21.15.31:0/3026163300 conn(0x5606baaa8ca0 msgr2=0x5606baaae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 -- 172.21.15.31:0/3026163300 shutdown_connections 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 -- 172.21.15.31:0/3026163300 wait complete. 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 -- start start 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.151+0000 7f0b4a617b80 1 Processor -- start 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baaba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 0x5606baabacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ba9a0f90 con 0x5606baabdb50 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ba9a0c30 con 0x5606baab2c90 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ba8886a0 con 0x5606baab2380 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b4a617b80 10 client.0 _ll_register_callbacks cb 0x5606ba9c4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 0x5606baabacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 0x5606baabacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52360/0 (socket says 172.21.15.31:52360) 2023-12-14T18:34:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 -- 172.21.15.31:0/4246015759 learned_addr learned my addr 172.21.15.31:0/4246015759 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b37fff700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b377fe700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baaba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 -- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 msgr2=0x5606baabd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 -- 172.21.15.31:0/4246015759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 msgr2=0x5606baaba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baaba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 -- 172.21.15.31:0/4246015759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b28017070 con 0x5606baabdb50 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b377fe700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baaba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 0x5606baabacf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0b30004070 tx=0x7f0b30004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6620fad6696f32 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.152+0000 7f0b177fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b177fe700 1 -- 172.21.15.31:0/4246015759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300079d0 con 0x5606baabdb50 2023-12-14T18:34:33.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b177fe700 1 -- 172.21.15.31:0/4246015759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f0b30007b30 con 0x5606baabdb50 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606baaa62b0 con 0x5606baabdb50 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b177fe700 1 -- 172.21.15.31:0/4246015759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b30007cf0 con 0x5606baabdb50 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 10 client.17007 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5606ba9cb170 con 0x5606baabdb50 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b177fe700 1 -- 172.21.15.31:0/4246015759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606baaa62b0 con 0x5606baabdb50 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b177fe700 1 -- 172.21.15.31:0/4246015759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0b30014040 con 0x5606baabdb50 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 10 client.17007 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:33.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 20 client.17007 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baacd5a0 0x5606baacf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 4 client.17007 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606ba9fdce0 con 0x5606baacd5a0 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.153+0000 7f0b37fff700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baacd5a0 0x5606baacf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.154+0000 7f0b37fff700 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baacd5a0 0x5606baacf9b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b25b7a505e0b50e9 server_cookie=9c74e0de88dd47d8 in_seq=0 out_seq=0 2023-12-14T18:34:33.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.154+0000 7f0b177fe700 10 client.17007 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b177fe700 1 -- 172.21.15.31:0/4246015759 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5606ba9fdce0 con 0x5606baacd5a0 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b177fe700 10 client.17007 handle_command_reply: tid=0 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 client.17007 shutdown 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 msgr2=0x5606baabacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 0x5606baabacf0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0b30004070 tx=0x7f0b30004380 comp rx=0 tx=0).stop 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 shutdown_connections 2023-12-14T18:34:33.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baacd5a0 0x5606baacf9b0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baabdb50 0x5606baabacf0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b177fe700 0 client.17007 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4246015759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baaba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 >> 172.21.15.31:0/4246015759 conn(0x5606baaa8ca0 msgr2=0x5606baaae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 shutdown_connections 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4246015759 wait complete. 2023-12-14T18:34:33.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.155+0000 7f0b4a617b80 20 client.17007 trim_cache size 0 max 16384 2023-12-14T18:34:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 Processor -- start 2023-12-14T18:34:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 -- start start 2023-12-14T18:34:33.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baaa8ca0 0x5606baab9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 0x5606baabd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baac9ed0 con 0x5606baab2c90 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baacab80 con 0x5606baaa8ca0 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baab8cd0 con 0x5606baab2380 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 0x5606baabd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 0x5606baabd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52370/0 (socket says 172.21.15.31:52370) 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 -- 172.21.15.31:0/1244554784 learned_addr learned my addr 172.21.15.31:0/1244554784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b37fff700 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baaa8ca0 0x5606baab9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b377fe700 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 -- 172.21.15.31:0/1244554784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 msgr2=0x5606baabcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 -- 172.21.15.31:0/1244554784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baaa8ca0 msgr2=0x5606baab9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.173+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baaa8ca0 0x5606baab9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b3ca26700 1 -- 172.21.15.31:0/1244554784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ba9caec0 con 0x5606baab2c90 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b377fe700 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 0x5606baabd4c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0b30002230 tx=0x7f0b300152d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b261a51d87cf5cc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b2effd700 1 -- 172.21.15.31:0/1244554784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30015760 con 0x5606baab2c90 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b2effd700 1 -- 172.21.15.31:0/1244554784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b3000f040 con 0x5606baab2c90 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b2effd700 1 -- 172.21.15.31:0/1244554784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b30007a30 con 0x5606baab2c90 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 -- 172.21.15.31:0/1244554784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 msgr2=0x5606baabd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 0x5606baabd4c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0b30002230 tx=0x7f0b300152d0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 -- 172.21.15.31:0/1244554784 shutdown_connections 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2c90 0x5606baabd4c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baaa8ca0 0x5606baab9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/1244554784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baab2380 0x5606baabcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.174+0000 7f0b4a617b80 1 -- 172.21.15.31:0/1244554784 >> 172.21.15.31:0/1244554784 conn(0x5606baac0200 msgr2=0x5606baab4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 -- 172.21.15.31:0/1244554784 shutdown_connections 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 -- 172.21.15.31:0/1244554784 wait complete. 2023-12-14T18:34:33.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 -- start start 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 Processor -- start 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baaa8ca0 0x5606baac45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baac2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baac2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baac3420 con 0x5606baab2380 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baac3560 con 0x5606baab2c90 2023-12-14T18:34:33.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606baac3720 con 0x5606baaa8ca0 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b4a617b80 10 client.0 _ll_register_callbacks cb 0x7f0b30009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.175+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baac2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baac2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52386/0 (socket says 172.21.15.31:52386) 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 -- 172.21.15.31:0/4241742940 learned_addr learned my addr 172.21.15.31:0/4241742940 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b37fff700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baaa8ca0 0x5606baac45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baac2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 -- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baaa8ca0 msgr2=0x5606baac45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baaa8ca0 0x5606baac45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 -- 172.21.15.31:0/4241742940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 msgr2=0x5606baac2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baac2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 -- 172.21.15.31:0/4241742940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606ba9cb170 con 0x5606baab2380 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b3ca26700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baac2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b377fe700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baac2910 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0b200036c0 tx=0x7f0b2009a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb5d92910b11eed server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b177fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:33.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b177fe700 1 -- 172.21.15.31:0/4241742940 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2009a7d0 con 0x5606baab2380 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b177fe700 1 -- 172.21.15.31:0/4241742940 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f0b2009e800 con 0x5606baab2380 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b28021070 con 0x5606baab2380 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b177fe700 1 -- 172.21.15.31:0/4241742940 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b200a2b90 con 0x5606baab2380 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b4a617b80 10 client.17013 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.176+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5606ba9b19a0 con 0x5606baab2380 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b177fe700 1 -- 172.21.15.31:0/4241742940 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b28021070 con 0x5606baab2380 2023-12-14T18:34:33.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b177fe700 1 -- 172.21.15.31:0/4241742940 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0b200a2820 con 0x5606baab2380 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b4a617b80 10 client.17013 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b4a617b80 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b4a617b80 20 client.17013 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baabdb50 0x5606bab28b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b4a617b80 4 client.17013 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5606ba9fda00 con 0x5606baabdb50 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.177+0000 7f0b37fff700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baabdb50 0x5606bab28b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b177fe700 10 client.17013 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b37fff700 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baabdb50 0x5606bab28b70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab1c4f54f2ca2a47 server_cookie=a6b89d43a9d9b837 in_seq=0 out_seq=0 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b177fe700 1 -- 172.21.15.31:0/4241742940 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5606ba9fda00 con 0x5606baabdb50 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b177fe700 10 client.17013 handle_command_reply: tid=0 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 client.17013 shutdown 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 msgr2=0x5606baac2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baac2910 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0b200036c0 tx=0x7f0b2009a690 comp rx=0 tx=0).stop 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 shutdown_connections 2023-12-14T18:34:33.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5606baabdb50 0x5606bab28b70 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5606baab2c90 0x5606baac2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b177fe700 0 client.17013 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5606baaa8ca0 0x5606baac45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 --2- 172.21.15.31:0/4241742940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5606baab2380 0x5606baac2910 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 >> 172.21.15.31:0/4241742940 conn(0x5606baac0200 msgr2=0x5606baac96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 shutdown_connections 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 1 -- 172.21.15.31:0/4241742940 wait complete. 2023-12-14T18:34:33.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.178+0000 7f0b4a617b80 20 client.17013 trim_cache size 0 max 16384 2023-12-14T18:34:33.193 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:34:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:34:33.193 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:34:33.193 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0b240a1340 msgr2=0x7f0b240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0b240a1340 0x7f0b240a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009c60 tx=0x7f0b28009260 comp rx=0 tx=0).stop 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 msgr2=0x7f0b38196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38196970 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009990 tx=0x7f0b2000f690 comp rx=0 tx=0).stop 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 shutdown_connections 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0b38104410 0x7f0b3819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0b240a1340 0x7f0b240a37f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b380ff3d0 0x7f0b38196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 --2- 172.21.15.31:0/162335137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b380ffd70 0x7f0b38196970 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 >> 172.21.15.31:0/162335137 conn(0x7f0b38076230 msgr2=0x7f0b38068de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 shutdown_connections 2023-12-14T18:34:33.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.179+0000 7f0b3e489700 1 -- 172.21.15.31:0/162335137 wait complete. 2023-12-14T18:34:33.196 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:34:33.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:34:33.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 -- 172.21.15.31:0/2385175944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ffdd0 msgr2=0x7f81e0103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 --2- 172.21.15.31:0/2385175944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ffdd0 0x7f81e0103d90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f81d00099c0 tx=0x7f81d0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 -- 172.21.15.31:0/2385175944 shutdown_connections 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 --2- 172.21.15.31:0/2385175944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e0104470 0x7f81e010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 --2- 172.21.15.31:0/2385175944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81e00ff430 0x7f81e00ff800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 --2- 172.21.15.31:0/2385175944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ffdd0 0x7f81e0103d90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 -- 172.21.15.31:0/2385175944 >> 172.21.15.31:0/2385175944 conn(0x7f81e0076370 msgr2=0x7f81e0076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 -- 172.21.15.31:0/2385175944 shutdown_connections 2023-12-14T18:34:33.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.333+0000 7f81e7608700 1 -- 172.21.15.31:0/2385175944 wait complete. 2023-12-14T18:34:33.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 Processor -- start 2023-12-14T18:34:33.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 -- start start 2023-12-14T18:34:33.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 0x7f81e0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81e00ffdd0 0x7f81e0196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e0104470 0x7f81e019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e019b430 con 0x7f81e00ff430 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e019b5a0 con 0x7f81e00ffdd0 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e7608700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e019b760 con 0x7f81e0104470 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e53a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 0x7f81e0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e53a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 0x7f81e0196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52406/0 (socket says 172.21.15.31:52406) 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e53a4700 1 -- 172.21.15.31:0/315147287 learned_addr learned my addr 172.21.15.31:0/315147287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e4ba3700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81e00ffdd0 0x7f81e0196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.334+0000 7f81e5ba5700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e0104470 0x7f81e019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e53a4700 1 -- 172.21.15.31:0/315147287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e0104470 msgr2=0x7f81e019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e53a4700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e0104470 0x7f81e019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e53a4700 1 -- 172.21.15.31:0/315147287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81e00ffdd0 msgr2=0x7f81e0196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e53a4700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81e00ffdd0 0x7f81e0196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e53a4700 1 -- 172.21.15.31:0/315147287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00095f0 con 0x7f81e00ff430 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e53a4700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 0x7f81e0196450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00d950 tx=0x7f81dc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00f890 con 0x7f81e00ff430 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f81dc010460 con 0x7f81e00ff430 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e010f1f0 con 0x7f81e00ff430 2023-12-14T18:34:33.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00e610 con 0x7f81e00ff430 2023-12-14T18:34:33.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.335+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e010f060 con 0x7f81e00ff430 2023-12-14T18:34:33.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.336+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e01087a0 con 0x7f81e00ff430 2023-12-14T18:34:33.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.337+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f81dc0108d0 con 0x7f81e00ff430 2023-12-14T18:34:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.338+0000 7f81d67fc700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f81cc0a1340 0x7f81cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.338+0000 7f81e4ba3700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f81cc0a1340 0x7f81cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.338+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f81dc0cce50 con 0x7f81e00ff430 2023-12-14T18:34:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.338+0000 7f81e4ba3700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f81cc0a1340 0x7f81cc0a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f81d00098e0 tx=0x7f81d0000bc0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.343+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f81dc095e70 con 0x7f81e00ff430 2023-12-14T18:34:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.601+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81e002cfe0 con 0x7f81e00ff430 2023-12-14T18:34:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.601+0000 7f81d67fc700 1 -- 172.21.15.31:0/315147287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10822 (secure 0 0 0) 0x7f81dc016090 con 0x7f81e00ff430 2023-12-14T18:34:33.604 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:34:33.604 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:31:40.963020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:34:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f81cc0a1340 msgr2=0x7f81cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f81cc0a1340 0x7f81cc0a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f81d00098e0 tx=0x7f81d0000bc0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 msgr2=0x7f81e0196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 0x7f81e0196450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00d950 tx=0x7f81dc00dc60 comp rx=0 tx=0).stop 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 shutdown_connections 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f81cc0a1340 0x7f81cc0a37f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e0104470 0x7f81e019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e00ff430 0x7f81e0196450 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 --2- 172.21.15.31:0/315147287 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81e00ffdd0 0x7f81e0196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 >> 172.21.15.31:0/315147287 conn(0x7f81e0076370 msgr2=0x7f81e0069110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 shutdown_connections 2023-12-14T18:34:33.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.604+0000 7f81e7608700 1 -- 172.21.15.31:0/315147287 wait complete. 2023-12-14T18:34:33.608 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-12-14T18:34:33.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:34:33.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:33 smithi176 ceph-mon[43506]: pgmap v207: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:34:33.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 -- 172.21.15.31:0/2041412340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff878101c30 msgr2=0x7ff87810a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 --2- 172.21.15.31:0/2041412340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff878101c30 0x7ff87810a300 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009990 tx=0x7ff86c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 -- 172.21.15.31:0/2041412340 shutdown_connections 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 --2- 172.21.15.31:0/2041412340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff87810a840 0x7ff87810cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 --2- 172.21.15.31:0/2041412340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff878101320 0x7ff8781016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 --2- 172.21.15.31:0/2041412340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff878101c30 0x7ff87810a300 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.756+0000 7ff87d681700 1 -- 172.21.15.31:0/2041412340 >> 172.21.15.31:0/2041412340 conn(0x7ff878076000 msgr2=0x7ff878076400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.757+0000 7ff87d681700 1 -- 172.21.15.31:0/2041412340 shutdown_connections 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.757+0000 7ff87d681700 1 -- 172.21.15.31:0/2041412340 wait complete. 2023-12-14T18:34:33.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.757+0000 7ff87d681700 1 Processor -- start 2023-12-14T18:34:33.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 -- start start 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff878101320 0x7ff8780ff150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff878101c30 0x7ff8780ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 0x7ff8781a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780ffec0 con 0x7ff87810a840 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8781a4ac0 con 0x7ff878101320 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff87d681700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8781a4c00 con 0x7ff878101c30 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 0x7ff8781a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 0x7ff8781a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52418/0 (socket says 172.21.15.31:52418) 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 -- 172.21.15.31:0/2301117316 learned_addr learned my addr 172.21.15.31:0/2301117316 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff878101320 0x7ff8780ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff878101c30 0x7ff8780ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 -- 172.21.15.31:0/2301117316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff878101c30 msgr2=0x7ff8780ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff878101c30 0x7ff8780ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 -- 172.21.15.31:0/2301117316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff878101320 msgr2=0x7ff8780ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff878101320 0x7ff8780ff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 -- 172.21.15.31:0/2301117316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c0095f0 con 0x7ff87810a840 2023-12-14T18:34:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.758+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 0x7ff8781a4580 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff870007ee0 tx=0x7ff87000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.759+0000 7ff85ffff700 1 -- 172.21.15.31:0/2301117316 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870017070 con 0x7ff87810a840 2023-12-14T18:34:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.759+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff878110310 con 0x7ff87810a840 2023-12-14T18:34:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.759+0000 7ff85ffff700 1 -- 172.21.15.31:0/2301117316 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff87000f040 con 0x7ff87810a840 2023-12-14T18:34:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.759+0000 7ff85ffff700 1 -- 172.21.15.31:0/2301117316 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870005550 con 0x7ff87810a840 2023-12-14T18:34:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.759+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff878110180 con 0x7ff87810a840 2023-12-14T18:34:33.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.761+0000 7ff85ffff700 1 -- 172.21.15.31:0/2301117316 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff87000db20 con 0x7ff87810a840 2023-12-14T18:34:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.762+0000 7ff85ffff700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff8600a12c0 0x7ff8600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.762+0000 7ff85ffff700 1 -- 172.21.15.31:0/2301117316 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff8700cc7a0 con 0x7ff87810a840 2023-12-14T18:34:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.762+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff8600a12c0 0x7ff8600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.762+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff8600a12c0 0x7ff8600a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff868009c60 tx=0x7ff868009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.762+0000 7ff88980fb80 1 Processor -- start 2023-12-14T18:34:33.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 -- start start 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x56043511d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511dc90 0x560435128610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 0x56043512af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604350075e0 con 0x560435128b50 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x560435076870 con 0x56043511d380 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff88980fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043506fd00 con 0x56043511dc90 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 0x56043512af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 0x56043512af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52422/0 (socket says 172.21.15.31:52422) 2023-12-14T18:34:33.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 -- 172.21.15.31:0/848501677 learned_addr learned my addr 172.21.15.31:0/848501677 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff876ffd700 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x56043511d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8767fc700 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511dc90 0x560435128610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 -- 172.21.15.31:0/848501677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511dc90 msgr2=0x560435128610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511dc90 0x560435128610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 -- 172.21.15.31:0/848501677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 msgr2=0x56043511d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x56043511d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.763+0000 7ff8777fe700 1 -- 172.21.15.31:0/848501677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560434ef13b0 con 0x560435128b50 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.764+0000 7ff8777fe700 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 0x56043512af30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff870095980 tx=0x7ff870004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3ad23e0e98d941 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.764+0000 7ff85d7fa700 1 -- 172.21.15.31:0/848501677 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87009d4b0 con 0x560435128b50 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.764+0000 7ff85d7fa700 1 -- 172.21.15.31:0/848501677 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8700a18d0 con 0x560435128b50 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.764+0000 7ff85d7fa700 1 -- 172.21.15.31:0/848501677 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870095b90 con 0x560435128b50 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.764+0000 7ff88980fb80 1 -- 172.21.15.31:0/848501677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 msgr2=0x56043512af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.764+0000 7ff88980fb80 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 0x56043512af30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff870095980 tx=0x7ff870004f10 comp rx=0 tx=0).stop 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- 172.21.15.31:0/848501677 shutdown_connections 2023-12-14T18:34:33.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560435128b50 0x56043512af30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x56043511d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 --2- 172.21.15.31:0/848501677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511dc90 0x560435128610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- 172.21.15.31:0/848501677 >> 172.21.15.31:0/848501677 conn(0x560435113ca0 msgr2=0x5604351197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- 172.21.15.31:0/848501677 shutdown_connections 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- 172.21.15.31:0/848501677 wait complete. 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- start start 2023-12-14T18:34:33.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 Processor -- start 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511d380 0x560435128490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435125820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560435128b50 0x560435125de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043500bf90 con 0x56043511dc90 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043500bc30 con 0x560435128b50 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x560434ef36a0 con 0x56043511d380 2023-12-14T18:34:33.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.765+0000 7ff88980fb80 10 client.0 _ll_register_callbacks cb 0x56043502f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435125820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435125820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52438/0 (socket says 172.21.15.31:52438) 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 -- 172.21.15.31:0/2854753633 learned_addr learned my addr 172.21.15.31:0/2854753633 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511d380 0x560435128490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560435128b50 0x560435125de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 -- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511d380 msgr2=0x560435128490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511d380 0x560435128490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 -- 172.21.15.31:0/2854753633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560435128b50 msgr2=0x560435125de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560435128b50 0x560435125de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 -- 172.21.15.31:0/2854753633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff878110180 con 0x56043511dc90 2023-12-14T18:34:33.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435125820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009990 tx=0x7ff86c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3069c75f9f9a3510 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff85aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:33.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff85aff5700 1 -- 172.21.15.31:0/2854753633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c01d070 con 0x56043511dc90 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff85aff5700 1 -- 172.21.15.31:0/2854753633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff86c005630 con 0x56043511dc90 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.766+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604351112b0 con 0x56043511dc90 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff85aff5700 1 -- 172.21.15.31:0/2854753633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff86c004500 con 0x56043511dc90 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 10 client.17031 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560435036170 con 0x56043511dc90 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff85aff5700 1 -- 172.21.15.31:0/2854753633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604351112b0 con 0x56043511dc90 2023-12-14T18:34:33.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff85aff5700 1 -- 172.21.15.31:0/2854753633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff86c00f690 con 0x56043511dc90 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 10 client.17031 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 10 client.17031 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 20 client.17031 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x560435138670 0x56043513aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 4 client.17031 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560435068ce0 con 0x560435138670 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.767+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x560435138670 0x56043513aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.768+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x560435138670 0x56043513aa80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9972b5700e2d7b5 server_cookie=ed65f1a94a251b7f in_seq=0 out_seq=0 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.768+0000 7ff85aff5700 10 client.17031 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.768+0000 7ff85aff5700 1 -- 172.21.15.31:0/2854753633 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560435068ce0 con 0x560435138670 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.768+0000 7ff85aff5700 10 client.17031 handle_command_reply: tid=0 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 client.17031 shutdown 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 msgr2=0x560435125820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435125820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009990 tx=0x7ff86c00bf60 comp rx=0 tx=0).stop 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 shutdown_connections 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x560435138670 0x56043513aa80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x560435128b50 0x560435125de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff85aff5700 0 client.17031 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56043511d380 0x560435128490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2854753633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435125820 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 >> 172.21.15.31:0/2854753633 conn(0x560435113ca0 msgr2=0x5604351193d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 shutdown_connections 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 1 -- 172.21.15.31:0/2854753633 wait complete. 2023-12-14T18:34:33.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.769+0000 7ff88980fb80 20 client.17031 trim_cache size 0 max 16384 2023-12-14T18:34:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.782+0000 7ff88980fb80 1 Processor -- start 2023-12-14T18:34:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 -- start start 2023-12-14T18:34:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x5604351280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x5604351210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x560435134f70 con 0x56043511dc90 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043511e3c0 con 0x56043511d380 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff88980fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043511e600 con 0x560435128b50 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435120bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52446/0 (socket says 172.21.15.31:52446) 2023-12-14T18:34:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 -- 172.21.15.31:0/2923089010 learned_addr learned my addr 172.21.15.31:0/2923089010 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x5604351280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x5604351210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 -- 172.21.15.31:0/2923089010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 msgr2=0x5604351210f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x5604351210f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 -- 172.21.15.31:0/2923089010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 msgr2=0x5604351280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x5604351280c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 -- 172.21.15.31:0/2923089010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56043512b200 con 0x56043511dc90 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x5604351210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x5604351280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.783+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435120bb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009ce0 tx=0x7ff86c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b48abb332dff108 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff85dffb700 1 -- 172.21.15.31:0/2923089010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c03a030 con 0x56043511dc90 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff85dffb700 1 -- 172.21.15.31:0/2923089010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff86c01e040 con 0x56043511dc90 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff85dffb700 1 -- 172.21.15.31:0/2923089010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c01d070 con 0x56043511dc90 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 -- 172.21.15.31:0/2923089010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 msgr2=0x560435120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435120bb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009ce0 tx=0x7ff86c009310 comp rx=0 tx=0).stop 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 -- 172.21.15.31:0/2923089010 shutdown_connections 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x5604351210f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511d380 0x5604351280c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2923089010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511dc90 0x560435120bb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 -- 172.21.15.31:0/2923089010 >> 172.21.15.31:0/2923089010 conn(0x560435113ca0 msgr2=0x560435134830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 -- 172.21.15.31:0/2923089010 shutdown_connections 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.784+0000 7ff88980fb80 1 -- 172.21.15.31:0/2923089010 wait complete. 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 -- start start 2023-12-14T18:34:33.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 Processor -- start 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 0x56043512d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511dc90 0x56043512dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x56043512e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043518b4f0 con 0x56043511d380 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043518b630 con 0x56043511dc90 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56043518b7a0 con 0x560435128b50 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff88980fb80 10 client.0 _ll_register_callbacks cb 0x56043502f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff876ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 0x56043512d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff876ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 0x56043512d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52462/0 (socket says 172.21.15.31:52462) 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff876ffd700 1 -- 172.21.15.31:0/2396861376 learned_addr learned my addr 172.21.15.31:0/2396861376 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff8777fe700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x56043512e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511dc90 0x56043512dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff876ffd700 1 -- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 msgr2=0x56043512e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.785+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x56043512e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff876ffd700 1 -- 172.21.15.31:0/2396861376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511dc90 msgr2=0x56043512dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511dc90 0x56043512dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff876ffd700 1 -- 172.21.15.31:0/2396861376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560435036170 con 0x56043511d380 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff876ffd700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 0x56043512d6a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x56043513b320 tx=0x7ff8680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe9d0997f558d51 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff85aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff85aff5700 1 -- 172.21.15.31:0/2396861376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868022040 con 0x56043511d380 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff85aff5700 1 -- 172.21.15.31:0/2396861376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7ff86802b4f0 con 0x56043511d380 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8700a18d0 con 0x56043511d380 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff85aff5700 1 -- 172.21.15.31:0/2396861376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff86802be50 con 0x56043511d380 2023-12-14T18:34:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff88980fb80 10 client.17037 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560434e9db60 con 0x56043511d380 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.786+0000 7ff85aff5700 1 -- 172.21.15.31:0/2396861376 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8700a18d0 con 0x56043511d380 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff85aff5700 1 -- 172.21.15.31:0/2396861376 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff868029800 con 0x56043511d380 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 10 client.17037 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 10 client.17037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 20 client.17037 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56043518cee0 0x56043518f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 4 client.17037 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:34:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560435068a00 con 0x56043518cee0 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56043518cee0 0x56043518f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff85aff5700 10 client.17037 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff8767fc700 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56043518cee0 0x56043518f2f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea9783ea5850e40e server_cookie=a0e9b87c9cf9bbbc in_seq=0 out_seq=0 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff85aff5700 1 -- 172.21.15.31:0/2396861376 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560435068a00 con 0x56043518cee0 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff85aff5700 10 client.17037 handle_command_reply: tid=0 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.787+0000 7ff88980fb80 1 client.17037 shutdown 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 msgr2=0x56043512d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 0x56043512d6a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x56043513b320 tx=0x7ff8680047c0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 shutdown_connections 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x56043518cee0 0x56043518f2f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560435128b50 0x56043512e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56043511d380 0x56043512d6a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff85aff5700 0 client.17037 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 --2- 172.21.15.31:0/2396861376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56043511dc90 0x56043512dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 >> 172.21.15.31:0/2396861376 conn(0x560435113ca0 msgr2=0x5604351280a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 shutdown_connections 2023-12-14T18:34:33.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 1 -- 172.21.15.31:0/2396861376 wait complete. 2023-12-14T18:34:33.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff88980fb80 20 client.17037 trim_cache size 0 max 16384 2023-12-14T18:34:33.802 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:34:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff8600a12c0 msgr2=0x7ff8600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff8600a12c0 0x7ff8600a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff868009c60 tx=0x7ff868009260 comp rx=0 tx=0).stop 2023-12-14T18:34:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 msgr2=0x7ff8781a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 0x7ff8781a4580 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff870007ee0 tx=0x7ff87000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 shutdown_connections 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff87810a840 0x7ff8781a4580 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff878101320 0x7ff8780ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff8600a12c0 0x7ff8600a3770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 --2- 172.21.15.31:0/2301117316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff878101c30 0x7ff8780ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.788+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 >> 172.21.15.31:0/2301117316 conn(0x7ff878076000 msgr2=0x7ff87810b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.789+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 shutdown_connections 2023-12-14T18:34:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.789+0000 7ff87d681700 1 -- 172.21.15.31:0/2301117316 wait complete. 2023-12-14T18:34:33.805 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:33 smithi031 ceph-mon[42085]: pgmap v207: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:34:33.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:34:33.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:33 smithi204 ceph-mon[43866]: pgmap v207: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:34:33.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 -- 172.21.15.31:0/1457465398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c1043d0 msgr2=0x7fa64c10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 --2- 172.21.15.31:0/1457465398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c1043d0 0x7fa64c10bfa0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa644009920 tx=0x7fa644009c30 comp rx=0 tx=0).stop 2023-12-14T18:34:33.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 -- 172.21.15.31:0/1457465398 shutdown_connections 2023-12-14T18:34:33.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 --2- 172.21.15.31:0/1457465398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c1043d0 0x7fa64c10bfa0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 --2- 172.21.15.31:0/1457465398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c0ff390 0x7fa64c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 --2- 172.21.15.31:0/1457465398 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 0x7fa64c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.943+0000 7fa6529a7700 1 -- 172.21.15.31:0/1457465398 >> 172.21.15.31:0/1457465398 conn(0x7fa64c075fd0 msgr2=0x7fa64c0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.944+0000 7fa6529a7700 1 -- 172.21.15.31:0/1457465398 shutdown_connections 2023-12-14T18:34:33.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.944+0000 7fa6529a7700 1 -- 172.21.15.31:0/1457465398 wait complete. 2023-12-14T18:34:33.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.944+0000 7fa6529a7700 1 Processor -- start 2023-12-14T18:34:33.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.944+0000 7fa6529a7700 1 -- start start 2023-12-14T18:34:33.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa6529a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 0x7fa64c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa6529a7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 0x7fa64c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa6529a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1043d0 0x7fa64c1a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa6529a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c198060 con 0x7fa64c0ff390 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa6529a7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c1a27a0 con 0x7fa64c0ffd30 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa6529a7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c1a2960 con 0x7fa64c1043d0 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 0x7fa64c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 0x7fa64c1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52474/0 (socket says 172.21.15.31:52474) 2023-12-14T18:34:33.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 -- 172.21.15.31:0/236803900 learned_addr learned my addr 172.21.15.31:0/236803900 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 0x7fa64c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa650f44700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1043d0 0x7fa64c1a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 -- 172.21.15.31:0/236803900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1043d0 msgr2=0x7fa64c1a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1043d0 0x7fa64c1a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 -- 172.21.15.31:0/236803900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 msgr2=0x7fa64c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 0x7fa64c197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 -- 172.21.15.31:0/236803900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009520 con 0x7fa64c0ff390 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 0x7fa64c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa650f44700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1043d0 0x7fa64c1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.945+0000 7fa64bfff700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 0x7fa64c1971e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c0c0 tx=0x7fa63c00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.946+0000 7fa6497fa700 1 -- 172.21.15.31:0/236803900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c018070 con 0x7fa64c0ff390 2023-12-14T18:34:33.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.946+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c10f1a0 con 0x7fa64c0ff390 2023-12-14T18:34:33.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.946+0000 7fa6497fa700 1 -- 172.21.15.31:0/236803900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63c00f040 con 0x7fa64c0ff390 2023-12-14T18:34:33.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.946+0000 7fa6497fa700 1 -- 172.21.15.31:0/236803900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00e020 con 0x7fa64c0ff390 2023-12-14T18:34:33.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.946+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c10f010 con 0x7fa64c0ff390 2023-12-14T18:34:33.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.948+0000 7fa6497fa700 1 -- 172.21.15.31:0/236803900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa63c0075c0 con 0x7fa64c0ff390 2023-12-14T18:34:33.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.949+0000 7fa6497fa700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa6340a12c0 0x7fa6340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.949+0000 7fa6497fa700 1 -- 172.21.15.31:0/236803900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fa63c0ccf70 con 0x7fa64c0ff390 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.949+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa6340a12c0 0x7fa6340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.949+0000 7fa65eb35b80 1 Processor -- start 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.949+0000 7fa65eb35b80 1 -- start start 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.949+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa6340a12c0 0x7fa6340a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa640005e10 tx=0x7fa640005d60 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556662bb5e0 con 0x5556663e51f0 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55566632a870 con 0x5556663d18c0 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555666323d00 con 0x5556663d2df0 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52488/0 (socket says 172.21.15.31:52488) 2023-12-14T18:34:33.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 -- 172.21.15.31:0/2516048232 learned_addr learned my addr 172.21.15.31:0/2516048232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa64bfff700 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 -- 172.21.15.31:0/2516048232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 msgr2=0x5556663d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 -- 172.21.15.31:0/2516048232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 msgr2=0x5556663d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 -- 172.21.15.31:0/2516048232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556661a53b0 con 0x5556663e51f0 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa650f44700 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e75d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa644009230 tx=0x7fa6440058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8898f295c05af75f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa6327fc700 1 -- 172.21.15.31:0/2516048232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64401c070 con 0x5556663e51f0 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa6327fc700 1 -- 172.21.15.31:0/2516048232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64400b560 con 0x5556663e51f0 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.950+0000 7fa6327fc700 1 -- 172.21.15.31:0/2516048232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440053c0 con 0x5556663e51f0 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 -- 172.21.15.31:0/2516048232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 msgr2=0x5556663e75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e75d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa644009230 tx=0x7fa6440058e0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 -- 172.21.15.31:0/2516048232 shutdown_connections 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e75d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/2516048232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 -- 172.21.15.31:0/2516048232 >> 172.21.15.31:0/2516048232 conn(0x5556663c7ca0 msgr2=0x5556663cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 -- 172.21.15.31:0/2516048232 shutdown_connections 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.951+0000 7fa65eb35b80 1 -- 172.21.15.31:0/2516048232 wait complete. 2023-12-14T18:34:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 -- start start 2023-12-14T18:34:33.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 Processor -- start 2023-12-14T18:34:33.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 0x5556663d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663e51f0 0x5556663d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556662bff90 con 0x5556663d18c0 2023-12-14T18:34:33.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556662bfc30 con 0x5556663e51f0 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556661a76a0 con 0x5556663d2df0 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa65eb35b80 10 client.0 _ll_register_callbacks cb 0x5556662e3910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa64bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 0x5556663d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa64b7fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa64bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 0x5556663d7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52504/0 (socket says 172.21.15.31:52504) 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.952+0000 7fa64bfff700 1 -- 172.21.15.31:0/4195670098 learned_addr learned my addr 172.21.15.31:0/4195670098 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa650f44700 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663e51f0 0x5556663d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa64bfff700 1 -- 172.21.15.31:0/4195670098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 msgr2=0x5556663d52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa64bfff700 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa64bfff700 1 -- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663e51f0 msgr2=0x5556663d5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa64bfff700 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663e51f0 0x5556663d5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa64bfff700 1 -- 172.21.15.31:0/4195670098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c10c7e0 con 0x5556663d18c0 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa64bfff700 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 0x5556663d7fc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0998f0 tx=0x7fa63c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdba607dd743c1f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa62b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa62b7fe700 1 -- 172.21.15.31:0/4195670098 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0a2040 con 0x5556663d18c0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa62b7fe700 1 -- 172.21.15.31:0/4195670098 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fa63c0b2200 con 0x5556663d18c0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556663c52b0 con 0x5556663d18c0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa62b7fe700 1 -- 172.21.15.31:0/4195670098 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa63c0b23c0 con 0x5556663d18c0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa65eb35b80 10 client.17049 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5556662ea170 con 0x5556663d18c0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.953+0000 7fa62b7fe700 1 -- 172.21.15.31:0/4195670098 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa63c0c8080 con 0x5556663d18c0 2023-12-14T18:34:33.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa62b7fe700 1 -- 172.21.15.31:0/4195670098 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa63c004c60 con 0x5556663d18c0 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa65eb35b80 10 client.17049 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa65eb35b80 10 client.17049 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa65eb35b80 20 client.17049 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5556663f4b90 0x5556663f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa65eb35b80 4 client.17049 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55566631cce0 con 0x5556663f4b90 2023-12-14T18:34:33.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5556663f4b90 0x5556663f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa62b7fe700 10 client.17049 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.954+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5556663f4b90 0x5556663f6fa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=493f5dd8995b0e0 server_cookie=6e8ee91e969bb7f7 in_seq=0 out_seq=0 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa62b7fe700 1 -- 172.21.15.31:0/4195670098 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55566631cce0 con 0x5556663f4b90 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa62b7fe700 10 client.17049 handle_command_reply: tid=0 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 client.17049 shutdown 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 msgr2=0x5556663d7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 0x5556663d7fc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0998f0 tx=0x7fa63c00ce30 comp rx=0 tx=0).stop 2023-12-14T18:34:33.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 shutdown_connections 2023-12-14T18:34:33.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663e51f0 0x5556663d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663d18c0 0x5556663d7fc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5556663f4b90 0x5556663f6fa0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4195670098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa62b7fe700 0 client.17049 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:33.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 >> 172.21.15.31:0/4195670098 conn(0x5556663c7ca0 msgr2=0x5556663cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.955+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 shutdown_connections 2023-12-14T18:34:33.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.956+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4195670098 wait complete. 2023-12-14T18:34:33.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.956+0000 7fa65eb35b80 20 client.17049 trim_cache size 0 max 16384 2023-12-14T18:34:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 Processor -- start 2023-12-14T18:34:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 -- start start 2023-12-14T18:34:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663d7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556663d25f0 con 0x5556663e51f0 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556663c7820 con 0x5556663d18c0 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.969+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556663d5e40 con 0x5556663d2df0 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663d7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663d7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52520/0 (socket says 172.21.15.31:52520) 2023-12-14T18:34:33.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 -- 172.21.15.31:0/4057191050 learned_addr learned my addr 172.21.15.31:0/4057191050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa64bfff700 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 -- 172.21.15.31:0/4057191050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 msgr2=0x5556663d7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 -- 172.21.15.31:0/4057191050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 msgr2=0x5556663e14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663e14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 -- 172.21.15.31:0/4057191050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556663c94e0 con 0x5556663e51f0 2023-12-14T18:34:33.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa64bfff700 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663e14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa650f44700 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663d7fc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa644025040 tx=0x7fa644005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f92973b63c4f4cd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa632ffd700 1 -- 172.21.15.31:0/4057191050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64401c040 con 0x5556663e51f0 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa632ffd700 1 -- 172.21.15.31:0/4057191050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64400bb90 con 0x5556663e51f0 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.970+0000 7fa632ffd700 1 -- 172.21.15.31:0/4057191050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440046f0 con 0x5556663e51f0 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4057191050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 msgr2=0x5556663d7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663d7fc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa644025040 tx=0x7fa644005b40 comp rx=0 tx=0).stop 2023-12-14T18:34:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4057191050 shutdown_connections 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663d7fc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d18c0 0x5556663e14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/4057191050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d2df0 0x5556663d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4057191050 >> 172.21.15.31:0/4057191050 conn(0x5556663c7ca0 msgr2=0x5556663f0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4057191050 shutdown_connections 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 -- 172.21.15.31:0/4057191050 wait complete. 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.971+0000 7fa65eb35b80 1 -- start start 2023-12-14T18:34:33.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 Processor -- start 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d18c0 0x5556663eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d2df0 0x5556663ebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556663ea2c0 con 0x5556663e51f0 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556663ea430 con 0x5556663d2df0 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556663ea5f0 con 0x5556663d18c0 2023-12-14T18:34:33.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa65eb35b80 10 client.0 _ll_register_callbacks cb 0x7fa63c099800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52526/0 (socket says 172.21.15.31:52526) 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 -- 172.21.15.31:0/1019606554 learned_addr learned my addr 172.21.15.31:0/1019606554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa64b7fe700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d2df0 0x5556663ebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa64bfff700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d18c0 0x5556663eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 -- 172.21.15.31:0/1019606554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d18c0 msgr2=0x5556663eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d18c0 0x5556663eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 -- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d2df0 msgr2=0x5556663ebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d2df0 0x5556663ebdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.972+0000 7fa650f44700 1 -- 172.21.15.31:0/1019606554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556661a53b0 con 0x5556663e51f0 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa650f44700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e9bb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa64400a010 tx=0x7fa644005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb3da7088c56939 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa62b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa62b7fe700 1 -- 172.21.15.31:0/1019606554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64401c040 con 0x5556663e51f0 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa62b7fe700 1 -- 172.21.15.31:0/1019606554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fa64400f7e0 con 0x5556663e51f0 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa64400b560 con 0x5556663e51f0 2023-12-14T18:34:33.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa62b7fe700 1 -- 172.21.15.31:0/1019606554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64400b9e0 con 0x5556663e51f0 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 10 client.17055 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555666151b60 con 0x5556663e51f0 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa62b7fe700 1 -- 172.21.15.31:0/1019606554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa64400b560 con 0x5556663e51f0 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa62b7fe700 1 -- 172.21.15.31:0/1019606554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa644029070 con 0x5556663e51f0 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 10 client.17055 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 10 client.17055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 20 client.17055 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:33.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555666449750 0x55566644bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 4 client.17055 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.973+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55566631ca00 con 0x555666449750 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.974+0000 7fa64bfff700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555666449750 0x55566644bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.974+0000 7fa64bfff700 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555666449750 0x55566644bb30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94ebd1aeb8798de3 server_cookie=6b70d6cecc8b79a4 in_seq=0 out_seq=0 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.974+0000 7fa62b7fe700 10 client.17055 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.974+0000 7fa62b7fe700 1 -- 172.21.15.31:0/1019606554 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55566631ca00 con 0x555666449750 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.974+0000 7fa62b7fe700 10 client.17055 handle_command_reply: tid=0 2023-12-14T18:34:33.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.974+0000 7fa65eb35b80 1 client.17055 shutdown 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 msgr2=0x5556663e9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e9bb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa64400a010 tx=0x7fa644005670 comp rx=0 tx=0).stop 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 shutdown_connections 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5556663e51f0 0x5556663e9bb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555666449750 0x55566644bb30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5556663d18c0 0x5556663eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 --2- 172.21.15.31:0/1019606554 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5556663d2df0 0x5556663ebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa62b7fe700 0 client.17055 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:34:33.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 >> 172.21.15.31:0/1019606554 conn(0x5556663c7ca0 msgr2=0x5556663f0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 shutdown_connections 2023-12-14T18:34:33.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 1 -- 172.21.15.31:0/1019606554 wait complete. 2023-12-14T18:34:33.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa65eb35b80 20 client.17055 trim_cache size 0 max 16384 2023-12-14T18:34:33.992 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:34:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa6340a12c0 msgr2=0x7fa6340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa6340a12c0 0x7fa6340a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa640005e10 tx=0x7fa640005d60 comp rx=0 tx=0).stop 2023-12-14T18:34:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 msgr2=0x7fa64c1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 0x7fa64c1971e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c0c0 tx=0x7fa63c00c480 comp rx=0 tx=0).stop 2023-12-14T18:34:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 shutdown_connections 2023-12-14T18:34:33.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa6340a12c0 0x7fa6340a3770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa64c1043d0 0x7fa64c1a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa64c0ff390 0x7fa64c1971e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 --2- 172.21.15.31:0/236803900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa64c0ffd30 0x7fa64c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.975+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 >> 172.21.15.31:0/236803900 conn(0x7fa64c075fd0 msgr2=0x7fa64c0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.976+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 shutdown_connections 2023-12-14T18:34:33.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:33.976+0000 7fa6529a7700 1 -- 172.21.15.31:0/236803900 wait complete. 2023-12-14T18:34:33.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:34:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 -- 172.21.15.31:0/1060124473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c0690f0 msgr2=0x7fdf0c0694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 --2- 172.21.15.31:0/1060124473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c0690f0 0x7fdf0c0694c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdf080099c0 tx=0x7fdf08009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 -- 172.21.15.31:0/1060124473 shutdown_connections 2023-12-14T18:34:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 --2- 172.21.15.31:0/1060124473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf0c1054f0 0x7fdf0c10d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 --2- 172.21.15.31:0/1060124473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c0690f0 0x7fdf0c0694c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 --2- 172.21.15.31:0/1060124473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf0c069a90 0x7fdf0c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 -- 172.21.15.31:0/1060124473 >> 172.21.15.31:0/1060124473 conn(0x7fdf0c075fa0 msgr2=0x7fdf0c0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 -- 172.21.15.31:0/1060124473 shutdown_connections 2023-12-14T18:34:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.133+0000 7fdf143d4700 1 -- 172.21.15.31:0/1060124473 wait complete. 2023-12-14T18:34:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 Processor -- start 2023-12-14T18:34:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 -- start start 2023-12-14T18:34:34.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf0c0690f0 0x7fdf0c196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 0x7fdf0c1969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf0c1054f0 0x7fdf0c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c19b470 con 0x7fdf0c069a90 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c19b5e0 con 0x7fdf0c0690f0 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf143d4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c19b7a0 con 0x7fdf0c1054f0 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 0x7fdf0c1969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 0x7fdf0c1969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52554/0 (socket says 172.21.15.31:52554) 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 -- 172.21.15.31:0/499074087 learned_addr learned my addr 172.21.15.31:0/499074087 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf12170700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf0c0690f0 0x7fdf0c196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf12971700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf0c1054f0 0x7fdf0c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 -- 172.21.15.31:0/499074087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf0c1054f0 msgr2=0x7fdf0c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf0c1054f0 0x7fdf0c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 -- 172.21.15.31:0/499074087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf0c0690f0 msgr2=0x7fdf0c196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.134+0000 7fdf1196f700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf0c0690f0 0x7fdf0c196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf1196f700 1 -- 172.21.15.31:0/499074087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080095f0 con 0x7fdf0c069a90 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf1196f700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 0x7fdf0c1969d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00e940 tx=0x7fdefc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf037fe700 1 -- 172.21.15.31:0/499074087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc009730 con 0x7fdf0c069a90 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf037fe700 1 -- 172.21.15.31:0/499074087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc010430 con 0x7fdf0c069a90 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c110300 con 0x7fdf0c069a90 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf037fe700 1 -- 172.21.15.31:0/499074087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00cea0 con 0x7fdf0c069a90 2023-12-14T18:34:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.135+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c110170 con 0x7fdf0c069a90 2023-12-14T18:34:34.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.137+0000 7fdf037fe700 1 -- 172.21.15.31:0/499074087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdefc010810 con 0x7fdf0c069a90 2023-12-14T18:34:34.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.138+0000 7fdf037fe700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdef80a12c0 0x7fdef80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.138+0000 7fdf037fe700 1 -- 172.21.15.31:0/499074087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fdefc014070 con 0x7fdf0c069a90 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.138+0000 7fdf12170700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdef80a12c0 0x7fdef80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.139+0000 7fdf12170700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdef80a12c0 0x7fdef80a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009990 tx=0x7fdf08005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.139+0000 7fdf20562b80 1 Processor -- start 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.139+0000 7fdf20562b80 1 -- start start 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 0x559250287750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287c90 0x559250292610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250294f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf20562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592501715e0 con 0x559250287380 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf20562b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592501e0870 con 0x559250287c90 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf20562b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592501d9d00 con 0x559250292b50 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 0x559250287750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 0x559250287750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52568/0 (socket says 172.21.15.31:52568) 2023-12-14T18:34:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 -- 172.21.15.31:0/1365295337 learned_addr learned my addr 172.21.15.31:0/1365295337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12971700 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250294f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf1196f700 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287c90 0x559250292610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 -- 172.21.15.31:0/1365295337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 msgr2=0x559250294f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250294f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 -- 172.21.15.31:0/1365295337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287c90 msgr2=0x559250292610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287c90 0x559250292610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 -- 172.21.15.31:0/1365295337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925005b3b0 con 0x559250287380 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf1196f700 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287c90 0x559250292610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf12170700 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 0x559250287750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdf080095a0 tx=0x7fdf0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a1e43630aa5da server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.140+0000 7fdf00dac700 1 -- 172.21.15.31:0/1365295337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08003880 con 0x559250287380 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf00dac700 1 -- 172.21.15.31:0/1365295337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0801e040 con 0x559250287380 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf00dac700 1 -- 172.21.15.31:0/1365295337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800f480 con 0x559250287380 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 -- 172.21.15.31:0/1365295337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 msgr2=0x559250287750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 0x559250287750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdf080095a0 tx=0x7fdf0801f000 comp rx=0 tx=0).stop 2023-12-14T18:34:34.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 -- 172.21.15.31:0/1365295337 shutdown_connections 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250294f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287380 0x559250287750 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 --2- 172.21.15.31:0/1365295337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287c90 0x559250292610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 -- 172.21.15.31:0/1365295337 >> 172.21.15.31:0/1365295337 conn(0x55925027dca0 msgr2=0x5592502837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 -- 172.21.15.31:0/1365295337 shutdown_connections 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.141+0000 7fdf20562b80 1 -- 172.21.15.31:0/1365295337 wait complete. 2023-12-14T18:34:34.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 -- start start 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 Processor -- start 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x55925028f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x55925028fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250175f90 con 0x559250287c90 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250175c30 con 0x559250287380 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925005d6a0 con 0x559250292b50 2023-12-14T18:34:34.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf20562b80 10 client.0 _ll_register_callbacks cb 0x559250199910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x55925028f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x55925028f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52576/0 (socket says 172.21.15.31:52576) 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf1196f700 1 -- 172.21.15.31:0/3208449721 learned_addr learned my addr 172.21.15.31:0/3208449721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf12170700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.142+0000 7fdf12971700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x55925028fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf1196f700 1 -- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 msgr2=0x55925028fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf1196f700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x55925028fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf1196f700 1 -- 172.21.15.31:0/3208449721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 msgr2=0x5592502924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf1196f700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf1196f700 1 -- 172.21.15.31:0/3208449721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c10d900 con 0x559250287c90 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf1196f700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x55925028f760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdefc099c50 tx=0x7fdefc09a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12311460df9ddb5a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdef5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdef5ffb700 1 -- 172.21.15.31:0/3208449721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc09a3b0 con 0x559250287c90 2023-12-14T18:34:34.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdef5ffb700 1 -- 172.21.15.31:0/3208449721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fdefc0a23a0 con 0x559250287c90 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55925027b2b0 con 0x559250287c90 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdef5ffb700 1 -- 172.21.15.31:0/3208449721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc0b8170 con 0x559250287c90 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf20562b80 10 client.17067 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5592501a0170 con 0x559250287c90 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.143+0000 7fdef5ffb700 1 -- 172.21.15.31:0/3208449721 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdefc0c8080 con 0x559250287c90 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdef5ffb700 1 -- 172.21.15.31:0/3208449721 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdefc0a2010 con 0x559250287c90 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf20562b80 10 client.17067 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:34.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf20562b80 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf20562b80 20 client.17067 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf20562b80 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502a2490 0x5592502a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf20562b80 4 client.17067 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592501d2ce0 con 0x5592502a2490 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.144+0000 7fdf12170700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502a2490 0x5592502a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.145+0000 7fdef5ffb700 10 client.17067 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:34.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.145+0000 7fdf12170700 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502a2490 0x5592502a48a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01ff2e7e28dce9 server_cookie=583f8ea3085e053c in_seq=0 out_seq=0 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.145+0000 7fdef5ffb700 1 -- 172.21.15.31:0/3208449721 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592501d2ce0 con 0x5592502a2490 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.145+0000 7fdef5ffb700 10 client.17067 handle_command_reply: tid=0 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.145+0000 7fdf20562b80 1 client.17067 shutdown 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 msgr2=0x55925028f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x55925028f760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdefc099c50 tx=0x7fdefc09a040 comp rx=0 tx=0).stop 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 shutdown_connections 2023-12-14T18:34:34.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502a2490 0x5592502a48a0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x55925028fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdef5ffb700 0 client.17067 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 --2- 172.21.15.31:0/3208449721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x55925028f760 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 >> 172.21.15.31:0/3208449721 conn(0x55925027dca0 msgr2=0x5592502833d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 shutdown_connections 2023-12-14T18:34:34.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 1 -- 172.21.15.31:0/3208449721 wait complete. 2023-12-14T18:34:34.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.146+0000 7fdf20562b80 20 client.17067 trim_cache size 0 max 16384 2023-12-14T18:34:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 Processor -- start 2023-12-14T18:34:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 -- start start 2023-12-14T18:34:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250287380 0x559250291f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x5592502924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250292b50 0x55925028aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250290010 con 0x559250287c90 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55925028b5f0 con 0x559250292b50 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf20562b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250288d30 con 0x559250287380 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x5592502924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x5592502924b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52586/0 (socket says 172.21.15.31:52586) 2023-12-14T18:34:34.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 -- 172.21.15.31:0/78193474 learned_addr learned my addr 172.21.15.31:0/78193474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf12971700 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250292b50 0x55925028aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf12170700 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250287380 0x559250291f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 -- 172.21.15.31:0/78193474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250287380 msgr2=0x559250291f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250287380 0x559250291f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 -- 172.21.15.31:0/78193474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250292b50 msgr2=0x55925028aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250292b50 0x55925028aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf1196f700 1 -- 172.21.15.31:0/78193474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55925019fec0 con 0x559250287c90 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.160+0000 7fdf12170700 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250287380 0x559250291f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf1196f700 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x5592502924b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0ca680 tx=0x7fdefc094ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f7a8fedd1518d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf015ad700 1 -- 172.21.15.31:0/78193474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc099d20 con 0x559250287c90 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf015ad700 1 -- 172.21.15.31:0/78193474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55925027b2b0 con 0x559250287c90 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf015ad700 1 -- 172.21.15.31:0/78193474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc099440 con 0x559250287c90 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 -- 172.21.15.31:0/78193474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 msgr2=0x5592502924b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x5592502924b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0ca680 tx=0x7fdefc094ef0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 -- 172.21.15.31:0/78193474 shutdown_connections 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250292b50 0x55925028aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250287380 0x559250291f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 --2- 172.21.15.31:0/78193474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x5592502924b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.161+0000 7fdf20562b80 1 -- 172.21.15.31:0/78193474 >> 172.21.15.31:0/78193474 conn(0x55925027dca0 msgr2=0x55925029ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 -- 172.21.15.31:0/78193474 shutdown_connections 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 -- 172.21.15.31:0/78193474 wait complete. 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 -- start start 2023-12-14T18:34:34.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 Processor -- start 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x559250299810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250295800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250295f10 con 0x559250287c90 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250296080 con 0x559250287380 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.162+0000 7fdf20562b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559250296240 con 0x559250292b50 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf20562b80 10 client.0 _ll_register_callbacks cb 0x7fdefc099b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:34.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x559250299810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x559250299810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52598/0 (socket says 172.21.15.31:52598) 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 -- 172.21.15.31:0/772479148 learned_addr learned my addr 172.21.15.31:0/772479148 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf12971700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250295800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf12170700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 -- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 msgr2=0x559250295800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250295800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 -- 172.21.15.31:0/772479148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 msgr2=0x5592502992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 -- 172.21.15.31:0/772479148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592501a0170 con 0x559250287c90 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf12971700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250295800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf1196f700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x559250299810 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdefc099ee0 tx=0x7fdefc00c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bf94fc1ca06b04 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdef5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdef5ffb700 1 -- 172.21.15.31:0/772479148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc099640 con 0x559250287c90 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdef5ffb700 1 -- 172.21.15.31:0/772479148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fdefc0a2120 con 0x559250287c90 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf0801e040 con 0x559250287c90 2023-12-14T18:34:34.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdef5ffb700 1 -- 172.21.15.31:0/772479148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc099f50 con 0x559250287c90 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf20562b80 10 client.17073 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.163+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5592501869a0 con 0x559250287c90 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdef5ffb700 1 -- 172.21.15.31:0/772479148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0801e040 con 0x559250287c90 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdef5ffb700 1 -- 172.21.15.31:0/772479148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdefc0ca170 con 0x559250287c90 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf20562b80 10 client.17073 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf20562b80 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:34.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf20562b80 20 client.17073 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf20562b80 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502f7090 0x5592502f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf20562b80 4 client.17073 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5592501d2a00 con 0x5592502f7090 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.164+0000 7fdf12170700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502f7090 0x5592502f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf12170700 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502f7090 0x5592502f9470 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d09d4b6ccb756e server_cookie=5c5812df431f386d in_seq=0 out_seq=0 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdef5ffb700 10 client.17073 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdef5ffb700 1 -- 172.21.15.31:0/772479148 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5592501d2a00 con 0x5592502f7090 2023-12-14T18:34:34.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdef5ffb700 10 client.17073 handle_command_reply: tid=0 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 client.17073 shutdown 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 msgr2=0x559250299810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x559250299810 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdefc099ee0 tx=0x7fdefc00c3d0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 shutdown_connections 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5592502f7090 0x5592502f9470 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559250292b50 0x559250295800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdef5ffb700 0 client.17073 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:34.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559250287380 0x5592502992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 --2- 172.21.15.31:0/772479148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559250287c90 0x559250299810 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 >> 172.21.15.31:0/772479148 conn(0x55925027dca0 msgr2=0x55925028aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:34.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 shutdown_connections 2023-12-14T18:34:34.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 1 -- 172.21.15.31:0/772479148 wait complete. 2023-12-14T18:34:34.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.165+0000 7fdf20562b80 20 client.17073 trim_cache size 0 max 16384 2023-12-14T18:34:34.180 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:34:34.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdef80a12c0 msgr2=0x7fdef80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdef80a12c0 0x7fdef80a3770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009990 tx=0x7fdf08005fb0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 msgr2=0x7fdf0c1969d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:34.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 0x7fdf0c1969d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00e940 tx=0x7fdefc00ec50 comp rx=0 tx=0).stop 2023-12-14T18:34:34.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 shutdown_connections 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdef80a12c0 0x7fdef80a3770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf0c1054f0 0x7fdf0c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdf0c0690f0 0x7fdf0c196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 --2- 172.21.15.31:0/499074087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf0c069a90 0x7fdf0c1969d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 >> 172.21.15.31:0/499074087 conn(0x7fdf0c075fa0 msgr2=0x7fdf0c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 shutdown_connections 2023-12-14T18:34:34.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:34.166+0000 7fdf143d4700 1 -- 172.21.15.31:0/499074087 wait complete. 2023-12-14T18:34:34.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:34 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/315147287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:34:34.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:34 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/315147287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:34:34.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:34 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/315147287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:34:35.184 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:34:35.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:34:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 -- 172.21.15.31:0/3349971141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ffd30 msgr2=0x7f9840103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 --2- 172.21.15.31:0/3349971141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ffd30 0x7f9840103cf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f982c0099c0 tx=0x7f982c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 -- 172.21.15.31:0/3349971141 shutdown_connections 2023-12-14T18:34:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 --2- 172.21.15.31:0/3349971141 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98401043d0 0x7f984010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 --2- 172.21.15.31:0/3349971141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98400ff390 0x7f98400ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 --2- 172.21.15.31:0/3349971141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ffd30 0x7f9840103cf0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 -- 172.21.15.31:0/3349971141 >> 172.21.15.31:0/3349971141 conn(0x7f9840075fd0 msgr2=0x7f98400763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 -- 172.21.15.31:0/3349971141 shutdown_connections 2023-12-14T18:34:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.333+0000 7f98471d7700 1 -- 172.21.15.31:0/3349971141 wait complete. 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 Processor -- start 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 -- start start 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 0x7f9840196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98400ffd30 0x7f98401968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98401043d0 0x7f984019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984019b340 con 0x7f98400ff390 2023-12-14T18:34:35.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984019b4b0 con 0x7f98400ffd30 2023-12-14T18:34:35.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f98471d7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984019b670 con 0x7f98401043d0 2023-12-14T18:34:35.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 0x7f9840196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 0x7f9840196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53382/0 (socket says 172.21.15.31:53382) 2023-12-14T18:34:35.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 -- 172.21.15.31:0/2255138786 learned_addr learned my addr 172.21.15.31:0/2255138786 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9845774700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98401043d0 0x7f984019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f983ffff700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98400ffd30 0x7f98401968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 -- 172.21.15.31:0/2255138786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98401043d0 msgr2=0x7f984019ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98401043d0 0x7f984019ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 -- 172.21.15.31:0/2255138786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98400ffd30 msgr2=0x7f98401968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98400ffd30 0x7f98401968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.334+0000 7f9844f73700 1 -- 172.21.15.31:0/2255138786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9834009520 con 0x7f98400ff390 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.335+0000 7f9844f73700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 0x7f9840196360 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f983400eb30 tx=0x7f983400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.335+0000 7f983dffb700 1 -- 172.21.15.31:0/2255138786 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834009880 con 0x7f98400ff390 2023-12-14T18:34:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.335+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982c0095f0 con 0x7f98400ff390 2023-12-14T18:34:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.335+0000 7f983dffb700 1 -- 172.21.15.31:0/2255138786 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98340044e0 con 0x7f98400ff390 2023-12-14T18:34:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.335+0000 7f983dffb700 1 -- 172.21.15.31:0/2255138786 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834010430 con 0x7f98400ff390 2023-12-14T18:34:35.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.335+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9840110310 con 0x7f98400ff390 2023-12-14T18:34:35.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.337+0000 7f983dffb700 1 -- 172.21.15.31:0/2255138786 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9834003ee0 con 0x7f98400ff390 2023-12-14T18:34:35.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.338+0000 7f983dffb700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f98300a12c0 0x7f98300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.338+0000 7f983ffff700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f98300a12c0 0x7f98300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.338+0000 7f983dffb700 1 -- 172.21.15.31:0/2255138786 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9834014070 con 0x7f98400ff390 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f983ffff700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f98300a12c0 0x7f98300a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f982c009990 tx=0x7f982c019040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 Processor -- start 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 -- start start 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cbd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbdd10 0x555f97cc8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 0x555f97ccafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97ba7660 con 0x555f97cc8bd0 2023-12-14T18:34:35.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97c168f0 con 0x555f97cbd400 2023-12-14T18:34:35.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.339+0000 7f9853365b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97c0fd80 con 0x555f97cbdd10 2023-12-14T18:34:35.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 0x555f97ccafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 0x555f97ccafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53392/0 (socket says 172.21.15.31:53392) 2023-12-14T18:34:35.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 -- 172.21.15.31:0/39615077 learned_addr learned my addr 172.21.15.31:0/39615077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f983ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbdd10 0x555f97cc8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9844f73700 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cbd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 -- 172.21.15.31:0/39615077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbdd10 msgr2=0x555f97cc8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbdd10 0x555f97cc8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 -- 172.21.15.31:0/39615077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 msgr2=0x555f97cbd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cbd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 -- 172.21.15.31:0/39615077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f97a91430 con 0x555f97cc8bd0 2023-12-14T18:34:35.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f9845774700 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 0x555f97ccafb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9838009440 tx=0x7f9838009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e5f386f819f23a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f982affd700 1 -- 172.21.15.31:0/39615077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983800e070 con 0x555f97cc8bd0 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f982affd700 1 -- 172.21.15.31:0/39615077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983800f040 con 0x555f97cc8bd0 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.340+0000 7f982affd700 1 -- 172.21.15.31:0/39615077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838015b00 con 0x555f97cc8bd0 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 -- 172.21.15.31:0/39615077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 msgr2=0x555f97ccafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 0x555f97ccafb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9838009440 tx=0x7f9838009920 comp rx=0 tx=0).stop 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 -- 172.21.15.31:0/39615077 shutdown_connections 2023-12-14T18:34:35.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cc8bd0 0x555f97ccafb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cbd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 --2- 172.21.15.31:0/39615077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbdd10 0x555f97cc8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 -- 172.21.15.31:0/39615077 >> 172.21.15.31:0/39615077 conn(0x555f97cb3d20 msgr2=0x555f97cb9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 -- 172.21.15.31:0/39615077 shutdown_connections 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.341+0000 7f9853365b80 1 -- 172.21.15.31:0/39615077 wait complete. 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 -- start start 2023-12-14T18:34:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 Processor -- start 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cc8bd0 0x555f97cc5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97bac010 con 0x555f97cbdd10 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97babcb0 con 0x555f97cc8bd0 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.342+0000 7f9853365b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97a93720 con 0x555f97cbd400 2023-12-14T18:34:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f9853365b80 10 client.0 _ll_register_callbacks cb 0x555f97bcf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53406/0 (socket says 172.21.15.31:53406) 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 -- 172.21.15.31:0/3155349658 learned_addr learned my addr 172.21.15.31:0/3155349658 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f9844f73700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f9845774700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cc8bd0 0x555f97cc5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 -- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 msgr2=0x555f97cc8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 -- 172.21.15.31:0/3155349658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cc8bd0 msgr2=0x555f97cc5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cc8bd0 0x555f97cc5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 -- 172.21.15.31:0/3155349658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98380090f0 con 0x555f97cbdd10 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f9845774700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cc8bd0 0x555f97cc5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f983ffff700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc5800 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9840100100 tx=0x7f982c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4651eb9c3f683f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f981ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f981ffff700 1 -- 172.21.15.31:0/3155349658 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c01c040 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f981ffff700 1 -- 172.21.15.31:0/3155349658 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f982c00f460 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.343+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f97cb1330 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f981ffff700 1 -- 172.21.15.31:0/3155349658 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982c00f620 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 10 client.17085 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f97bd61f0 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f981ffff700 1 -- 172.21.15.31:0/3155349658 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f97cb1330 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f981ffff700 1 -- 172.21.15.31:0/3155349658 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f982c02f830 con 0x555f97cbdd10 2023-12-14T18:34:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 10 client.17085 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 20 client.17085 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cd85f0 0x555f97cdaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 4 client.17085 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f97c08d60 con 0x555f97cd85f0 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.344+0000 7f9844f73700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cd85f0 0x555f97cdaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.345+0000 7f9844f73700 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cd85f0 0x555f97cdaa00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ab8fcce62fd939 server_cookie=485093eaa2fd120f in_seq=0 out_seq=0 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.345+0000 7f981ffff700 10 client.17085 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f981ffff700 1 -- 172.21.15.31:0/3155349658 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f97c08d60 con 0x555f97cd85f0 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f981ffff700 10 client.17085 handle_command_reply: tid=0 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 client.17085 shutdown 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 msgr2=0x555f97cc5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc5800 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9840100100 tx=0x7f982c009310 comp rx=0 tx=0).stop 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 shutdown_connections 2023-12-14T18:34:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cd85f0 0x555f97cdaa00 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cc8bd0 0x555f97cc5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 --2- 172.21.15.31:0/3155349658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc5800 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f981ffff700 0 client.17085 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 >> 172.21.15.31:0/3155349658 conn(0x555f97cb3d20 msgr2=0x555f97cb9450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 shutdown_connections 2023-12-14T18:34:35.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 1 -- 172.21.15.31:0/3155349658 wait complete. 2023-12-14T18:34:35.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.346+0000 7f9853365b80 20 client.17085 trim_cache size 0 max 16384 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 Processor -- start 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 -- start start 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cb3d20 0x555f97cc8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97cc63a0 con 0x555f97cbdd10 2023-12-14T18:34:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97cbd240 con 0x555f97cbd400 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9853365b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97cc3d80 con 0x555f97cb3d20 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9845774700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9845774700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53418/0 (socket says 172.21.15.31:53418) 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9845774700 1 -- 172.21.15.31:0/1367942944 learned_addr learned my addr 172.21.15.31:0/1367942944 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f9844f73700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cb3d20 0x555f97cc8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.366+0000 7f983ffff700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9844f73700 1 -- 172.21.15.31:0/1367942944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 msgr2=0x555f97cc8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9844f73700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9844f73700 1 -- 172.21.15.31:0/1367942944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 msgr2=0x555f97cc2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9844f73700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9844f73700 1 -- 172.21.15.31:0/1367942944 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f97bd5f40 con 0x555f97cb3d20 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9845774700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f983ffff700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f9844f73700 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cb3d20 0x555f97cc8010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98340ca9b0 tx=0x7f9834085050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73483c5cc9c6c861 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f982b7fe700 1 -- 172.21.15.31:0/1367942944 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834016330 con 0x555f97cb3d20 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f982b7fe700 1 -- 172.21.15.31:0/1367942944 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983400cd40 con 0x555f97cb3d20 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.367+0000 7f982b7fe700 1 -- 172.21.15.31:0/1367942944 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834003710 con 0x555f97cb3d20 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 -- 172.21.15.31:0/1367942944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cb3d20 msgr2=0x555f97cc8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cb3d20 0x555f97cc8010 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f98340ca9b0 tx=0x7f9834085050 comp rx=0 tx=0).stop 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 -- 172.21.15.31:0/1367942944 shutdown_connections 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cbdd10 0x555f97cc2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cb3d20 0x555f97cc8010 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 --2- 172.21.15.31:0/1367942944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbd400 0x555f97cc8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 -- 172.21.15.31:0/1367942944 >> 172.21.15.31:0/1367942944 conn(0x555f97ccb280 msgr2=0x555f97cc57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 -- 172.21.15.31:0/1367942944 shutdown_connections 2023-12-14T18:34:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 -- 172.21.15.31:0/1367942944 wait complete. 2023-12-14T18:34:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 -- start start 2023-12-14T18:34:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.368+0000 7f9853365b80 1 Processor -- start 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 0x555f97cce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97ccec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbdd10 0x555f97d26500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97d26a40 con 0x555f97cb3d20 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97d26bb0 con 0x555f97cbdd10 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f97d26d70 con 0x555f97cbd400 2023-12-14T18:34:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9853365b80 10 client.0 _ll_register_callbacks cb 0x7f982c00bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 0x555f97cce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 0x555f97cce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53422/0 (socket says 172.21.15.31:53422) 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 -- 172.21.15.31:0/2181298541 learned_addr learned my addr 172.21.15.31:0/2181298541 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f983ffff700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97ccec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9845774700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbdd10 0x555f97d26500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 -- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 msgr2=0x555f97ccec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97ccec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 -- 172.21.15.31:0/2181298541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbdd10 msgr2=0x555f97d26500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.369+0000 7f9844f73700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbdd10 0x555f97d26500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f9844f73700 1 -- 172.21.15.31:0/2181298541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f97bd61f0 con 0x555f97cb3d20 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f9845774700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbdd10 0x555f97d26500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f983ffff700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97ccec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f9844f73700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 0x555f97cce700 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9834009230 tx=0x7f98340163a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4953773c6825becd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f981ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:35.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f981ffff700 1 -- 172.21.15.31:0/2181298541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340a2780 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f981ffff700 1 -- 172.21.15.31:0/2181298541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f983409e120 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f983800f040 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f981ffff700 1 -- 172.21.15.31:0/2181298541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9834003680 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f9853365b80 10 client.17091 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555f97a91430 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f981ffff700 1 -- 172.21.15.31:0/2181298541 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983800f040 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.370+0000 7f981ffff700 1 -- 172.21.15.31:0/2181298541 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9834099820 con 0x555f97cb3d20 2023-12-14T18:34:35.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f9853365b80 10 client.17091 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f9853365b80 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f9853365b80 20 client.17091 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f9853365b80 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cc8bd0 0x555f97d302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f9853365b80 4 client.17091 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555f97c08a80 con 0x555f97cc8bd0 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f983ffff700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cc8bd0 0x555f97d302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f983ffff700 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cc8bd0 0x555f97d302b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da89b6b5a454fa6 server_cookie=5463fd420f81583d in_seq=0 out_seq=0 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.371+0000 7f981ffff700 10 client.17091 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.372+0000 7f981ffff700 1 -- 172.21.15.31:0/2181298541 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f97c08a80 con 0x555f97cc8bd0 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.372+0000 7f981ffff700 10 client.17091 handle_command_reply: tid=0 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.372+0000 7f9853365b80 1 client.17091 shutdown 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 msgr2=0x555f97cce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 0x555f97cce700 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9834009230 tx=0x7f98340163a0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 shutdown_connections 2023-12-14T18:34:35.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555f97cc8bd0 0x555f97d302b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555f97cbdd10 0x555f97d26500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555f97cb3d20 0x555f97cce700 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 --2- 172.21.15.31:0/2181298541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f97cbd400 0x555f97ccec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 >> 172.21.15.31:0/2181298541 conn(0x555f97ccb280 msgr2=0x555f97cc3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f981ffff700 0 client.17091 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 shutdown_connections 2023-12-14T18:34:35.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 1 -- 172.21.15.31:0/2181298541 wait complete. 2023-12-14T18:34:35.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.373+0000 7f9853365b80 20 client.17091 trim_cache size 0 max 16384 2023-12-14T18:34:35.387 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:34:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:34:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "c879de96-269a-4c41-b5e2-43ade98b11af", 2023-12-14T18:34:35.387 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:34:35.387 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:34:35.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f98300a12c0 msgr2=0x7f98300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f98300a12c0 0x7f98300a3770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f982c009990 tx=0x7f982c019040 comp rx=0 tx=0).stop 2023-12-14T18:34:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 msgr2=0x7f9840196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 0x7f9840196360 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f983400eb30 tx=0x7f983400eef0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 shutdown_connections 2023-12-14T18:34:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f98300a12c0 0x7f98300a3770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98401043d0 0x7f984019ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98400ff390 0x7f9840196360 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 --2- 172.21.15.31:0/2255138786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98400ffd30 0x7f98401968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 >> 172.21.15.31:0/2255138786 conn(0x7f9840075fd0 msgr2=0x7f98400fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 shutdown_connections 2023-12-14T18:34:35.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.374+0000 7f98471d7700 1 -- 172.21.15.31:0/2255138786 wait complete. 2023-12-14T18:34:35.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:34:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.530+0000 7f13a29f2700 1 -- 172.21.15.31:0/4269618996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c1054f0 msgr2=0x7f139c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.530+0000 7f13a29f2700 1 --2- 172.21.15.31:0/4269618996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c1054f0 0x7f139c10d0c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f139400b0d0 tx=0x7f139400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 -- 172.21.15.31:0/4269618996 shutdown_connections 2023-12-14T18:34:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 --2- 172.21.15.31:0/4269618996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c1054f0 0x7f139c10d0c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 --2- 172.21.15.31:0/4269618996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c0690f0 0x7f139c0694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 --2- 172.21.15.31:0/4269618996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c069a90 0x7f139c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 -- 172.21.15.31:0/4269618996 >> 172.21.15.31:0/4269618996 conn(0x7f139c075fa0 msgr2=0x7f139c0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 -- 172.21.15.31:0/4269618996 shutdown_connections 2023-12-14T18:34:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 -- 172.21.15.31:0/4269618996 wait complete. 2023-12-14T18:34:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 Processor -- start 2023-12-14T18:34:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.531+0000 7f13a29f2700 1 -- start start 2023-12-14T18:34:35.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a29f2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c0690f0 0x7f139c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a29f2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 0x7f139c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a29f2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c1054f0 0x7f139c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a29f2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c19b490 con 0x7f139c069a90 2023-12-14T18:34:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a29f2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c19b600 con 0x7f139c1054f0 2023-12-14T18:34:35.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a29f2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139c19b7c0 con 0x7f139c0690f0 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 0x7f139c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 0x7f139c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53436/0 (socket says 172.21.15.31:53436) 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 -- 172.21.15.31:0/3888997847 learned_addr learned my addr 172.21.15.31:0/3888997847 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c1054f0 0x7f139c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139bfff700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c0690f0 0x7f139c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 -- 172.21.15.31:0/3888997847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c0690f0 msgr2=0x7f139c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c0690f0 0x7f139c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 -- 172.21.15.31:0/3888997847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c1054f0 msgr2=0x7f139c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c1054f0 0x7f139c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.532+0000 7f139b7fe700 1 -- 172.21.15.31:0/3888997847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1394009d00 con 0x7f139c069a90 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.533+0000 7f139b7fe700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 0x7f139c1969f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f138400e940 tx=0x7f138400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.533+0000 7f13997fa700 1 -- 172.21.15.31:0/3888997847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384009730 con 0x7f139c069a90 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.533+0000 7f13997fa700 1 -- 172.21.15.31:0/3888997847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1384010430 con 0x7f139c069a90 2023-12-14T18:34:35.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.533+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139c1102c0 con 0x7f139c069a90 2023-12-14T18:34:35.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.533+0000 7f13997fa700 1 -- 172.21.15.31:0/3888997847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400cea0 con 0x7f139c069a90 2023-12-14T18:34:35.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.533+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c110130 con 0x7f139c069a90 2023-12-14T18:34:35.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.535+0000 7f13997fa700 1 -- 172.21.15.31:0/3888997847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1384010810 con 0x7f139c069a90 2023-12-14T18:34:35.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.536+0000 7f13997fa700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f13880a1340 0x7f13880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.536+0000 7f139bfff700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f13880a1340 0x7f13880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.536+0000 7f13997fa700 1 -- 172.21.15.31:0/3888997847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1384014070 con 0x7f139c069a90 2023-12-14T18:34:35.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.536+0000 7f139bfff700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f13880a1340 0x7f13880a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f138c009c60 tx=0x7f138c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 Processor -- start 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 -- start start 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f163f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f164ab50 0x5574f164cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f15295e0 con 0x5574f163fc90 2023-12-14T18:34:35.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1598870 con 0x5574f164ab50 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1591d00 con 0x5574f163f380 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53446/0 (socket says 172.21.15.31:53446) 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 -- 172.21.15.31:0/2651315298 learned_addr learned my addr 172.21.15.31:0/2651315298 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139bfff700 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f163f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f164ab50 0x5574f164cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 -- 172.21.15.31:0/2651315298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 msgr2=0x5574f163f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f163f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 -- 172.21.15.31:0/2651315298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f164ab50 msgr2=0x5574f164cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f164ab50 0x5574f164cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f139b7fe700 1 -- 172.21.15.31:0/2651315298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f14133b0 con 0x5574f163fc90 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.537+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f164ab50 0x5574f164cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.538+0000 7f139b7fe700 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a610 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1384009230 tx=0x7f13840997a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4987ce2bc04ef0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.538+0000 7f13927fc700 1 -- 172.21.15.31:0/2651315298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840a2290 con 0x5574f163fc90 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.538+0000 7f13927fc700 1 -- 172.21.15.31:0/2651315298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13840981e0 con 0x5574f163fc90 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.538+0000 7f13927fc700 1 -- 172.21.15.31:0/2651315298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840a1e00 con 0x5574f163fc90 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2651315298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 msgr2=0x5574f164a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a610 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1384009230 tx=0x7f13840997a0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2651315298 shutdown_connections 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f164ab50 0x5574f164cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f163f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2651315298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a610 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2651315298 >> 172.21.15.31:0/2651315298 conn(0x5574f1635ca0 msgr2=0x5574f163b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2651315298 shutdown_connections 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2651315298 wait complete. 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 -- start start 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.539+0000 7f13aeb80b80 1 Processor -- start 2023-12-14T18:34:35.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f164a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163fc90 0x5574f16478f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f1647fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f152df90 con 0x5574f164ab50 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f152dc30 con 0x5574f163fc90 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f14156a0 con 0x5574f163f380 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13aeb80b80 10 client.0 _ll_register_callbacks cb 0x5574f1551910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f1647fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f1647fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53460/0 (socket says 172.21.15.31:53460) 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 -- 172.21.15.31:0/4120262356 learned_addr learned my addr 172.21.15.31:0/4120262356 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f139bfff700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f164a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f139b7fe700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163fc90 0x5574f16478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 -- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 msgr2=0x5574f164a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f164a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 -- 172.21.15.31:0/4120262356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163fc90 msgr2=0x5574f16478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163fc90 0x5574f16478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 -- 172.21.15.31:0/4120262356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139c110130 con 0x5574f164ab50 2023-12-14T18:34:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f139b7fe700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163fc90 0x5574f16478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f139bfff700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f164a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:34:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f1647fe0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1394008ef0 tx=0x7f1394008fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bca787ca47840 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.540+0000 7f137b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f137b7fe700 1 -- 172.21.15.31:0/4120262356 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139400e040 con 0x5574f164ab50 2023-12-14T18:34:35.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f137b7fe700 1 -- 172.21.15.31:0/4120262356 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1394006000 con 0x5574f164ab50 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574f16332b0 con 0x5574f164ab50 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f137b7fe700 1 -- 172.21.15.31:0/4120262356 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13940076e0 con 0x5574f164ab50 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 10 client.17103 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5574f1558170 con 0x5574f164ab50 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f137b7fe700 1 -- 172.21.15.31:0/4120262356 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5574f16332b0 con 0x5574f164ab50 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f137b7fe700 1 -- 172.21.15.31:0/4120262356 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f139400b100 con 0x5574f164ab50 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 10 client.17103 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:35.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 10 client.17103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 20 client.17103 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f165a600 0x5574f165ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 4 client.17103 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.541+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574f158ace0 con 0x5574f165a600 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.542+0000 7f139bfff700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f165a600 0x5574f165ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.542+0000 7f139bfff700 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f165a600 0x5574f165ca10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5422383bd918d server_cookie=824b757cc6f0fcae in_seq=0 out_seq=0 2023-12-14T18:34:35.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.542+0000 7f137b7fe700 10 client.17103 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f137b7fe700 1 -- 172.21.15.31:0/4120262356 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5574f158ace0 con 0x5574f165a600 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f137b7fe700 10 client.17103 handle_command_reply: tid=0 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 client.17103 shutdown 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 msgr2=0x5574f1647fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f1647fe0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1394008ef0 tx=0x7f1394008fd0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 shutdown_connections 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f165a600 0x5574f165ca10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f1647fe0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163f380 0x5574f164a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/4120262356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163fc90 0x5574f16478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f137b7fe700 0 client.17103 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 >> 172.21.15.31:0/4120262356 conn(0x5574f1635ca0 msgr2=0x5574f163b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 shutdown_connections 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 1 -- 172.21.15.31:0/4120262356 wait complete. 2023-12-14T18:34:35.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.543+0000 7f13aeb80b80 20 client.17103 trim_cache size 0 max 16384 2023-12-14T18:34:35.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 Processor -- start 2023-12-14T18:34:35.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 -- start start 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f164ab50 0x5574f1642bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f165d0b0 con 0x5574f163fc90 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f1643740 con 0x5574f163f380 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f16422c0 con 0x5574f164ab50 2023-12-14T18:34:35.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f139b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f139b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53472/0 (socket says 172.21.15.31:53472) 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.561+0000 7f139b7fe700 1 -- 172.21.15.31:0/1550028258 learned_addr learned my addr 172.21.15.31:0/1550028258 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f164ab50 0x5574f1642bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139bfff700 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139b7fe700 1 -- 172.21.15.31:0/1550028258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f164ab50 msgr2=0x5574f1642bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139b7fe700 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f164ab50 0x5574f1642bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139b7fe700 1 -- 172.21.15.31:0/1550028258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 msgr2=0x5574f164a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139b7fe700 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139b7fe700 1 -- 172.21.15.31:0/1550028258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574f1557ec0 con 0x5574f163fc90 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139bfff700 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f139b7fe700 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a5d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f13840cd130 tx=0x7f1384096180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc236998014922b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f1392ffd700 1 -- 172.21.15.31:0/1550028258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840a2290 con 0x5574f163fc90 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f1392ffd700 1 -- 172.21.15.31:0/1550028258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f138409a0a0 con 0x5574f163fc90 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.562+0000 7f1392ffd700 1 -- 172.21.15.31:0/1550028258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840a1e00 con 0x5574f163fc90 2023-12-14T18:34:35.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 -- 172.21.15.31:0/1550028258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 msgr2=0x5574f164a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a5d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f13840cd130 tx=0x7f1384096180 comp rx=0 tx=0).stop 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 -- 172.21.15.31:0/1550028258 shutdown_connections 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f164ab50 0x5574f1642bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/1550028258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f163fc90 0x5574f164a5d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 -- 172.21.15.31:0/1550028258 >> 172.21.15.31:0/1550028258 conn(0x5574f1635ca0 msgr2=0x5574f1655e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 -- 172.21.15.31:0/1550028258 shutdown_connections 2023-12-14T18:34:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 -- 172.21.15.31:0/1550028258 wait complete. 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 -- start start 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.563+0000 7f13aeb80b80 1 Processor -- start 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163fc90 0x5574f164f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f164fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f16ad600 con 0x5574f164ab50 2023-12-14T18:34:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f16ad770 con 0x5574f163f380 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574f16ad930 con 0x5574f163fc90 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 10 client.0 _ll_register_callbacks cb 0x7f1384005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f164fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f164fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53486/0 (socket says 172.21.15.31:53486) 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 -- 172.21.15.31:0/2709310261 learned_addr learned my addr 172.21.15.31:0/2709310261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f139b7fe700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163fc90 0x5574f164f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f139bfff700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 -- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163fc90 msgr2=0x5574f164f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163fc90 0x5574f164f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 -- 172.21.15.31:0/2709310261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 msgr2=0x5574f164ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 -- 172.21.15.31:0/2709310261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574f14133b0 con 0x5574f164ab50 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f139bfff700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13a0f8f700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f164fa10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1394008ef0 tx=0x7f139400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e737bb5c5b0fa08 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f137b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:34:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f137b7fe700 1 -- 172.21.15.31:0/2709310261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139400e040 con 0x5574f164ab50 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f137b7fe700 1 -- 172.21.15.31:0/2709310261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1394006000 con 0x5574f164ab50 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.564+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13840981e0 con 0x5574f164ab50 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f137b7fe700 1 -- 172.21.15.31:0/2709310261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f139400bb50 con 0x5574f164ab50 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 10 client.17109 fetch_fsmap learned FSMap version 28 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5574f153e9a0 con 0x5574f164ab50 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f137b7fe700 1 -- 172.21.15.31:0/2709310261 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13840981e0 con 0x5574f164ab50 2023-12-14T18:34:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f137b7fe700 1 -- 172.21.15.31:0/2709310261 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f139401d590 con 0x5574f164ab50 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 10 client.17109 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 20 client.17109 populate_metadata read hostname 'smithi031' 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f16aed80 0x5574f16b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 4 client.17109 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5574f158aa00 con 0x5574f16aed80 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.565+0000 7f139bfff700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f16aed80 0x5574f16b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:34:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f139bfff700 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f16aed80 0x5574f16b1190 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb532e2136aa4e1e server_cookie=5dc2974dadae0d6 in_seq=0 out_seq=0 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f137b7fe700 10 client.17109 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f137b7fe700 1 -- 172.21.15.31:0/2709310261 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5574f158aa00 con 0x5574f16aed80 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f137b7fe700 10 client.17109 handle_command_reply: tid=0 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 client.17109 shutdown 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 msgr2=0x5574f164fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f164fa10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1394008ef0 tx=0x7f139400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 shutdown_connections 2023-12-14T18:34:35.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5574f16aed80 0x5574f16b1190 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5574f164ab50 0x5574f164fa10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5574f163f380 0x5574f164ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 --2- 172.21.15.31:0/2709310261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574f163fc90 0x5574f164f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f137b7fe700 0 client.17109 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 >> 172.21.15.31:0/2709310261 conn(0x5574f1635ca0 msgr2=0x5574f1642bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 shutdown_connections 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 1 -- 172.21.15.31:0/2709310261 wait complete. 2023-12-14T18:34:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.566+0000 7f13aeb80b80 20 client.17109 trim_cache size 0 max 16384 2023-12-14T18:34:35.581 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:34:35.581 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:34:35.581 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:34:35.581 INFO:teuthology.orchestra.run.smithi031.stdout: "c879de96-269a-4c41-b5e2-43ade98b11af": { 2023-12-14T18:34:35.581 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:34:35.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "c879de96-269a-4c41-b5e2-43ade98b11af", 2023-12-14T18:34:35.582 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:34:35.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:34:35.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:34:35.582 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:34:35.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f13880a1340 msgr2=0x7f13880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f13880a1340 0x7f13880a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f138c009c60 tx=0x7f138c009260 comp rx=0 tx=0).stop 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 msgr2=0x7f139c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 0x7f139c1969f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f138400e940 tx=0x7f138400ec50 comp rx=0 tx=0).stop 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 shutdown_connections 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f139c1054f0 0x7f139c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f13880a1340 0x7f13880a37f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f139c0690f0 0x7f139c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 --2- 172.21.15.31:0/3888997847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f139c069a90 0x7f139c1969f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:34:35.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 >> 172.21.15.31:0/3888997847 conn(0x7f139c075fa0 msgr2=0x7f139c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:34:35.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 shutdown_connections 2023-12-14T18:34:35.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:34:35.567+0000 7f13a29f2700 1 -- 172.21.15.31:0/3888997847 wait complete. 2023-12-14T18:34:35.585 INFO:tasks.cephfs.filesystem:scrub status for tag:c879de96-269a-4c41-b5e2-43ade98b11af - {'path': '/', 'tag': 'c879de96-269a-4c41-b5e2-43ade98b11af', 'options': 'recursive,force'} 2023-12-14T18:34:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:35 smithi176 ceph-mon[43506]: pgmap v208: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-12-14T18:34:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:35 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:34:35.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:35 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:35 smithi204 ceph-mon[43866]: pgmap v208: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-12-14T18:34:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:35 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:34:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:35 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:35.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:35 smithi031 ceph-mon[42085]: pgmap v208: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-12-14T18:34:35.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:35 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:34:35.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:35 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:36.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:36 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:34:36.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:36 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:36.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:36 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:34:36.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:36 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:36 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:34:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:36 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:36 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:34:36.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:36 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:36 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:34:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:36 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:36 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:34:36.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:36 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:34:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:37 smithi204 ceph-mon[43866]: pgmap v209: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2023-12-14T18:34:37.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:37 smithi031 ceph-mon[42085]: pgmap v209: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2023-12-14T18:34:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:37 smithi176 ceph-mon[43506]: pgmap v209: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.6 KiB/s wr, 5 op/s 2023-12-14T18:34:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:39 smithi204 ceph-mon[43866]: pgmap v210: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-14T18:34:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:39 smithi031 ceph-mon[42085]: pgmap v210: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-14T18:34:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:39 smithi176 ceph-mon[43506]: pgmap v210: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-14T18:34:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:41 smithi204 ceph-mon[43866]: pgmap v211: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:41 smithi031 ceph-mon[42085]: pgmap v211: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:41 smithi176 ceph-mon[43506]: pgmap v211: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:43 smithi204 ceph-mon[43866]: pgmap v212: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:43.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:43 smithi031 ceph-mon[42085]: pgmap v212: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:43 smithi176 ceph-mon[43506]: pgmap v212: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:44.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:44 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:34:44.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:44 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:34:44.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:44 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:34:44.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:44 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:34:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:44 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:34:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:44 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:34:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:45 smithi204 ceph-mon[43866]: pgmap v213: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-12-14T18:34:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:45 smithi031 ceph-mon[42085]: pgmap v213: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-12-14T18:34:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:45 smithi176 ceph-mon[43506]: pgmap v213: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-12-14T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:47 smithi204 ceph-mon[43866]: pgmap v214: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2023-12-14T18:34:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:47 smithi031 ceph-mon[42085]: pgmap v214: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2023-12-14T18:34:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:47 smithi176 ceph-mon[43506]: pgmap v214: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.8 KiB/s wr, 6 op/s 2023-12-14T18:34:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:49 smithi204 ceph-mon[43866]: pgmap v215: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-12-14T18:34:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:49 smithi031 ceph-mon[42085]: pgmap v215: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-12-14T18:34:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:49 smithi176 ceph-mon[43506]: pgmap v215: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2023-12-14T18:34:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:51 smithi204 ceph-mon[43866]: pgmap v216: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:51 smithi031 ceph-mon[42085]: pgmap v216: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:51.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:51 smithi176 ceph-mon[43506]: pgmap v216: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:53 smithi204 ceph-mon[43866]: pgmap v217: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:53.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:53 smithi031 ceph-mon[42085]: pgmap v217: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:53 smithi176 ceph-mon[43506]: pgmap v217: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:55.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:55 smithi204 ceph-mon[43866]: pgmap v218: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-12-14T18:34:55.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:55 smithi031 ceph-mon[42085]: pgmap v218: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-12-14T18:34:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:55 smithi176 ceph-mon[43506]: pgmap v218: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-12-14T18:34:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:57 smithi204 ceph-mon[43866]: pgmap v219: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:57.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:57 smithi031 ceph-mon[42085]: pgmap v219: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:57 smithi176 ceph-mon[43506]: pgmap v219: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:34:59 smithi204 ceph-mon[43866]: pgmap v220: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:34:59 smithi031 ceph-mon[42085]: pgmap v220: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:34:59 smithi176 ceph-mon[43506]: pgmap v220: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:01 smithi204 ceph-mon[43866]: pgmap v221: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:01.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:01 smithi031 ceph-mon[42085]: pgmap v221: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:01 smithi176 ceph-mon[43506]: pgmap v221: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:03 smithi204 ceph-mon[43866]: pgmap v222: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:03 smithi031 ceph-mon[42085]: pgmap v222: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:03 smithi176 ceph-mon[43506]: pgmap v222: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:05.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:35:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 -- 172.21.15.31:0/4256958600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c100e70 msgr2=0x7f7b8c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 --2- 172.21.15.31:0/4256958600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c100e70 0x7f7b8c104e30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7b880099c0 tx=0x7f7b88009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 -- 172.21.15.31:0/4256958600 shutdown_connections 2023-12-14T18:35:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 --2- 172.21.15.31:0/4256958600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b8c105510 0x7f7b8c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 --2- 172.21.15.31:0/4256958600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 --2- 172.21.15.31:0/4256958600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c100e70 0x7f7b8c104e30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 -- 172.21.15.31:0/4256958600 >> 172.21.15.31:0/4256958600 conn(0x7f7b8c078210 msgr2=0x7f7b8c078610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 -- 172.21.15.31:0/4256958600 shutdown_connections 2023-12-14T18:35:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.735+0000 7f7b942da700 1 -- 172.21.15.31:0/4256958600 wait complete. 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 Processor -- start 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 -- start start 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c100e70 0x7f7b8c196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b8c105510 0x7f7b8c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19b3d0 con 0x7f7b8c1004d0 2023-12-14T18:35:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19b540 con 0x7f7b8c105510 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b942da700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c19b700 con 0x7f7b8c100e70 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b92076700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b92076700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60580/0 (socket says 172.21.15.31:60580) 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b92076700 1 -- 172.21.15.31:0/1076955084 learned_addr learned my addr 172.21.15.31:0/1076955084 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b91875700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c100e70 0x7f7b8c196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.736+0000 7f7b92877700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b8c105510 0x7f7b8c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b92076700 1 -- 172.21.15.31:0/1076955084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c100e70 msgr2=0x7f7b8c196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b92076700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c100e70 0x7f7b8c196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b92076700 1 -- 172.21.15.31:0/1076955084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b8c105510 msgr2=0x7f7b8c19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b92076700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b8c105510 0x7f7b8c19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b92076700 1 -- 172.21.15.31:0/1076955084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880095f0 con 0x7f7b8c1004d0 2023-12-14T18:35:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b92076700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1963f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9a0 tx=0x7f7b8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b7f7fe700 1 -- 172.21.15.31:0/1076955084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80009790 con 0x7f7b8c1004d0 2023-12-14T18:35:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c1102e0 con 0x7f7b8c1004d0 2023-12-14T18:35:05.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b7f7fe700 1 -- 172.21.15.31:0/1076955084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b80010430 con 0x7f7b8c1004d0 2023-12-14T18:35:05.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b7f7fe700 1 -- 172.21.15.31:0/1076955084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80018680 con 0x7f7b8c1004d0 2023-12-14T18:35:05.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.737+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c110150 con 0x7f7b8c1004d0 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.739+0000 7f7b7f7fe700 1 -- 172.21.15.31:0/1076955084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7b800105a0 con 0x7f7b8c1004d0 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.740+0000 7f7b7f7fe700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7b780a12c0 0x7f7b780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.740+0000 7f7b7f7fe700 1 -- 172.21.15.31:0/1076955084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7b80014070 con 0x7f7b8c1004d0 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.740+0000 7f7b91875700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7b780a12c0 0x7f7b780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7b91875700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7b780a12c0 0x7f7b780a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b580 tx=0x7f7b88000bc0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 Processor -- start 2023-12-14T18:35:05.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 -- start start 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e7c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e8ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68d665e0 con 0x558f68e7cc90 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68dd5870 con 0x558f68e8bf60 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68dced00 con 0x558f68e7c380 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7b91875700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e8ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7b91875700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e8ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60596/0 (socket says 172.21.15.31:60596) 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7b91875700 1 -- 172.21.15.31:0/196477269 learned_addr learned my addr 172.21.15.31:0/196477269 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:05.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7b92076700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e7c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.741+0000 7f7b92877700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b91875700 1 -- 172.21.15.31:0/196477269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 msgr2=0x558f68e7c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b91875700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e7c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b91875700 1 -- 172.21.15.31:0/196477269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 msgr2=0x558f68e8e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b91875700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e8e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b91875700 1 -- 172.21.15.31:0/196477269 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f68c503b0 con 0x558f68e7cc90 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b92877700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b92076700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e7c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:35:05.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b91875700 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e8ba20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7b88015000 tx=0x7f7b8801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53184151ee713389 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b7cdac700 1 -- 172.21.15.31:0/196477269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800ba10 con 0x558f68e7cc90 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b7cdac700 1 -- 172.21.15.31:0/196477269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8801e040 con 0x558f68e7cc90 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.742+0000 7f7b7cdac700 1 -- 172.21.15.31:0/196477269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800f6c0 con 0x558f68e7cc90 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 -- 172.21.15.31:0/196477269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 msgr2=0x558f68e8ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e8ba20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7b88015000 tx=0x7f7b8801f000 comp rx=0 tx=0).stop 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 -- 172.21.15.31:0/196477269 shutdown_connections 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e7c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/196477269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e8ba20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 -- 172.21.15.31:0/196477269 >> 172.21.15.31:0/196477269 conn(0x558f68e72ca0 msgr2=0x558f68e787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 -- 172.21.15.31:0/196477269 shutdown_connections 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 -- 172.21.15.31:0/196477269 wait complete. 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.743+0000 7f7ba0468b80 1 -- start start 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 Processor -- start 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e8b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e89100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68d6af90 con 0x558f68e7cc90 2023-12-14T18:35:05.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68d6ac30 con 0x558f68e8bf60 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68c526a0 con 0x558f68e7c380 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7ba0468b80 10 client.0 _ll_register_callbacks cb 0x558f68d8e910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60608/0 (socket says 172.21.15.31:60608) 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 -- 172.21.15.31:0/1268501404 learned_addr learned my addr 172.21.15.31:0/1268501404 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b92076700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e8b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b92877700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e89100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 -- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 msgr2=0x558f68e8b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e8b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 -- 172.21.15.31:0/1268501404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 msgr2=0x558f68e89100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e89100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.744+0000 7f7b91875700 1 -- 172.21.15.31:0/1268501404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c10d920 con 0x558f68e7cc90 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b91875700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a010 tx=0x7f7b8800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbe5313b3879eeb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1268501404 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88034070 con 0x558f68e7cc90 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1268501404 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7b88004450 con 0x558f68e7cc90 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f68e702b0 con 0x558f68e7cc90 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1268501404 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b880047a0 con 0x558f68e7cc90 2023-12-14T18:35:05.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7ba0468b80 10 client.17121 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f68d95170 con 0x558f68e7cc90 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1268501404 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f68e702b0 con 0x558f68e7cc90 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.745+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1268501404 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7b880305a0 con 0x558f68e7cc90 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7ba0468b80 10 client.17121 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7ba0468b80 10 client.17121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7ba0468b80 20 client.17121 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68e9b450 0x558f68e9d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7ba0468b80 4 client.17121 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f68dc7ce0 con 0x558f68e9b450 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7b92076700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68e9b450 0x558f68e9d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7b75ffb700 10 client.17121 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.746+0000 7f7b92076700 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68e9b450 0x558f68e9d830 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694dbbae3783fe8e server_cookie=aa8a47ec8ec98bfc in_seq=0 out_seq=0 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1268501404 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f68dc7ce0 con 0x558f68e9b450 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7b75ffb700 10 client.17121 handle_command_reply: tid=0 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 client.17121 shutdown 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 msgr2=0x558f68e88b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a010 tx=0x7f7b8800f710 comp rx=0 tx=0).stop 2023-12-14T18:35:05.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 shutdown_connections 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68e9b450 0x558f68e9d830 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68e89100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7c380 0x558f68e8b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7b75ffb700 0 client.17121 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1268501404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 >> 172.21.15.31:0/1268501404 conn(0x558f68e72ca0 msgr2=0x558f68e783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 shutdown_connections 2023-12-14T18:35:05.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1268501404 wait complete. 2023-12-14T18:35:05.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.747+0000 7f7ba0468b80 20 client.17121 trim_cache size 0 max 16384 2023-12-14T18:35:05.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.765+0000 7f7ba0468b80 1 Processor -- start 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.765+0000 7f7ba0468b80 1 -- start start 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.765+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e7c380 0x558f68e880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e8bf60 0x558f68e89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68e72ab0 con 0x558f68e7cc90 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68e8a160 con 0x558f68e7c380 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68e78d50 con 0x558f68e8bf60 2023-12-14T18:35:05.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60614/0 (socket says 172.21.15.31:60614) 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 -- 172.21.15.31:0/1637699959 learned_addr learned my addr 172.21.15.31:0/1637699959 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b92076700 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e7c380 0x558f68e880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b92877700 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e8bf60 0x558f68e89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 -- 172.21.15.31:0/1637699959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e8bf60 msgr2=0x558f68e89080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e8bf60 0x558f68e89080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 -- 172.21.15.31:0/1637699959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e7c380 msgr2=0x558f68e880f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e7c380 0x558f68e880f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 -- 172.21.15.31:0/1637699959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f68d94ec0 con 0x558f68e7cc90 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b92076700 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e7c380 0x558f68e880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b91875700 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c010 tx=0x7f7b88009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e368939e3fe2279 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b7d5ad700 1 -- 172.21.15.31:0/1637699959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88034070 con 0x558f68e7cc90 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b7d5ad700 1 -- 172.21.15.31:0/1637699959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b88030ba0 con 0x558f68e7cc90 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.766+0000 7f7b7d5ad700 1 -- 172.21.15.31:0/1637699959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88030480 con 0x558f68e7cc90 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1637699959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 msgr2=0x558f68e88b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c010 tx=0x7f7b88009310 comp rx=0 tx=0).stop 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1637699959 shutdown_connections 2023-12-14T18:35:05.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e8bf60 0x558f68e89080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e7c380 0x558f68e880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1637699959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7cc90 0x558f68e88b40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1637699959 >> 172.21.15.31:0/1637699959 conn(0x558f68e72ca0 msgr2=0x558f68e8b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1637699959 shutdown_connections 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1637699959 wait complete. 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 -- start start 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.767+0000 7f7ba0468b80 1 Processor -- start 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 0x558f68e90470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7cc90 0x558f68e909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68eee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68e912c0 con 0x558f68e7c380 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68eee7e0 con 0x558f68e8bf60 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f68eee920 con 0x558f68e7cc90 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7ba0468b80 10 client.0 _ll_register_callbacks cb 0x7f7b880039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 0x558f68e90470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 0x558f68e90470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60624/0 (socket says 172.21.15.31:60624) 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 -- 172.21.15.31:0/1862575203 learned_addr learned my addr 172.21.15.31:0/1862575203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b91875700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7cc90 0x558f68e909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92877700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68eee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 -- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7cc90 msgr2=0x558f68e909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7cc90 0x558f68e909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 -- 172.21.15.31:0/1862575203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 msgr2=0x558f68eee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68eee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 -- 172.21.15.31:0/1862575203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f68d95170 con 0x558f68e7c380 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b91875700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7cc90 0x558f68e909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92877700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68eee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.768+0000 7f7b92076700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 0x558f68e90470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005bf0 tx=0x7f7b80096420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e189502e75c4e73 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7b75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1862575203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8009a410 con 0x558f68e7c380 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1862575203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7b80098410 con 0x558f68e7c380 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b8801e040 con 0x558f68e7c380 2023-12-14T18:35:05.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1862575203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b800994a0 con 0x558f68e7c380 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 10 client.17127 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558f68d7b9a0 con 0x558f68e7c380 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1862575203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8801e040 con 0x558f68e7c380 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1862575203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7b80098f10 con 0x558f68e7c380 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 10 client.17127 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 20 client.17127 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68ef00a0 0x558f68ef24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 4 client.17127 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:35:05.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.769+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f68dc7a00 con 0x558f68ef00a0 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7b91875700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68ef00a0 0x558f68ef24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7b91875700 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68ef00a0 0x558f68ef24b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a1f4a9bf057b20 server_cookie=95547ecae750ec66 in_seq=0 out_seq=0 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7b75ffb700 10 client.17127 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7b75ffb700 1 -- 172.21.15.31:0/1862575203 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558f68dc7a00 con 0x558f68ef00a0 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7b75ffb700 10 client.17127 handle_command_reply: tid=0 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 client.17127 shutdown 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 msgr2=0x558f68e90470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 0x558f68e90470 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7b80005bf0 tx=0x7f7b80096420 comp rx=0 tx=0).stop 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 shutdown_connections 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558f68ef00a0 0x558f68ef24b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558f68e8bf60 0x558f68eee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558f68e7c380 0x558f68e90470 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7b75ffb700 0 client.17127 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 --2- 172.21.15.31:0/1862575203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558f68e7cc90 0x558f68e909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.770+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 >> 172.21.15.31:0/1862575203 conn(0x558f68e72ca0 msgr2=0x558f68e83e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 shutdown_connections 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7ba0468b80 1 -- 172.21.15.31:0/1862575203 wait complete. 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7ba0468b80 20 client.17127 trim_cache size 0 max 16384 2023-12-14T18:35:05.781 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:35:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:35:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:35:05.782 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7b780a12c0 msgr2=0x7f7b780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7b780a12c0 0x7f7b780a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800b580 tx=0x7f7b88000bc0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 msgr2=0x7f7b8c1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1963f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9a0 tx=0x7f7b8000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 shutdown_connections 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7b8c105510 0x7f7b8c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b8c1004d0 0x7f7b8c1963f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7b780a12c0 0x7f7b780a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 --2- 172.21.15.31:0/1076955084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b8c100e70 0x7f7b8c196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 >> 172.21.15.31:0/1076955084 conn(0x7f7b8c078210 msgr2=0x7f7b8c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 shutdown_connections 2023-12-14T18:35:05.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.771+0000 7f7b942da700 1 -- 172.21.15.31:0/1076955084 wait complete. 2023-12-14T18:35:05.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:35:05.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:35:05.813 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:05 smithi031 ceph-mon[42085]: pgmap v223: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:35:05.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:05 smithi204 ceph-mon[43866]: pgmap v223: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 -- 172.21.15.31:0/1739275935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc10a880 msgr2=0x7f7abc10cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 --2- 172.21.15.31:0/1739275935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc10a880 0x7f7abc10cc80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400b0a0 tx=0x7f7ab400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 -- 172.21.15.31:0/1739275935 shutdown_connections 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 --2- 172.21.15.31:0/1739275935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc10a880 0x7f7abc10cc80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 --2- 172.21.15.31:0/1739275935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7abc101360 0x7f7abc101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 --2- 172.21.15.31:0/1739275935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7abc101c70 0x7f7abc10a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.922+0000 7f7ac2040700 1 -- 172.21.15.31:0/1739275935 >> 172.21.15.31:0/1739275935 conn(0x7f7abc075fd0 msgr2=0x7f7abc0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.923+0000 7f7ac2040700 1 -- 172.21.15.31:0/1739275935 shutdown_connections 2023-12-14T18:35:05.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.923+0000 7f7ac2040700 1 -- 172.21.15.31:0/1739275935 wait complete. 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.923+0000 7f7ac2040700 1 Processor -- start 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.923+0000 7f7ac2040700 1 -- start start 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ac2040700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 0x7f7abc0ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ac2040700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7abc101c70 0x7f7abc0ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ac2040700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7abc10a880 0x7f7abc1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ac2040700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0fff00 con 0x7f7abc101360 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ac2040700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1a4bc0 con 0x7f7abc10a880 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ac2040700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc1a4d00 con 0x7f7abc101c70 2023-12-14T18:35:05.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 0x7f7abc0ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 0x7f7abc0ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60642/0 (socket says 172.21.15.31:60642) 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 -- 172.21.15.31:0/3737764070 learned_addr learned my addr 172.21.15.31:0/3737764070 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abbfff700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7abc10a880 0x7f7abc1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abaffd700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7abc101c70 0x7f7abc0ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 -- 172.21.15.31:0/3737764070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7abc101c70 msgr2=0x7f7abc0ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7abc101c70 0x7f7abc0ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 -- 172.21.15.31:0/3737764070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7abc10a880 msgr2=0x7f7abc1a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7abc10a880 0x7f7abc1a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:05.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 -- 172.21.15.31:0/3737764070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4009d00 con 0x7f7abc101360 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7abb7fe700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 0x7f7abc0ff190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00e9a0 tx=0x7f7aac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.924+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac009790 con 0x7f7abc101360 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.925+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc1102b0 con 0x7f7abc101360 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.925+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aac010430 con 0x7f7abc101360 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.925+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac018680 con 0x7f7abc101360 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.925+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc110120 con 0x7f7abc101360 2023-12-14T18:35:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.926+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc110440 con 0x7f7abc101360 2023-12-14T18:35:05.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.927+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7aac0105a0 con 0x7f7abc101360 2023-12-14T18:35:05.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.927+0000 7f7ab8ff9700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7aa40a12c0 0x7f7aa40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:05.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.928+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7aac014070 con 0x7f7abc101360 2023-12-14T18:35:05.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.928+0000 7f7abaffd700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7aa40a12c0 0x7f7aa40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:05.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.928+0000 7f7abaffd700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7aa40a12c0 0x7f7aa40a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7abc100980 tx=0x7f7ab0009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:05.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:05.931+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7aac096480 con 0x7f7abc101360 2023-12-14T18:35:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:05 smithi176 ceph-mon[43506]: pgmap v223: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:35:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.187+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7abc1a4e40 con 0x7f7abc101360 2023-12-14T18:35:06.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.188+0000 7f7ab8ff9700 1 -- 172.21.15.31:0/3737764070 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10822 (secure 0 0 0) 0x7f7aac026020 con 0x7f7abc101360 2023-12-14T18:35:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:35:06.190 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:31:40.963020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7aa40a12c0 msgr2=0x7f7aa40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7aa40a12c0 0x7f7aa40a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7abc100980 tx=0x7f7ab0009260 comp rx=0 tx=0).stop 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 msgr2=0x7f7abc0ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 0x7f7abc0ff190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00e9a0 tx=0x7f7aac00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 shutdown_connections 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7abc10a880 0x7f7abc1a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7abc101360 0x7f7abc0ff190 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7aa40a12c0 0x7f7aa40a3770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 --2- 172.21.15.31:0/3737764070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7abc101c70 0x7f7abc0ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 >> 172.21.15.31:0/3737764070 conn(0x7f7abc075fd0 msgr2=0x7f7abc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 shutdown_connections 2023-12-14T18:35:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.191+0000 7f7ac2040700 1 -- 172.21.15.31:0/3737764070 wait complete. 2023-12-14T18:35:06.193 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-12-14T18:35:06.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:35:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 -- 172.21.15.31:0/3755367860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 msgr2=0x7f5b880ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 --2- 172.21.15.31:0/3755367860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b880ff760 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5b840099c0 tx=0x7f5b84009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 -- 172.21.15.31:0/3755367860 shutdown_connections 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 --2- 172.21.15.31:0/3755367860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b881043d0 0x7f5b8810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 --2- 172.21.15.31:0/3755367860 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b880ff760 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 --2- 172.21.15.31:0/3755367860 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b880ffd30 0x7f5b88103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 -- 172.21.15.31:0/3755367860 >> 172.21.15.31:0/3755367860 conn(0x7f5b88075fd0 msgr2=0x7f5b880763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 -- 172.21.15.31:0/3755367860 shutdown_connections 2023-12-14T18:35:06.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.388+0000 7f5b90705700 1 -- 172.21.15.31:0/3755367860 wait complete. 2023-12-14T18:35:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 Processor -- start 2023-12-14T18:35:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 -- start start 2023-12-14T18:35:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b881964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b880ffd30 0x7f5b88196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b881043d0 0x7f5b8819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8819b4a0 con 0x7f5b880ff390 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8819b610 con 0x7f5b881043d0 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b90705700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8819b7d0 con 0x7f5b880ffd30 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b881964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b881964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60652/0 (socket says 172.21.15.31:60652) 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/3230918989 learned_addr learned my addr 172.21.15.31:0/3230918989 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b880ffd30 0x7f5b88196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b881043d0 0x7f5b8819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/3230918989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b880ffd30 msgr2=0x7f5b88196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b880ffd30 0x7f5b88196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/3230918989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b881043d0 msgr2=0x7f5b8819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b881043d0 0x7f5b8819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.389+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/3230918989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840095f0 con 0x7f5b880ff390 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.390+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b881964c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400b580 tx=0x7f5b8400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.390+0000 7f5b7f7fe700 1 -- 172.21.15.31:0/3230918989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8401d070 con 0x7f5b880ff390 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.390+0000 7f5b7f7fe700 1 -- 172.21.15.31:0/3230918989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b84005050 con 0x7f5b880ff390 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.390+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8810f1e0 con 0x7f5b880ff390 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.390+0000 7f5b7f7fe700 1 -- 172.21.15.31:0/3230918989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b84004a40 con 0x7f5b880ff390 2023-12-14T18:35:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.390+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8810f050 con 0x7f5b880ff390 2023-12-14T18:35:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.392+0000 7f5b7f7fe700 1 -- 172.21.15.31:0/3230918989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5b840051c0 con 0x7f5b880ff390 2023-12-14T18:35:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.393+0000 7f5b7f7fe700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5b740a1340 0x7f5b740a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.393+0000 7f5b7f7fe700 1 -- 172.21.15.31:0/3230918989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f5b840cdda0 con 0x7f5b880ff390 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.393+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5b740a1340 0x7f5b740a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.393+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5b740a1340 0x7f5b740a37f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b88197ad0 tx=0x7f5b78009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 Processor -- start 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 -- start start 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa741750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa74c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa62b5e0 con 0x556dfa74cb50 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa69a870 con 0x556dfa741380 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa693d00 con 0x556dfa741c90 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b8eca2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b8eca2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60668/0 (socket says 172.21.15.31:60668) 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.394+0000 7f5b8eca2700 1 -- 172.21.15.31:0/3982168642 learned_addr learned my addr 172.21.15.31:0/3982168642 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa74c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa741750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8eca2700 1 -- 172.21.15.31:0/3982168642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 msgr2=0x556dfa74c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa74c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8eca2700 1 -- 172.21.15.31:0/3982168642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 msgr2=0x556dfa741750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa741750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8eca2700 1 -- 172.21.15.31:0/3982168642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dfa5153b0 con 0x556dfa74cb50 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa741750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74ef30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5b80011680 tx=0x7f5b80009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52af9bc44379efb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b7cff9700 1 -- 172.21.15.31:0/3982168642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000e040 con 0x556dfa74cb50 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b7cff9700 1 -- 172.21.15.31:0/3982168642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b8000f040 con 0x556dfa74cb50 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.395+0000 7f5b7cff9700 1 -- 172.21.15.31:0/3982168642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80004860 con 0x556dfa74cb50 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 -- 172.21.15.31:0/3982168642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 msgr2=0x556dfa74ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74ef30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5b80011680 tx=0x7f5b80009920 comp rx=0 tx=0).stop 2023-12-14T18:35:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 -- 172.21.15.31:0/3982168642 shutdown_connections 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74ef30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa741750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/3982168642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa74c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 -- 172.21.15.31:0/3982168642 >> 172.21.15.31:0/3982168642 conn(0x556dfa737ca0 msgr2=0x556dfa73d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 -- 172.21.15.31:0/3982168642 shutdown_connections 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 -- 172.21.15.31:0/3982168642 wait complete. 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.396+0000 7f5b9c893b80 1 -- start start 2023-12-14T18:35:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 Processor -- start 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa74c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741c90 0x556dfa749780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa74cb50 0x556dfa749d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa62ff90 con 0x556dfa741380 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa62fc30 con 0x556dfa741c90 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa5176a0 con 0x556dfa74cb50 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b9c893b80 10 client.0 _ll_register_callbacks cb 0x556dfa653910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa74c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa74c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60676/0 (socket says 172.21.15.31:60676) 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/1039662704 learned_addr learned my addr 172.21.15.31:0/1039662704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa74cb50 0x556dfa749d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741c90 0x556dfa749780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa74cb50 msgr2=0x556dfa749d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa74cb50 0x556dfa749d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/1039662704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741c90 msgr2=0x556dfa749780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741c90 0x556dfa749780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.397+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/1039662704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b80011330 con 0x556dfa741380 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa74c4d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b8409ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4eb81ac0f86f1d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b71ffb700 1 -- 172.21.15.31:0/1039662704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8409af70 con 0x556dfa741380 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b71ffb700 1 -- 172.21.15.31:0/1039662704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f5b8409a000 con 0x556dfa741380 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556dfa7352b0 con 0x556dfa741380 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b71ffb700 1 -- 172.21.15.31:0/1039662704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b840056b0 con 0x556dfa741380 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 10 client.17145 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556dfa65a170 con 0x556dfa741380 2023-12-14T18:35:06.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b71ffb700 1 -- 172.21.15.31:0/1039662704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556dfa7352b0 con 0x556dfa741380 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b71ffb700 1 -- 172.21.15.31:0/1039662704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5b840c6020 con 0x556dfa741380 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 10 client.17145 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 10 client.17145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 20 client.17145 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa75c100 0x556dfa75e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 4 client.17145 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.398+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556dfa68cce0 con 0x556dfa75c100 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.399+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa75c100 0x556dfa75e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.399+0000 7f5b71ffb700 10 client.17145 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.399+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa75c100 0x556dfa75e4e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d765d77b60bdcf9 server_cookie=3b856cbcfb36e5fc in_seq=0 out_seq=0 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b71ffb700 1 -- 172.21.15.31:0/1039662704 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556dfa68cce0 con 0x556dfa75c100 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b71ffb700 10 client.17145 handle_command_reply: tid=0 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 client.17145 shutdown 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 msgr2=0x556dfa74c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa74c4d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5b84000c00 tx=0x7f5b8409ac00 comp rx=0 tx=0).stop 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 shutdown_connections 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa75c100 0x556dfa75e4e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa74cb50 0x556dfa749d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa74c4d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b71ffb700 0 client.17145 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/1039662704 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741c90 0x556dfa749780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 >> 172.21.15.31:0/1039662704 conn(0x556dfa737ca0 msgr2=0x556dfa73d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 shutdown_connections 2023-12-14T18:35:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 1 -- 172.21.15.31:0/1039662704 wait complete. 2023-12-14T18:35:06.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.400+0000 7f5b9c893b80 20 client.17145 trim_cache size 0 max 16384 2023-12-14T18:35:06.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 Processor -- start 2023-12-14T18:35:06.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 -- start start 2023-12-14T18:35:06.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa753690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa757e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa7425b0 con 0x556dfa74cb50 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa73d0e0 con 0x556dfa741380 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa73ebc0 con 0x556dfa741c90 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b8eca2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b8eca2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60690/0 (socket says 172.21.15.31:60690) 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b8eca2700 1 -- 172.21.15.31:0/2449939990 learned_addr learned my addr 172.21.15.31:0/2449939990 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa757e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.414+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa753690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b8eca2700 1 -- 172.21.15.31:0/2449939990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 msgr2=0x556dfa757e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa757e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b8eca2700 1 -- 172.21.15.31:0/2449939990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 msgr2=0x556dfa753690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa753690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b8eca2700 1 -- 172.21.15.31:0/2449939990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556dfa659ec0 con 0x556dfa74cb50 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74bf90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5b80011680 tx=0x7f5b80009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcf321a248b7f2c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b7d7fa700 1 -- 172.21.15.31:0/2449939990 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8000e070 con 0x556dfa74cb50 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b7d7fa700 1 -- 172.21.15.31:0/2449939990 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b80022070 con 0x556dfa74cb50 2023-12-14T18:35:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b7d7fa700 1 -- 172.21.15.31:0/2449939990 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b80007850 con 0x556dfa74cb50 2023-12-14T18:35:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2449939990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 msgr2=0x556dfa74bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.415+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74bf90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5b80011680 tx=0x7f5b80009e80 comp rx=0 tx=0).stop 2023-12-14T18:35:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2449939990 shutdown_connections 2023-12-14T18:35:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa74cb50 0x556dfa74bf90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa741380 0x556dfa753690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2449939990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa757e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2449939990 >> 172.21.15.31:0/2449939990 conn(0x556dfa737ca0 msgr2=0x556dfa73c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2449939990 shutdown_connections 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2449939990 wait complete. 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- start start 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 Processor -- start 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa751860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa751da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa74cb50 0x556dfa7a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa7a9510 con 0x556dfa741380 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa7a9680 con 0x556dfa74cb50 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556dfa7a9840 con 0x556dfa741c90 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b9c893b80 10 client.0 _ll_register_callbacks cb 0x7f5b8409a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b8e4a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa751860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b8e4a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa751860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60694/0 (socket says 172.21.15.31:60694) 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.416+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/2557197632 learned_addr learned my addr 172.21.15.31:0/2557197632 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa751da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa74cb50 0x556dfa7a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 msgr2=0x556dfa751da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa751da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/2557197632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa74cb50 msgr2=0x556dfa7a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa74cb50 0x556dfa7a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8e4a1700 1 -- 172.21.15.31:0/2557197632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556dfa5153b0 con 0x556dfa741380 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8eca2700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa74cb50 0x556dfa7a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b8e4a1700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa751860 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5b84097010 tx=0x7f5b840a2c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f64c019616d5832 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b71ffb700 1 -- 172.21.15.31:0/2557197632 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b840056b0 con 0x556dfa741380 2023-12-14T18:35:06.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b71ffb700 1 -- 172.21.15.31:0/2557197632 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f5b84096a50 con 0x556dfa741380 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b8000f040 con 0x556dfa741380 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b71ffb700 1 -- 172.21.15.31:0/2557197632 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b84096c90 con 0x556dfa741380 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b9c893b80 10 client.17151 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x556dfa6409a0 con 0x556dfa741380 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.417+0000 7f5b71ffb700 1 -- 172.21.15.31:0/2557197632 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b8000f040 con 0x556dfa741380 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b71ffb700 1 -- 172.21.15.31:0/2557197632 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5b840c6070 con 0x556dfa741380 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b9c893b80 10 client.17151 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b9c893b80 10 client.17151 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b9c893b80 20 client.17151 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa7b0b20 0x556dfa7b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b9c893b80 4 client.17151 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556dfa68ca00 con 0x556dfa7b0b20 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa7b0b20 0x556dfa7b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b71ffb700 10 client.17151 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.418+0000 7f5b8dca0700 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa7b0b20 0x556dfa7b2f30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=701161db4a1b5726 server_cookie=41f44fbc156d8a98 in_seq=0 out_seq=0 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b71ffb700 1 -- 172.21.15.31:0/2557197632 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556dfa68ca00 con 0x556dfa7b0b20 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b71ffb700 10 client.17151 handle_command_reply: tid=0 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 client.17151 shutdown 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 msgr2=0x556dfa751860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa751860 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5b84097010 tx=0x7f5b840a2c50 comp rx=0 tx=0).stop 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 shutdown_connections 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x556dfa7b0b20 0x556dfa7b2f30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556dfa74cb50 0x556dfa7a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556dfa741380 0x556dfa751860 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b71ffb700 0 client.17151 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 --2- 172.21.15.31:0/2557197632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556dfa741c90 0x556dfa751da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 >> 172.21.15.31:0/2557197632 conn(0x556dfa737ca0 msgr2=0x556dfa757e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 shutdown_connections 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 1 -- 172.21.15.31:0/2557197632 wait complete. 2023-12-14T18:35:06.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b9c893b80 20 client.17151 trim_cache size 0 max 16384 2023-12-14T18:35:06.429 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:35:06.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5b740a1340 msgr2=0x7f5b740a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b90705700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5b740a1340 0x7f5b740a37f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b88197ad0 tx=0x7f5b78009260 comp rx=0 tx=0).stop 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 msgr2=0x7f5b881964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b90705700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b881964c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400b580 tx=0x7f5b8400bfd0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.419+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 shutdown_connections 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5b881043d0 0x7f5b8819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b880ff390 0x7f5b881964c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5b740a1340 0x7f5b740a37f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 --2- 172.21.15.31:0/3230918989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b880ffd30 0x7f5b88196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 >> 172.21.15.31:0/3230918989 conn(0x7f5b88075fd0 msgr2=0x7f5b88067f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 shutdown_connections 2023-12-14T18:35:06.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.420+0000 7f5b90705700 1 -- 172.21.15.31:0/3230918989 wait complete. 2023-12-14T18:35:06.434 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:35:06.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 -- 172.21.15.31:0/1232636235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a840ff3d0 msgr2=0x7f9a840ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 --2- 172.21.15.31:0/1232636235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a840ff3d0 0x7f9a840ff7a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9a780099c0 tx=0x7f9a78009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 -- 172.21.15.31:0/1232636235 shutdown_connections 2023-12-14T18:35:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 --2- 172.21.15.31:0/1232636235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a84104410 0x7f9a8410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 --2- 172.21.15.31:0/1232636235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a840ff3d0 0x7f9a840ff7a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 --2- 172.21.15.31:0/1232636235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a840ffd70 0x7f9a84103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 -- 172.21.15.31:0/1232636235 >> 172.21.15.31:0/1232636235 conn(0x7f9a84076000 msgr2=0x7f9a84076400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 -- 172.21.15.31:0/1232636235 shutdown_connections 2023-12-14T18:35:06.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.573+0000 7f9a893af700 1 -- 172.21.15.31:0/1232636235 wait complete. 2023-12-14T18:35:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 Processor -- start 2023-12-14T18:35:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 -- start start 2023-12-14T18:35:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a840ff3d0 0x7f9a841952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a840ffd70 0x7f9a84195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 0x7f9a84199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419a2b0 con 0x7f9a84104410 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419a420 con 0x7f9a840ff3d0 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a893af700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8419a5e0 con 0x7f9a840ffd70 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 0x7f9a84199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 0x7f9a84199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60712/0 (socket says 172.21.15.31:60712) 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a837fe700 1 -- 172.21.15.31:0/2640614030 learned_addr learned my addr 172.21.15.31:0/2640614030 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a840ff3d0 0x7f9a841952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.574+0000 7f9a827fc700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a840ffd70 0x7f9a84195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a837fe700 1 -- 172.21.15.31:0/2640614030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a840ffd70 msgr2=0x7f9a84195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a837fe700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a840ffd70 0x7f9a84195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a837fe700 1 -- 172.21.15.31:0/2640614030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a840ff3d0 msgr2=0x7f9a841952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a837fe700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a840ff3d0 0x7f9a841952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a837fe700 1 -- 172.21.15.31:0/2640614030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780095f0 con 0x7f9a84104410 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a837fe700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 0x7f9a84199c70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00ef30 tx=0x7f9a7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a6bfff700 1 -- 172.21.15.31:0/2640614030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c018070 con 0x7f9a84104410 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a6bfff700 1 -- 172.21.15.31:0/2640614030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7c00f040 con 0x7f9a84104410 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8410f1d0 con 0x7f9a84104410 2023-12-14T18:35:06.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a6bfff700 1 -- 172.21.15.31:0/2640614030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00cd00 con 0x7f9a84104410 2023-12-14T18:35:06.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.575+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8410f040 con 0x7f9a84104410 2023-12-14T18:35:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.578+0000 7f9a6bfff700 1 -- 172.21.15.31:0/2640614030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9a7c003a00 con 0x7f9a84104410 2023-12-14T18:35:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.578+0000 7f9a6bfff700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a6c0a12c0 0x7f9a6c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a6bfff700 1 -- 172.21.15.31:0/2640614030 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9a7c0ccbe0 con 0x7f9a84104410 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a6c0a12c0 0x7f9a6c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 Processor -- start 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a6c0a12c0 0x7f9a6c0a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009fd0 tx=0x7f9a78005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 -- start start 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1cc90 0x5609f3d29820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d29d60 0x5609f3d2c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3c065e0 con 0x5609f3d1c380 2023-12-14T18:35:06.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3c75870 con 0x5609f3d1cc90 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.579+0000 7f9a9553db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3c6ed00 con 0x5609f3d29d60 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d1c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60718/0 (socket says 172.21.15.31:60718) 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3318999293 learned_addr learned my addr 172.21.15.31:0/3318999293 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a837fe700 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d29d60 0x5609f3d2c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a827fc700 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1cc90 0x5609f3d29820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3318999293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d29d60 msgr2=0x5609f3d2c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d29d60 0x5609f3d2c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3318999293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1cc90 msgr2=0x5609f3d29820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1cc90 0x5609f3d29820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3318999293 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f3af03b0 con 0x5609f3d1c380 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d1c750 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9a780099c0 tx=0x7f9a7801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87bd054af6881ff server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a697fa700 1 -- 172.21.15.31:0/3318999293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7800b990 con 0x5609f3d1c380 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a697fa700 1 -- 172.21.15.31:0/3318999293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7801e040 con 0x5609f3d1c380 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.580+0000 7f9a697fa700 1 -- 172.21.15.31:0/3318999293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78004800 con 0x5609f3d1c380 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 -- 172.21.15.31:0/3318999293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 msgr2=0x5609f3d1c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d1c750 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9a780099c0 tx=0x7f9a7801f000 comp rx=0 tx=0).stop 2023-12-14T18:35:06.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 -- 172.21.15.31:0/3318999293 shutdown_connections 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d29d60 0x5609f3d2c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d1c750 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3318999293 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1cc90 0x5609f3d29820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 -- 172.21.15.31:0/3318999293 >> 172.21.15.31:0/3318999293 conn(0x5609f3d12ca0 msgr2=0x5609f3d187e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 -- 172.21.15.31:0/3318999293 shutdown_connections 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 -- 172.21.15.31:0/3318999293 wait complete. 2023-12-14T18:35:06.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.581+0000 7f9a9553db80 1 -- start start 2023-12-14T18:35:06.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 Processor -- start 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1c380 0x5609f3d296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 0x5609f3d26970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d26ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3c0af90 con 0x5609f3d1cc90 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3c0ac30 con 0x5609f3d29d60 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3af26a0 con 0x5609f3d1c380 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a9553db80 10 client.0 _ll_register_callbacks cb 0x5609f3c2e910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 0x5609f3d26970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 0x5609f3d26970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60720/0 (socket says 172.21.15.31:60720) 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 -- 172.21.15.31:0/1328692278 learned_addr learned my addr 172.21.15.31:0/1328692278 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1c380 0x5609f3d296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a837fe700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d26ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 -- 172.21.15.31:0/1328692278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1c380 msgr2=0x5609f3d296c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1c380 0x5609f3d296c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 -- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 msgr2=0x5609f3d26ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d26ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a827fc700 1 -- 172.21.15.31:0/1328692278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8410c820 con 0x5609f3d1cc90 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.582+0000 7f9a837fe700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d26ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:06.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a827fc700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 0x5609f3d26970 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9a74013500 tx=0x7f9a74013810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f19cfff330d3a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a66ff5700 1 -- 172.21.15.31:0/1328692278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74004460 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a66ff5700 1 -- 172.21.15.31:0/1328692278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9a74003680 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609f3d102b0 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a66ff5700 1 -- 172.21.15.31:0/1328692278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a74003840 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 10 client.17163 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609f3c35170 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a66ff5700 1 -- 172.21.15.31:0/1328692278 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609f3d102b0 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a66ff5700 1 -- 172.21.15.31:0/1328692278 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9a7401ca70 con 0x5609f3d1cc90 2023-12-14T18:35:06.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 10 client.17163 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 10 client.17163 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 20 client.17163 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d39720 0x5609f3d3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 4 client.17163 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.583+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609f3c67ce0 con 0x5609f3d39720 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.584+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d39720 0x5609f3d3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.584+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d39720 0x5609f3d3bb30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56a18639b261da3a server_cookie=e39050a82101041f in_seq=0 out_seq=0 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.584+0000 7f9a66ff5700 10 client.17163 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:06.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a66ff5700 1 -- 172.21.15.31:0/1328692278 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5609f3c67ce0 con 0x5609f3d39720 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a66ff5700 10 client.17163 handle_command_reply: tid=0 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 client.17163 shutdown 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 msgr2=0x5609f3d26970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 0x5609f3d26970 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9a74013500 tx=0x7f9a74013810 comp rx=0 tx=0).stop 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 shutdown_connections 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d26ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d39720 0x5609f3d3bb30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1c380 0x5609f3d296c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 --2- 172.21.15.31:0/1328692278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1cc90 0x5609f3d26970 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 >> 172.21.15.31:0/1328692278 conn(0x5609f3d12ca0 msgr2=0x5609f3d183d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a66ff5700 0 client.17163 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:06.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 shutdown_connections 2023-12-14T18:35:06.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 1 -- 172.21.15.31:0/1328692278 wait complete. 2023-12-14T18:35:06.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.585+0000 7f9a9553db80 20 client.17163 trim_cache size 0 max 16384 2023-12-14T18:35:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 Processor -- start 2023-12-14T18:35:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 -- start start 2023-12-14T18:35:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1c380 0x5609f3d30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d29180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 0x5609f3d296c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3d27530 con 0x5609f3d29d60 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3d217c0 con 0x5609f3d1c380 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a9553db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3d1d330 con 0x5609f3d1cc90 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 0x5609f3d296c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 0x5609f3d296c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60736/0 (socket says 172.21.15.31:60736) 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a837fe700 1 -- 172.21.15.31:0/655051274 learned_addr learned my addr 172.21.15.31:0/655051274 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1c380 0x5609f3d30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.599+0000 7f9a827fc700 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a837fe700 1 -- 172.21.15.31:0/655051274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 msgr2=0x5609f3d29180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a837fe700 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d29180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a837fe700 1 -- 172.21.15.31:0/655051274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1c380 msgr2=0x5609f3d30830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a837fe700 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1c380 0x5609f3d30830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a837fe700 1 -- 172.21.15.31:0/655051274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609f3d144e0 con 0x5609f3d29d60 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a827fc700 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d29180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a837fe700 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 0x5609f3d296c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c095df0 tx=0x7f9a7c0997f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f4670f37643371 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a69ffb700 1 -- 172.21.15.31:0/655051274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00ce60 con 0x5609f3d29d60 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a69ffb700 1 -- 172.21.15.31:0/655051274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7c0b7e90 con 0x5609f3d29d60 2023-12-14T18:35:06.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a69ffb700 1 -- 172.21.15.31:0/655051274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c099b50 con 0x5609f3d29d60 2023-12-14T18:35:06.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a9553db80 1 -- 172.21.15.31:0/655051274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 msgr2=0x5609f3d296c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a9553db80 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 0x5609f3d296c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c095df0 tx=0x7f9a7c0997f0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a9553db80 1 -- 172.21.15.31:0/655051274 shutdown_connections 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a9553db80 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d29d60 0x5609f3d296c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a9553db80 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d1c380 0x5609f3d30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.600+0000 7f9a9553db80 1 --2- 172.21.15.31:0/655051274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d29180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- 172.21.15.31:0/655051274 >> 172.21.15.31:0/655051274 conn(0x5609f3d12ca0 msgr2=0x5609f3d35950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- 172.21.15.31:0/655051274 shutdown_connections 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- 172.21.15.31:0/655051274 wait complete. 2023-12-14T18:35:06.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- start start 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 Processor -- start 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d24330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d22c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d23150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3d24930 con 0x5609f3d1c380 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3d24aa0 con 0x5609f3d29d60 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609f3d236e0 con 0x5609f3d1cc90 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a9553db80 10 client.0 _ll_register_callbacks cb 0x7f9a74009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d24330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a82ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d24330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60752/0 (socket says 172.21.15.31:60752) 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.601+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3424449108 learned_addr learned my addr 172.21.15.31:0/3424449108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a837fe700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d23150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a827fc700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d22c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3424449108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 msgr2=0x5609f3d22c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d22c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 msgr2=0x5609f3d23150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d23150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a82ffd700 1 -- 172.21.15.31:0/3424449108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8410f040 con 0x5609f3d1c380 2023-12-14T18:35:06.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a837fe700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d23150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a827fc700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d22c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a82ffd700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d24330 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009d10 tx=0x7f9a780059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b84283094def59 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a66ff5700 1 -- 172.21.15.31:0/3424449108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78036040 con 0x5609f3d1c380 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a66ff5700 1 -- 172.21.15.31:0/3424449108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9a7803c9c0 con 0x5609f3d1c380 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a7801e040 con 0x5609f3d1c380 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a66ff5700 1 -- 172.21.15.31:0/3424449108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a7803c620 con 0x5609f3d1c380 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a9553db80 10 client.17169 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.602+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5609f3a9cb60 con 0x5609f3d1c380 2023-12-14T18:35:06.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a66ff5700 1 -- 172.21.15.31:0/3424449108 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a7801e040 con 0x5609f3d1c380 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a66ff5700 1 -- 172.21.15.31:0/3424449108 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9a78030480 con 0x5609f3d1c380 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a9553db80 10 client.17169 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a9553db80 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a9553db80 20 client.17169 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d8e400 0x5609f3d90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a9553db80 4 client.17169 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609f3c67a00 con 0x5609f3d8e400 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a827fc700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d8e400 0x5609f3d90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a66ff5700 10 client.17169 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:06.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.603+0000 7f9a827fc700 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d8e400 0x5609f3d90810 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=232a89d7eed5c629 server_cookie=43823d31a8f750a7 in_seq=0 out_seq=0 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a66ff5700 1 -- 172.21.15.31:0/3424449108 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609f3c67a00 con 0x5609f3d8e400 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a66ff5700 10 client.17169 handle_command_reply: tid=0 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 client.17169 shutdown 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 msgr2=0x5609f3d24330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d24330 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009d10 tx=0x7f9a780059f0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 shutdown_connections 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5609f3d29d60 0x5609f3d23150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5609f3d1c380 0x5609f3d24330 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5609f3d8e400 0x5609f3d90810 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 --2- 172.21.15.31:0/3424449108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609f3d1cc90 0x5609f3d22c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a66ff5700 0 client.17169 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 >> 172.21.15.31:0/3424449108 conn(0x5609f3d12ca0 msgr2=0x5609f3d21cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 shutdown_connections 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 1 -- 172.21.15.31:0/3424449108 wait complete. 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a9553db80 20 client.17169 trim_cache size 0 max 16384 2023-12-14T18:35:06.614 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:35:06.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a6c0a12c0 msgr2=0x7f9a6c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a893af700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a6c0a12c0 0x7f9a6c0a3770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009fd0 tx=0x7f9a78005fb0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 msgr2=0x7f9a84199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.604+0000 7f9a893af700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 0x7f9a84199c70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00ef30 tx=0x7f9a7c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 shutdown_connections 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a84104410 0x7f9a84199c70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a840ff3d0 0x7f9a841952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9a6c0a12c0 0x7f9a6c0a3770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 --2- 172.21.15.31:0/2640614030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a840ffd70 0x7f9a84195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 >> 172.21.15.31:0/2640614030 conn(0x7f9a84076000 msgr2=0x7f9a840fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 shutdown_connections 2023-12-14T18:35:06.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.605+0000 7f9a893af700 1 -- 172.21.15.31:0/2640614030 wait complete. 2023-12-14T18:35:06.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:35:06.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:06 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3737764070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:35:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 -- 172.21.15.31:0/4135182722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec068a60 msgr2=0x7f73ec104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 --2- 172.21.15.31:0/4135182722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec068a60 0x7f73ec104df0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f73d80099c0 tx=0x7f73d8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 -- 172.21.15.31:0/4135182722 shutdown_connections 2023-12-14T18:35:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 --2- 172.21.15.31:0/4135182722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ec1054d0 0x7f73ec10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 --2- 172.21.15.31:0/4135182722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73ec0680c0 0x7f73ec068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 --2- 172.21.15.31:0/4135182722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec068a60 0x7f73ec104df0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.756+0000 7f73f322c700 1 -- 172.21.15.31:0/4135182722 >> 172.21.15.31:0/4135182722 conn(0x7f73ec075fd0 msgr2=0x7f73ec0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 -- 172.21.15.31:0/4135182722 shutdown_connections 2023-12-14T18:35:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 -- 172.21.15.31:0/4135182722 wait complete. 2023-12-14T18:35:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 Processor -- start 2023-12-14T18:35:06.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 -- start start 2023-12-14T18:35:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ec0680c0 0x7f73ec196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73ec068a60 0x7f73ec1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 0x7f73ec19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec19b360 con 0x7f73ec1054d0 2023-12-14T18:35:06.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec19b4d0 con 0x7f73ec068a60 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.757+0000 7f73f322c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec19b690 con 0x7f73ec0680c0 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 0x7f73ec19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 0x7f73ec19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60764/0 (socket says 172.21.15.31:60764) 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 -- 172.21.15.31:0/1938016729 learned_addr learned my addr 172.21.15.31:0/1938016729 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ec0680c0 0x7f73ec196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73ebfff700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73ec068a60 0x7f73ec1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 -- 172.21.15.31:0/1938016729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ec0680c0 msgr2=0x7f73ec196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ec0680c0 0x7f73ec196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 -- 172.21.15.31:0/1938016729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73ec068a60 msgr2=0x7f73ec1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73ec068a60 0x7f73ec1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 -- 172.21.15.31:0/1938016729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80095f0 con 0x7f73ec1054d0 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f17c9700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 0x7f73ec19ad20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007ee0 tx=0x7f73e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73e9ffb700 1 -- 172.21.15.31:0/1938016729 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4017070 con 0x7f73ec1054d0 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73e9ffb700 1 -- 172.21.15.31:0/1938016729 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e400f040 con 0x7f73ec1054d0 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ec110250 con 0x7f73ec1054d0 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.758+0000 7f73e9ffb700 1 -- 172.21.15.31:0/1938016729 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4005740 con 0x7f73ec1054d0 2023-12-14T18:35:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.759+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec1100c0 con 0x7f73ec1054d0 2023-12-14T18:35:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.761+0000 7f73e9ffb700 1 -- 172.21.15.31:0/1938016729 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f73e40058a0 con 0x7f73ec1054d0 2023-12-14T18:35:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.761+0000 7f73e9ffb700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f73dc0a12c0 0x7f73dc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.762+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f73dc0a12c0 0x7f73dc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.762+0000 7f73e9ffb700 1 -- 172.21.15.31:0/1938016729 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f73e40cc940 con 0x7f73ec1054d0 2023-12-14T18:35:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.762+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f73dc0a12c0 0x7f73dc0a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005fd0 tx=0x7f73e0005f60 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 Processor -- start 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 -- start start 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95c380 0x558c4a95c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 0x558c4a96ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a96e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a8465e0 con 0x558c4a95cc90 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a8b5870 con 0x558c4a96bf60 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a8aed00 con 0x558c4a95c380 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 0x558c4a96ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 0x558c4a96ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60774/0 (socket says 172.21.15.31:60774) 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73ebfff700 1 -- 172.21.15.31:0/2606856723 learned_addr learned my addr 172.21.15.31:0/2606856723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73f17c9700 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a96e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.763+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95c380 0x558c4a95c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73ebfff700 1 -- 172.21.15.31:0/2606856723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95c380 msgr2=0x558c4a95c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73ebfff700 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95c380 0x558c4a95c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73ebfff700 1 -- 172.21.15.31:0/2606856723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 msgr2=0x558c4a96e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73ebfff700 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a96e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73ebfff700 1 -- 172.21.15.31:0/2606856723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c4a7303b0 con 0x558c4a95cc90 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95c380 0x558c4a95c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73ebfff700 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 0x558c4a96ba20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f73ec068e30 tx=0x7f73d8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533982337c41595a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73d6ffd700 1 -- 172.21.15.31:0/2606856723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8005240 con 0x558c4a95cc90 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73d6ffd700 1 -- 172.21.15.31:0/2606856723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73d801e040 con 0x558c4a95cc90 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.764+0000 7f73d6ffd700 1 -- 172.21.15.31:0/2606856723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8003b20 con 0x558c4a95cc90 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 -- 172.21.15.31:0/2606856723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 msgr2=0x558c4a96ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 0x558c4a96ba20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f73ec068e30 tx=0x7f73d8009f80 comp rx=0 tx=0).stop 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 -- 172.21.15.31:0/2606856723 shutdown_connections 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a96e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95c380 0x558c4a95c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/2606856723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95cc90 0x558c4a96ba20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 -- 172.21.15.31:0/2606856723 >> 172.21.15.31:0/2606856723 conn(0x558c4a952ca0 msgr2=0x558c4a9587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 -- 172.21.15.31:0/2606856723 shutdown_connections 2023-12-14T18:35:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 -- 172.21.15.31:0/2606856723 wait complete. 2023-12-14T18:35:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 -- start start 2023-12-14T18:35:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.765+0000 7f73ff3bab80 1 Processor -- start 2023-12-14T18:35:06.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a96b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a968c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a969290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a84af90 con 0x558c4a96bf60 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a84ac30 con 0x558c4a95c380 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a7326a0 con 0x558c4a95cc90 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ff3bab80 10 client.0 _ll_register_callbacks cb 0x558c4a86e910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73f17c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a969290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73f17c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a969290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60788/0 (socket says 172.21.15.31:60788) 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73f17c9700 1 -- 172.21.15.31:0/3885799150 learned_addr learned my addr 172.21.15.31:0/3885799150 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73ebfff700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a968c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.766+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a96b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f17c9700 1 -- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 msgr2=0x558c4a968c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a968c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f17c9700 1 -- 172.21.15.31:0/3885799150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 msgr2=0x558c4a96b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a96b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f17c9700 1 -- 172.21.15.31:0/3885799150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ec10d8e0 con 0x558c4a96bf60 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a96b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a969290 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f73e400dfd0 tx=0x7f73e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d26a7d9952c4f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73cbfff700 1 -- 172.21.15.31:0/3885799150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40842c0 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73cbfff700 1 -- 172.21.15.31:0/3885799150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f73e409d5f0 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c4a9502b0 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73cbfff700 1 -- 172.21.15.31:0/3885799150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73e40a1a10 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73ff3bab80 10 client.17181 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558c4a875170 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.767+0000 7f73cbfff700 1 -- 172.21.15.31:0/3885799150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c4a9502b0 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73cbfff700 1 -- 172.21.15.31:0/3885799150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f73e4098ab0 con 0x558c4a96bf60 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73ff3bab80 10 client.17181 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:06.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73ff3bab80 10 client.17181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73ff3bab80 20 client.17181 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a97b7f0 0x558c4a97dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73ff3bab80 4 client.17181 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c4a8a7ce0 con 0x558c4a97b7f0 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a97b7f0 0x558c4a97dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a97b7f0 0x558c4a97dc00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71af7bc0a90f0400 server_cookie=eaa36622f08331ef in_seq=0 out_seq=0 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.768+0000 7f73cbfff700 10 client.17181 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73cbfff700 1 -- 172.21.15.31:0/3885799150 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558c4a8a7ce0 con 0x558c4a97b7f0 2023-12-14T18:35:06.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73cbfff700 10 client.17181 handle_command_reply: tid=0 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 client.17181 shutdown 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 msgr2=0x558c4a969290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a969290 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f73e400dfd0 tx=0x7f73e4004ad0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 shutdown_connections 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a97b7f0 0x558c4a97dc00 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a969290 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73cbfff700 0 client.17181 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a96b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3885799150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a968c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 >> 172.21.15.31:0/3885799150 conn(0x558c4a952ca0 msgr2=0x558c4a9583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 shutdown_connections 2023-12-14T18:35:06.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.769+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3885799150 wait complete. 2023-12-14T18:35:06.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.770+0000 7f73ff3bab80 20 client.17181 trim_cache size 0 max 16384 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.783+0000 7f73ff3bab80 1 Processor -- start 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.783+0000 7f73ff3bab80 1 -- start start 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.783+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a972a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.783+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a96b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a963f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a96a260 con 0x558c4a96bf60 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a959bf0 con 0x558c4a95c380 2023-12-14T18:35:06.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a9671e0 con 0x558c4a95cc90 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a963f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a963f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60792/0 (socket says 172.21.15.31:60792) 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 -- 172.21.15.31:0/3026400838 learned_addr learned my addr 172.21.15.31:0/3026400838 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73ebfff700 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a96b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a972a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 -- 172.21.15.31:0/3026400838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 msgr2=0x558c4a96b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a96b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 -- 172.21.15.31:0/3026400838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 msgr2=0x558c4a972a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a972a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 -- 172.21.15.31:0/3026400838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c4a874ec0 con 0x558c4a96bf60 2023-12-14T18:35:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a963f60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f73e4099780 tx=0x7f73e40b9190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec88b3a4d3752175 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73d77fe700 1 -- 172.21.15.31:0/3026400838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40842c0 con 0x558c4a96bf60 2023-12-14T18:35:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73d77fe700 1 -- 172.21.15.31:0/3026400838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73e40a15d0 con 0x558c4a96bf60 2023-12-14T18:35:06.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.784+0000 7f73d77fe700 1 -- 172.21.15.31:0/3026400838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4098bd0 con 0x558c4a96bf60 2023-12-14T18:35:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3026400838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 msgr2=0x558c4a963f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a963f60 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f73e4099780 tx=0x7f73e40b9190 comp rx=0 tx=0).stop 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3026400838 shutdown_connections 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a96bf60 0x558c4a963f60 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a95c380 0x558c4a972a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3026400838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a96b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3026400838 >> 172.21.15.31:0/3026400838 conn(0x558c4a952ca0 msgr2=0x558c4a977c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3026400838 shutdown_connections 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3026400838 wait complete. 2023-12-14T18:35:06.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.785+0000 7f73ff3bab80 1 -- start start 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 Processor -- start 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 0x558c4a972470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a9729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a970a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a971000 con 0x558c4a95c380 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a971170 con 0x558c4a96bf60 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c4a971330 con 0x558c4a95cc90 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ff3bab80 10 client.0 _ll_register_callbacks cb 0x7f73e40991d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 0x558c4a972470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 0x558c4a972470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60808/0 (socket says 172.21.15.31:60808) 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 -- 172.21.15.31:0/3671840667 learned_addr learned my addr 172.21.15.31:0/3671840667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a970a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73ebfff700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a9729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 -- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 msgr2=0x558c4a9729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a9729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 -- 172.21.15.31:0/3671840667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 msgr2=0x558c4a970a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.786+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a970a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73f0fc8700 1 -- 172.21.15.31:0/3671840667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c4a875170 con 0x558c4a95c380 2023-12-14T18:35:06.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73f17c9700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a970a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73f0fc8700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 0x558c4a972470 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f73e000b490 tx=0x7f73e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12fa7313a640cf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73cbfff700 1 -- 172.21.15.31:0/3671840667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0028ad0 con 0x558c4a95c380 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73cbfff700 1 -- 172.21.15.31:0/3671840667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f73e0025070 con 0x558c4a95c380 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73d801e040 con 0x558c4a95c380 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73cbfff700 1 -- 172.21.15.31:0/3671840667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73e0028c30 con 0x558c4a95c380 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73ff3bab80 10 client.17187 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558c4a85b9a0 con 0x558c4a95c380 2023-12-14T18:35:06.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73cbfff700 1 -- 172.21.15.31:0/3671840667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73d801e040 con 0x558c4a95c380 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.787+0000 7f73cbfff700 1 -- 172.21.15.31:0/3671840667 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f73e00316a0 con 0x558c4a95c380 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 10 client.17187 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 20 client.17187 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a9d0930 0x558c4a9d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 4 client.17187 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558c4a8a7a00 con 0x558c4a9d0930 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ebfff700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a9d0930 0x558c4a9d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:06.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73cbfff700 10 client.17187 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ebfff700 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a9d0930 0x558c4a9d2d40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214627e384b197b1 server_cookie=db5a3d8d9598e68a in_seq=0 out_seq=0 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73cbfff700 1 -- 172.21.15.31:0/3671840667 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558c4a8a7a00 con 0x558c4a9d0930 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73cbfff700 10 client.17187 handle_command_reply: tid=0 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.788+0000 7f73ff3bab80 1 client.17187 shutdown 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 msgr2=0x558c4a972470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 0x558c4a972470 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f73e000b490 tx=0x7f73e00047c0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 shutdown_connections 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558c4a9d0930 0x558c4a9d2d40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558c4a96bf60 0x558c4a970a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558c4a95c380 0x558c4a972470 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73cbfff700 0 client.17187 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 --2- 172.21.15.31:0/3671840667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c4a95cc90 0x558c4a9729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 >> 172.21.15.31:0/3671840667 conn(0x558c4a952ca0 msgr2=0x558c4a963f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 shutdown_connections 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 1 -- 172.21.15.31:0/3671840667 wait complete. 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73ff3bab80 20 client.17187 trim_cache size 0 max 16384 2023-12-14T18:35:06.799 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:35:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f73dc0a12c0 msgr2=0x7f73dc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73f322c700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f73dc0a12c0 0x7f73dc0a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f73e0005fd0 tx=0x7f73e0005f60 comp rx=0 tx=0).stop 2023-12-14T18:35:06.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 msgr2=0x7f73ec19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.789+0000 7f73f322c700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 0x7f73ec19ad20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f73e4007ee0 tx=0x7f73e400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 shutdown_connections 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73ec1054d0 0x7f73ec19ad20 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f73dc0a12c0 0x7f73dc0a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73ec0680c0 0x7f73ec196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 --2- 172.21.15.31:0/1938016729 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73ec068a60 0x7f73ec1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 >> 172.21.15.31:0/1938016729 conn(0x7f73ec075fd0 msgr2=0x7f73ec0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:06.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 shutdown_connections 2023-12-14T18:35:06.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:06.790+0000 7f73f322c700 1 -- 172.21.15.31:0/1938016729 wait complete. 2023-12-14T18:35:06.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:06 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3737764070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:35:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:06 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3737764070' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:35:07.805 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:35:07.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:35:07.837 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:07 smithi031 ceph-mon[42085]: pgmap v224: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:35:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:07 smithi204 ceph-mon[43866]: pgmap v224: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:35:07.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 -- 172.21.15.31:0/2982306824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc0ff420 msgr2=0x7febdc0ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:07.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 --2- 172.21.15.31:0/2982306824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc0ff420 0x7febdc0ff7f0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7febcc0099c0 tx=0x7febcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 -- 172.21.15.31:0/2982306824 shutdown_connections 2023-12-14T18:35:07.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 --2- 172.21.15.31:0/2982306824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febdc104410 0x7febdc10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 --2- 172.21.15.31:0/2982306824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc0ff420 0x7febdc0ff7f0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 --2- 172.21.15.31:0/2982306824 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febdc0ffdc0 0x7febdc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.954+0000 7febe270e700 1 -- 172.21.15.31:0/2982306824 >> 172.21.15.31:0/2982306824 conn(0x7febdc075fc0 msgr2=0x7febdc0763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:07.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.955+0000 7febe270e700 1 -- 172.21.15.31:0/2982306824 shutdown_connections 2023-12-14T18:35:07.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.955+0000 7febe270e700 1 -- 172.21.15.31:0/2982306824 wait complete. 2023-12-14T18:35:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.955+0000 7febe270e700 1 Processor -- start 2023-12-14T18:35:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 -- start start 2023-12-14T18:35:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febdc0ff420 0x7febdc197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febdc0ffdc0 0x7febdc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 0x7febdc1a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc198080 con 0x7febdc104410 2023-12-14T18:35:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a27c0 con 0x7febdc0ff420 2023-12-14T18:35:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe270e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a2980 con 0x7febdc0ffdc0 2023-12-14T18:35:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe0cab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 0x7febdc1a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe0cab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 0x7febdc1a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60824/0 (socket says 172.21.15.31:60824) 2023-12-14T18:35:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febe0cab700 1 -- 172.21.15.31:0/2885636053 learned_addr learned my addr 172.21.15.31:0/2885636053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:07.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febdb7fe700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febdc0ffdc0 0x7febdc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.956+0000 7febdbfff700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febdc0ff420 0x7febdc197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe0cab700 1 -- 172.21.15.31:0/2885636053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febdc0ffdc0 msgr2=0x7febdc197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe0cab700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febdc0ffdc0 0x7febdc197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe0cab700 1 -- 172.21.15.31:0/2885636053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febdc0ff420 msgr2=0x7febdc197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe0cab700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febdc0ff420 0x7febdc197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe0cab700 1 -- 172.21.15.31:0/2885636053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0095f0 con 0x7febdc104410 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe0cab700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 0x7febdc1a2280 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7febd4007ee0 tx=0x7febd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:07.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febd97fa700 1 -- 172.21.15.31:0/2885636053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4017070 con 0x7febdc104410 2023-12-14T18:35:07.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febd97fa700 1 -- 172.21.15.31:0/2885636053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd400f040 con 0x7febdc104410 2023-12-14T18:35:07.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc10f220 con 0x7febdc104410 2023-12-14T18:35:07.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febd97fa700 1 -- 172.21.15.31:0/2885636053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4005580 con 0x7febdc104410 2023-12-14T18:35:07.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.957+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc10f090 con 0x7febdc104410 2023-12-14T18:35:07.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.959+0000 7febd97fa700 1 -- 172.21.15.31:0/2885636053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7febd400db20 con 0x7febdc104410 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.960+0000 7febd97fa700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7febc40a12c0 0x7febc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.960+0000 7febd97fa700 1 -- 172.21.15.31:0/2885636053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7febd40cc7a0 con 0x7febdc104410 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.960+0000 7febdbfff700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7febc40a12c0 0x7febc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febdbfff700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7febc40a12c0 0x7febc40a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febcc009990 tx=0x7febcc005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 Processor -- start 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 -- start start 2023-12-14T18:35:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133f07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133fb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a132da660 con 0x555a133f0d10 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a133498f0 con 0x555a133fbbd0 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febee89cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13342d80 con 0x555a133f0400 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133fb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60830/0 (socket says 172.21.15.31:60830) 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.961+0000 7febdb7fe700 1 -- 172.21.15.31:0/3317164595 learned_addr learned my addr 172.21.15.31:0/3317164595 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdbfff700 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133f07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febe0cab700 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdb7fe700 1 -- 172.21.15.31:0/3317164595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 msgr2=0x555a133f07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdb7fe700 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133f07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdb7fe700 1 -- 172.21.15.31:0/3317164595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 msgr2=0x555a133fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdb7fe700 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdb7fe700 1 -- 172.21.15.31:0/3317164595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a131c4430 con 0x555a133f0d10 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febe0cab700 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febdb7fe700 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133fb690 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7febd0009cb0 tx=0x7febd0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa129f10b1c222 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febc27fc700 1 -- 172.21.15.31:0/3317164595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0011dd0 con 0x555a133f0d10 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febc27fc700 1 -- 172.21.15.31:0/3317164595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd0004510 con 0x555a133f0d10 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.962+0000 7febc27fc700 1 -- 172.21.15.31:0/3317164595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0005280 con 0x555a133f0d10 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 -- 172.21.15.31:0/3317164595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 msgr2=0x555a133fb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:07.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133fb690 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7febd0009cb0 tx=0x7febd0011440 comp rx=0 tx=0).stop 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 -- 172.21.15.31:0/3317164595 shutdown_connections 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133f07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 --2- 172.21.15.31:0/3317164595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133fb690 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 -- 172.21.15.31:0/3317164595 >> 172.21.15.31:0/3317164595 conn(0x555a133e6d20 msgr2=0x555a133ec860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 -- 172.21.15.31:0/3317164595 shutdown_connections 2023-12-14T18:35:07.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 -- 172.21.15.31:0/3317164595 wait complete. 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.963+0000 7febee89cb80 1 -- start start 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 Processor -- start 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133fb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a132df010 con 0x555a133f0d10 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a132decb0 con 0x555a133fbbd0 2023-12-14T18:35:07.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a131c6720 con 0x555a133f0400 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febee89cb80 10 client.0 _ll_register_callbacks cb 0x555a13302990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60844/0 (socket says 172.21.15.31:60844) 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febdb7fe700 1 -- 172.21.15.31:0/525815833 learned_addr learned my addr 172.21.15.31:0/525815833 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febe0cab700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.964+0000 7febdbfff700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febdb7fe700 1 -- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 msgr2=0x555a133fb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febdb7fe700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133fb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febdb7fe700 1 -- 172.21.15.31:0/525815833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 msgr2=0x555a133f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febdb7fe700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febdb7fe700 1 -- 172.21.15.31:0/525815833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd00098e0 con 0x555a133f0d10 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febdb7fe700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133f8830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7febd0004df0 tx=0x7febd0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e927713d945cf592 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:07.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febbb7fe700 1 -- 172.21.15.31:0/525815833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0011e60 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febbb7fe700 1 -- 172.21.15.31:0/525815833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7febd0004050 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a133e4330 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febbb7fe700 1 -- 172.21.15.31:0/525815833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd0004210 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febee89cb80 10 client.17199 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555a133091f0 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febbb7fe700 1 -- 172.21.15.31:0/525815833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555a133e4330 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.965+0000 7febbb7fe700 1 -- 172.21.15.31:0/525815833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7febd00052f0 con 0x555a133f0d10 2023-12-14T18:35:07.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febee89cb80 10 client.17199 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febee89cb80 10 client.17199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febee89cb80 20 client.17199 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febee89cb80 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a1340b4e0 0x555a1340d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febee89cb80 4 client.17199 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a1333bd60 con 0x555a1340b4e0 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febdbfff700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a1340b4e0 0x555a1340d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febbb7fe700 10 client.17199 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:07.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.966+0000 7febdbfff700 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a1340b4e0 0x555a1340d8f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242750489172f6d7 server_cookie=d27c3f34ec45ccb7 in_seq=0 out_seq=0 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febbb7fe700 1 -- 172.21.15.31:0/525815833 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555a1333bd60 con 0x555a1340b4e0 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febbb7fe700 10 client.17199 handle_command_reply: tid=0 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 client.17199 shutdown 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 msgr2=0x555a133f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133f8830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7febd0004df0 tx=0x7febd0005ce0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 shutdown_connections 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a1340b4e0 0x555a1340d8f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a133fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febbb7fe700 0 client.17199 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 --2- 172.21.15.31:0/525815833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133f8830 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 >> 172.21.15.31:0/525815833 conn(0x555a133e6d20 msgr2=0x555a133ec450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 shutdown_connections 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 1 -- 172.21.15.31:0/525815833 wait complete. 2023-12-14T18:35:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.967+0000 7febee89cb80 20 client.17199 trim_cache size 0 max 16384 2023-12-14T18:35:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 Processor -- start 2023-12-14T18:35:07.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 -- start start 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133f0400 0x555a13402730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0d10 0x555a133fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 0x555a133fb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a133f9390 con 0x555a133fbbd0 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a133ed4a0 con 0x555a133f0400 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febee89cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a133f3700 con 0x555a133f0d10 2023-12-14T18:35:07.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febe0cab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 0x555a133fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.985+0000 7febe0cab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 0x555a133fb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60858/0 (socket says 172.21.15.31:60858) 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 -- 172.21.15.31:0/758408691 learned_addr learned my addr 172.21.15.31:0/758408691 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febdb7fe700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0d10 0x555a133fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febdbfff700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133f0400 0x555a13402730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 -- 172.21.15.31:0/758408691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0d10 msgr2=0x555a133fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0d10 0x555a133fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 -- 172.21.15.31:0/758408691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133f0400 msgr2=0x555a13402730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133f0400 0x555a13402730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 -- 172.21.15.31:0/758408691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a133f83a0 con 0x555a133fbbd0 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febdbfff700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133f0400 0x555a13402730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febdb7fe700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0d10 0x555a133fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febe0cab700 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 0x555a133fb510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7febd4095980 tx=0x7febd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c258e1c9a93eb896 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febc2ffd700 1 -- 172.21.15.31:0/758408691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd40a18d0 con 0x555a133fbbd0 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febc2ffd700 1 -- 172.21.15.31:0/758408691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd40975c0 con 0x555a133fbbd0 2023-12-14T18:35:07.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.986+0000 7febc2ffd700 1 -- 172.21.15.31:0/758408691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4095b90 con 0x555a133fbbd0 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 -- 172.21.15.31:0/758408691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 msgr2=0x555a133fb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 0x555a133fb510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7febd4095980 tx=0x7febd400b040 comp rx=0 tx=0).stop 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 -- 172.21.15.31:0/758408691 shutdown_connections 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133fbbd0 0x555a133fb510 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133f0400 0x555a13402730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 --2- 172.21.15.31:0/758408691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0d10 0x555a133fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 -- 172.21.15.31:0/758408691 >> 172.21.15.31:0/758408691 conn(0x555a133e6d20 msgr2=0x555a133f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:07.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 -- 172.21.15.31:0/758408691 shutdown_connections 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 -- 172.21.15.31:0/758408691 wait complete. 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.987+0000 7febee89cb80 1 -- start start 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 Processor -- start 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a134024e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133ff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a13400280 con 0x555a133f0d10 2023-12-14T18:35:07.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a134003f0 con 0x555a133fbbd0 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a134005b0 con 0x555a133f0400 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febee89cb80 10 client.0 _ll_register_callbacks cb 0x7febd0009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133ff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133ff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60868/0 (socket says 172.21.15.31:60868) 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a134024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 -- 172.21.15.31:0/3877876830 learned_addr learned my addr 172.21.15.31:0/3877876830 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:07.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febe0cab700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 -- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 msgr2=0x555a134024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a134024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 -- 172.21.15.31:0/3877876830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 msgr2=0x555a133ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.988+0000 7febdb7fe700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133ffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febdb7fe700 1 -- 172.21.15.31:0/3877876830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc10f090 con 0x555a133f0d10 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febdbfff700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a134024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febe0cab700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:35:07.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febdb7fe700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133ff800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7febd0004ad0 tx=0x7febd0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be2a778ab38c93d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febbb7fe700 1 -- 172.21.15.31:0/3877876830 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0005d50 con 0x555a133f0d10 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febbb7fe700 1 -- 172.21.15.31:0/3877876830 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7febd0000e40 con 0x555a133f0d10 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febd0004510 con 0x555a133f0d10 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febbb7fe700 1 -- 172.21.15.31:0/3877876830 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd0011020 con 0x555a133f0d10 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 10 client.17205 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555a13170be0 con 0x555a133f0d10 2023-12-14T18:35:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febbb7fe700 1 -- 172.21.15.31:0/3877876830 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd0004510 con 0x555a133f0d10 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febbb7fe700 1 -- 172.21.15.31:0/3877876830 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7febd003d070 con 0x555a133f0d10 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 10 client.17205 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 20 client.17205 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.989+0000 7febee89cb80 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a13461560 0x555a13463970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.990+0000 7febee89cb80 4 client.17205 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.990+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555a1333ba80 con 0x555a13461560 2023-12-14T18:35:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.990+0000 7febdbfff700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a13461560 0x555a13463970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.990+0000 7febdbfff700 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a13461560 0x555a13463970 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb8d99804cb72e4 server_cookie=b74dc96f4a9311f2 in_seq=0 out_seq=0 2023-12-14T18:35:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.990+0000 7febbb7fe700 10 client.17205 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:08.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:07 smithi176 ceph-mon[43506]: pgmap v224: 129 pgs: 129 active+clean; 543 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:35:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.991+0000 7febbb7fe700 1 -- 172.21.15.31:0/3877876830 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555a1333ba80 con 0x555a13461560 2023-12-14T18:35:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.991+0000 7febbb7fe700 10 client.17205 handle_command_reply: tid=0 2023-12-14T18:35:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.991+0000 7febee89cb80 1 client.17205 shutdown 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.991+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 msgr2=0x555a133ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.991+0000 7febee89cb80 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133ff800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7febd0004ad0 tx=0x7febd0009750 comp rx=0 tx=0).stop 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 shutdown_connections 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x555a13461560 0x555a13463970 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555a133fbbd0 0x555a133ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555a133f0400 0x555a134024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 --2- 172.21.15.31:0/3877876830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555a133f0d10 0x555a133ff800 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febbb7fe700 0 client.17205 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:08.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 >> 172.21.15.31:0/3877876830 conn(0x555a133e6d20 msgr2=0x555a133f6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 shutdown_connections 2023-12-14T18:35:08.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 1 -- 172.21.15.31:0/3877876830 wait complete. 2023-12-14T18:35:08.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febee89cb80 20 client.17205 trim_cache size 0 max 16384 2023-12-14T18:35:08.004 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:35:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:35:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "25bda52a-cbd4-4eb8-88a9-2f461fb8eb81", 2023-12-14T18:35:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:35:08.005 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:35:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7febc40a12c0 msgr2=0x7febc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7febc40a12c0 0x7febc40a3770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7febcc009990 tx=0x7febcc005fb0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 msgr2=0x7febdc1a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 0x7febdc1a2280 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7febd4007ee0 tx=0x7febd400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 shutdown_connections 2023-12-14T18:35:08.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febdc104410 0x7febdc1a2280 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febdc0ff420 0x7febdc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7febc40a12c0 0x7febc40a3770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 --2- 172.21.15.31:0/2885636053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febdc0ffdc0 0x7febdc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.992+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 >> 172.21.15.31:0/2885636053 conn(0x7febdc075fc0 msgr2=0x7febdc0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.993+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 shutdown_connections 2023-12-14T18:35:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:07.993+0000 7febe270e700 1 -- 172.21.15.31:0/2885636053 wait complete. 2023-12-14T18:35:08.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:35:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/2675398580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069320 msgr2=0x7f0e740696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/2675398580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069320 0x7f0e740696f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f0e640099c0 tx=0x7f0e64009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/2675398580 shutdown_connections 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/2675398580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e74105520 0x7f0e7410d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/2675398580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069320 0x7f0e740696f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/2675398580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74069cc0 0x7f0e74104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/2675398580 >> 172.21.15.31:0/2675398580 conn(0x7f0e74075fc0 msgr2=0x7f0e740763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/2675398580 shutdown_connections 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.189+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/2675398580 wait complete. 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 Processor -- start 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 -- start start 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74069320 0x7f0e74196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 0x7f0e74196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e74105520 0x7f0e7419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7419b420 con 0x7f0e74069cc0 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7419b590 con 0x7f0e74105520 2023-12-14T18:35:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7b1fd700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7419b750 con 0x7f0e74069320 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 0x7f0e74196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 0x7f0e74196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60894/0 (socket says 172.21.15.31:60894) 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e73fff700 1 -- 172.21.15.31:0/969657267 learned_addr learned my addr 172.21.15.31:0/969657267 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e78f99700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74069320 0x7f0e74196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.190+0000 7f0e7979a700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e74105520 0x7f0e7419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e73fff700 1 -- 172.21.15.31:0/969657267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74069320 msgr2=0x7f0e74196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e73fff700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74069320 0x7f0e74196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e73fff700 1 -- 172.21.15.31:0/969657267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e74105520 msgr2=0x7f0e7419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e73fff700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e74105520 0x7f0e7419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e73fff700 1 -- 172.21.15.31:0/969657267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640095f0 con 0x7f0e74069cc0 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e73fff700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 0x7f0e74196980 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d950 tx=0x7f0e5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e71ffb700 1 -- 172.21.15.31:0/969657267 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00f890 con 0x7f0e74069cc0 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e741102e0 con 0x7f0e74069cc0 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e71ffb700 1 -- 172.21.15.31:0/969657267 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e5c010460 con 0x7f0e74069cc0 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e71ffb700 1 -- 172.21.15.31:0/969657267 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00e610 con 0x7f0e74069cc0 2023-12-14T18:35:08.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.191+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e74110150 con 0x7f0e74069cc0 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.194+0000 7f0e71ffb700 1 -- 172.21.15.31:0/969657267 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0e5c010860 con 0x7f0e74069cc0 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.194+0000 7f0e71ffb700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e600a12c0 0x7f0e600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.194+0000 7f0e71ffb700 1 -- 172.21.15.31:0/969657267 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f0e5c0ccc20 con 0x7f0e74069cc0 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.194+0000 7f0e78f99700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e600a12c0 0x7f0e600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.195+0000 7f0e78f99700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e600a12c0 0x7f0e600a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009990 tx=0x7f0e64003680 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.195+0000 7f0e8738bb80 1 Processor -- start 2023-12-14T18:35:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.195+0000 7f0e8738bb80 1 -- start start 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 0x55ca39187750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca39192610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39194f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca390715e0 con 0x55ca39187380 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca390e0870 con 0x55ca39187c90 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca390d9d00 con 0x55ca39192b50 2023-12-14T18:35:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 0x55ca39187750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 0x55ca39187750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60902/0 (socket says 172.21.15.31:60902) 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 -- 172.21.15.31:0/1873521855 learned_addr learned my addr 172.21.15.31:0/1873521855 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e73fff700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca39192610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 -- 172.21.15.31:0/1873521855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 msgr2=0x55ca39194f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e7979a700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39194f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 -- 172.21.15.31:0/1873521855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 msgr2=0x55ca39192610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca39192610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 -- 172.21.15.31:0/1873521855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca38f5b3b0 con 0x55ca39187380 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e73fff700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca39192610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e7979a700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39194f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e78f99700 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 0x55ca39187750 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e640099c0 tx=0x7f0e64020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5416283ca4fff3d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e6affd700 1 -- 172.21.15.31:0/1873521855 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6400bdf0 con 0x55ca39187380 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e6affd700 1 -- 172.21.15.31:0/1873521855 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6401e040 con 0x55ca39187380 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.196+0000 7f0e6affd700 1 -- 172.21.15.31:0/1873521855 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e64004760 con 0x55ca39187380 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1873521855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 msgr2=0x55ca39187750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 0x55ca39187750 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0e640099c0 tx=0x7f0e64020000 comp rx=0 tx=0).stop 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1873521855 shutdown_connections 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39194f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187380 0x55ca39187750 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1873521855 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca39192610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1873521855 >> 172.21.15.31:0/1873521855 conn(0x55ca3917dca0 msgr2=0x55ca391837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1873521855 shutdown_connections 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.197+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1873521855 wait complete. 2023-12-14T18:35:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.198+0000 7f0e8738bb80 1 -- start start 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.198+0000 7f0e8738bb80 1 Processor -- start 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca3918f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 0x55ca3918fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca39075f90 con 0x55ca39192b50 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca39075c30 con 0x55ca39187c90 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca38f5d6a0 con 0x55ca39187380 2023-12-14T18:35:08.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e8738bb80 10 client.0 _ll_register_callbacks cb 0x55ca39099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 0x55ca3918fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 0x55ca3918fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60918/0 (socket says 172.21.15.31:60918) 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 -- 172.21.15.31:0/3430549450 learned_addr learned my addr 172.21.15.31:0/3430549450 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e78f99700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e73fff700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca3918f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 -- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 msgr2=0x55ca391924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 -- 172.21.15.31:0/3430549450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 msgr2=0x55ca3918f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca3918f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 -- 172.21.15.31:0/3430549450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7410d930 con 0x55ca39192b50 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e73fff700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca3918f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e7979a700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 0x55ca3918fce0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c004040 tx=0x7f0e6c004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4c1e939fa91223 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.199+0000 7f0e53fff700 1 -- 172.21.15.31:0/3430549450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c007710 con 0x55ca39192b50 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e53fff700 1 -- 172.21.15.31:0/3430549450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f0e6c007870 con 0x55ca39192b50 2023-12-14T18:35:08.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca3917b2b0 con 0x55ca39192b50 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e53fff700 1 -- 172.21.15.31:0/3430549450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e6c007a30 con 0x55ca39192b50 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 10 client.17217 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ca390a0170 con 0x55ca39192b50 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e53fff700 1 -- 172.21.15.31:0/3430549450 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca3917b2b0 con 0x55ca39192b50 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e53fff700 1 -- 172.21.15.31:0/3430549450 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0e6c014070 con 0x55ca39192b50 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 10 client.17217 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 10 client.17217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 20 client.17217 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391a2500 0x55ca391a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 4 client.17217 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca390d2ce0 con 0x55ca391a2500 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.200+0000 7f0e78f99700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391a2500 0x55ca391a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.201+0000 7f0e78f99700 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391a2500 0x55ca391a4910 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297503635c8f9316 server_cookie=c05b3cb64a740468 in_seq=0 out_seq=0 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.201+0000 7f0e53fff700 10 client.17217 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.201+0000 7f0e53fff700 1 -- 172.21.15.31:0/3430549450 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca390d2ce0 con 0x55ca391a2500 2023-12-14T18:35:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.201+0000 7f0e53fff700 10 client.17217 handle_command_reply: tid=0 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 client.17217 shutdown 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 msgr2=0x55ca3918fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 0x55ca3918fce0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c004040 tx=0x7f0e6c004350 comp rx=0 tx=0).stop 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 shutdown_connections 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391a2500 0x55ca391a4910 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39192b50 0x55ca3918fce0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e53fff700 0 client.17217 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/3430549450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187c90 0x55ca3918f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 >> 172.21.15.31:0/3430549450 conn(0x55ca3917dca0 msgr2=0x55ca391833d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 shutdown_connections 2023-12-14T18:35:08.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 1 -- 172.21.15.31:0/3430549450 wait complete. 2023-12-14T18:35:08.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.202+0000 7f0e8738bb80 20 client.17217 trim_cache size 0 max 16384 2023-12-14T18:35:08.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 Processor -- start 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 -- start start 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187380 0x55ca39199620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca3918ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39191f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3918dce0 con 0x55ca39187c90 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca391825b0 con 0x55ca39187380 2023-12-14T18:35:08.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.220+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3918b5f0 con 0x55ca39192b50 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca3918ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca3918ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60920/0 (socket says 172.21.15.31:60920) 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 -- 172.21.15.31:0/871835770 learned_addr learned my addr 172.21.15.31:0/871835770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e7979a700 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39191f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e78f99700 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187380 0x55ca39199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 -- 172.21.15.31:0/871835770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 msgr2=0x55ca39191f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39191f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 -- 172.21.15.31:0/871835770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187380 msgr2=0x55ca39199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187380 0x55ca39199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 -- 172.21.15.31:0/871835770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca3909fec0 con 0x55ca39187c90 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e78f99700 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187380 0x55ca39199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e73fff700 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca3918ef40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c099830 tx=0x7f0e5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c0a5b6267cb6bc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e6b7fe700 1 -- 172.21.15.31:0/871835770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c09d8d0 con 0x55ca39187c90 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e6b7fe700 1 -- 172.21.15.31:0/871835770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e5c0a1cf0 con 0x55ca39187c90 2023-12-14T18:35:08.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.221+0000 7f0e6b7fe700 1 -- 172.21.15.31:0/871835770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c098ff0 con 0x55ca39187c90 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 -- 172.21.15.31:0/871835770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 msgr2=0x55ca3918ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca3918ef40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c099830 tx=0x7f0e5c000bc0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 -- 172.21.15.31:0/871835770 shutdown_connections 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39192b50 0x55ca39191f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39187380 0x55ca39199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/871835770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca3918ef40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 -- 172.21.15.31:0/871835770 >> 172.21.15.31:0/871835770 conn(0x55ca3917dca0 msgr2=0x55ca3919ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 -- 172.21.15.31:0/871835770 shutdown_connections 2023-12-14T18:35:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 -- 172.21.15.31:0/871835770 wait complete. 2023-12-14T18:35:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 -- start start 2023-12-14T18:35:08.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.222+0000 7f0e8738bb80 1 Processor -- start 2023-12-14T18:35:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca39195a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39192b50 0x55ca39195f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca391965f0 con 0x55ca39187c90 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca39196760 con 0x55ca39192b50 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca39196920 con 0x55ca39187380 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e8738bb80 10 client.0 _ll_register_callbacks cb 0x7f0e6c0098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca39195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca39195a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60936/0 (socket says 172.21.15.31:60936) 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 -- 172.21.15.31:0/1294433639 learned_addr learned my addr 172.21.15.31:0/1294433639 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:08.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e7979a700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39192b50 0x55ca39195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e78f99700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 -- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 msgr2=0x55ca391995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 -- 172.21.15.31:0/1294433639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39192b50 msgr2=0x55ca39195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39192b50 0x55ca39195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.223+0000 7f0e73fff700 1 -- 172.21.15.31:0/1294433639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e74110150 con 0x55ca39187c90 2023-12-14T18:35:08.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e73fff700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca39195a50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00dfd0 tx=0x7f0e5c0c9f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0dad9a51cba228 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e53fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e53fff700 1 -- 172.21.15.31:0/1294433639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c098f90 con 0x55ca39187c90 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e53fff700 1 -- 172.21.15.31:0/1294433639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f0e5c099b00 con 0x55ca39187c90 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e6401e040 con 0x55ca39187c90 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e53fff700 1 -- 172.21.15.31:0/1294433639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e5c099cc0 con 0x55ca39187c90 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 10 client.17223 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ca390869a0 con 0x55ca39187c90 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e53fff700 1 -- 172.21.15.31:0/1294433639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e6401e040 con 0x55ca39187c90 2023-12-14T18:35:08.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e53fff700 1 -- 172.21.15.31:0/1294433639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0e5c0b9240 con 0x55ca39187c90 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 10 client.17223 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 20 client.17223 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391f7550 0x55ca391f9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 4 client.17223 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.224+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca390d2a00 con 0x55ca391f7550 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e78f99700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391f7550 0x55ca391f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:08.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e53fff700 10 client.17223 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e78f99700 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391f7550 0x55ca391f9960 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bd0c9512189737 server_cookie=7529923ea3dc01fd in_seq=0 out_seq=0 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e53fff700 1 -- 172.21.15.31:0/1294433639 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ca390d2a00 con 0x55ca391f7550 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e53fff700 10 client.17223 handle_command_reply: tid=0 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 client.17223 shutdown 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 msgr2=0x55ca39195a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca39195a50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00dfd0 tx=0x7f0e5c0c9f40 comp rx=0 tx=0).stop 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 shutdown_connections 2023-12-14T18:35:08.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55ca391f7550 0x55ca391f9960 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55ca39192b50 0x55ca39195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca39187380 0x55ca391995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e53fff700 0 client.17223 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 --2- 172.21.15.31:0/1294433639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55ca39187c90 0x55ca39195a50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.225+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 >> 172.21.15.31:0/1294433639 conn(0x55ca3917dca0 msgr2=0x55ca391892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 shutdown_connections 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e8738bb80 1 -- 172.21.15.31:0/1294433639 wait complete. 2023-12-14T18:35:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e8738bb80 20 client.17223 trim_cache size 0 max 16384 2023-12-14T18:35:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:35:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:35:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:35:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: "25bda52a-cbd4-4eb8-88a9-2f461fb8eb81": { 2023-12-14T18:35:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:35:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "25bda52a-cbd4-4eb8-88a9-2f461fb8eb81", 2023-12-14T18:35:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:35:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:35:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:35:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:35:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e600a12c0 msgr2=0x7f0e600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e600a12c0 0x7f0e600a3770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009990 tx=0x7f0e64003680 comp rx=0 tx=0).stop 2023-12-14T18:35:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 msgr2=0x7f0e74196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 0x7f0e74196980 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00d950 tx=0x7f0e5c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:35:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 shutdown_connections 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e74105520 0x7f0e7419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f0e600a12c0 0x7f0e600a3770 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e74069320 0x7f0e74196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 --2- 172.21.15.31:0/969657267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e74069cc0 0x7f0e74196980 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 >> 172.21.15.31:0/969657267 conn(0x7f0e74075fc0 msgr2=0x7f0e74103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 shutdown_connections 2023-12-14T18:35:08.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:08.226+0000 7f0e7b1fd700 1 -- 172.21.15.31:0/969657267 wait complete. 2023-12-14T18:35:08.242 INFO:tasks.cephfs.filesystem:scrub status for tag:25bda52a-cbd4-4eb8-88a9-2f461fb8eb81 - {'path': '/', 'tag': '25bda52a-cbd4-4eb8-88a9-2f461fb8eb81', 'options': 'recursive,force'} 2023-12-14T18:35:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:08 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:35:08.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:08 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:08 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:35:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:08 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:08 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:35:08.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:08 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:09 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:35:09.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:09 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:09 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:35:09.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:09 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:09 smithi204 ceph-mon[43866]: pgmap v225: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 23 op/s 2023-12-14T18:35:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:09 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:35:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:09 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:09 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:35:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:09 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:09 smithi031 ceph-mon[42085]: pgmap v225: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 23 op/s 2023-12-14T18:35:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:09 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:35:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:09 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:09 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:35:09.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:09 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:09.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:09 smithi176 ceph-mon[43506]: pgmap v225: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 50 KiB/s wr, 23 op/s 2023-12-14T18:35:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:10 smithi204 ceph-mon[43866]: osdmap e93: 12 total, 12 up, 12 in 2023-12-14T18:35:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:10 smithi031 ceph-mon[42085]: osdmap e93: 12 total, 12 up, 12 in 2023-12-14T18:35:10.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:10 smithi176 ceph-mon[43506]: osdmap e93: 12 total, 12 up, 12 in 2023-12-14T18:35:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:11 smithi204 ceph-mon[43866]: pgmap v227: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 25 op/s 2023-12-14T18:35:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:11 smithi031 ceph-mon[42085]: pgmap v227: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 25 op/s 2023-12-14T18:35:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:11 smithi176 ceph-mon[43506]: pgmap v227: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 25 op/s 2023-12-14T18:35:13.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:13 smithi204 ceph-mon[43866]: pgmap v228: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 25 op/s 2023-12-14T18:35:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:13 smithi031 ceph-mon[42085]: pgmap v228: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 25 op/s 2023-12-14T18:35:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:13 smithi176 ceph-mon[43506]: pgmap v228: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 60 KiB/s wr, 25 op/s 2023-12-14T18:35:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:14 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:35:14.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:14 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:35:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:14 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:35:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:14 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:35:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:14 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:35:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:14 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:35:15.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:15 smithi204 ceph-mon[43866]: pgmap v229: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-12-14T18:35:15.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:15 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:35:15.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:15 smithi031 ceph-mon[42085]: pgmap v229: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-12-14T18:35:15.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:15 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:35:15.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:15 smithi176 ceph-mon[43506]: pgmap v229: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-12-14T18:35:15.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:15 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:35:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:35:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:35:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:35:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:35:16.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:35:16.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:35:16.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:16 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:35:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:35:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:35:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:35:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:35:16.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:35:16.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:35:16.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:16 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:35:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:35:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:35:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:35:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:35:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:35:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:35:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:16 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:35:17.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:17 smithi031 ceph-mon[42085]: pgmap v230: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-12-14T18:35:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:17 smithi176 ceph-mon[43506]: pgmap v230: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-12-14T18:35:18.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:17 smithi204 ceph-mon[43866]: pgmap v230: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2023-12-14T18:35:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:19 smithi176 ceph-mon[43506]: pgmap v231: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:19 smithi176 ceph-mon[43506]: osdmap e94: 12 total, 12 up, 12 in 2023-12-14T18:35:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:19 smithi204 ceph-mon[43866]: pgmap v231: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:19 smithi204 ceph-mon[43866]: osdmap e94: 12 total, 12 up, 12 in 2023-12-14T18:35:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:19 smithi031 ceph-mon[42085]: pgmap v231: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:19 smithi031 ceph-mon[42085]: osdmap e94: 12 total, 12 up, 12 in 2023-12-14T18:35:21.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:21 smithi176 ceph-mon[43506]: pgmap v233: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:21 smithi204 ceph-mon[43866]: pgmap v233: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:21 smithi031 ceph-mon[42085]: pgmap v233: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:23.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:23 smithi176 ceph-mon[43506]: pgmap v234: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:23 smithi204 ceph-mon[43866]: pgmap v234: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:23.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:23 smithi031 ceph-mon[42085]: pgmap v234: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:25.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:25 smithi176 ceph-mon[43506]: pgmap v235: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:25.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:25 smithi204 ceph-mon[43866]: pgmap v235: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:25.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:25 smithi031 ceph-mon[42085]: pgmap v235: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:27.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:27 smithi176 ceph-mon[43506]: pgmap v236: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:27 smithi204 ceph-mon[43866]: pgmap v236: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:27 smithi031 ceph-mon[42085]: pgmap v236: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:29.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:29 smithi176 ceph-mon[43506]: pgmap v237: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:29 smithi204 ceph-mon[43866]: pgmap v237: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:29.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:29 smithi031 ceph-mon[42085]: pgmap v237: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:31.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:31 smithi204 ceph-mon[43866]: pgmap v238: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:35:31.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:31 smithi031 ceph-mon[42085]: pgmap v238: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:35:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:31 smithi176 ceph-mon[43506]: pgmap v238: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:35:33.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:33 smithi204 ceph-mon[43866]: pgmap v239: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:33.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:33 smithi031 ceph-mon[42085]: pgmap v239: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:33.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:33 smithi176 ceph-mon[43506]: pgmap v239: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:35 smithi204 ceph-mon[43866]: pgmap v240: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:35.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:35 smithi031 ceph-mon[42085]: pgmap v240: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:35.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:35 smithi176 ceph-mon[43506]: pgmap v240: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:37 smithi204 ceph-mon[43866]: pgmap v241: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:37.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:37 smithi031 ceph-mon[42085]: pgmap v241: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:37 smithi176 ceph-mon[43506]: pgmap v241: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:38.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:35:38.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 -- 172.21.15.31:0/1388783935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ff390 msgr2=0x7f55200ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 --2- 172.21.15.31:0/1388783935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ff390 0x7f55200ff760 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f55100099c0 tx=0x7f5510009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 -- 172.21.15.31:0/1388783935 shutdown_connections 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 --2- 172.21.15.31:0/1388783935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55201043d0 0x7f552010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 --2- 172.21.15.31:0/1388783935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ff390 0x7f55200ff760 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 --2- 172.21.15.31:0/1388783935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55200ffd30 0x7f5520103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 -- 172.21.15.31:0/1388783935 >> 172.21.15.31:0/1388783935 conn(0x7f5520075fd0 msgr2=0x7f55200763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 -- 172.21.15.31:0/1388783935 shutdown_connections 2023-12-14T18:35:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 -- 172.21.15.31:0/1388783935 wait complete. 2023-12-14T18:35:38.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.391+0000 7f5525d45700 1 Processor -- start 2023-12-14T18:35:38.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 -- start start 2023-12-14T18:35:38.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200ff390 0x7f5520195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 0x7f5520195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55201043d0 0x7f5520199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552019a2f0 con 0x7f55200ffd30 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552019a460 con 0x7f55201043d0 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f5525d45700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552019a620 con 0x7f55200ff390 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 0x7f5520195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 0x7f5520195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49938/0 (socket says 172.21.15.31:49938) 2023-12-14T18:35:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 -- 172.21.15.31:0/282027215 learned_addr learned my addr 172.21.15.31:0/282027215 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551f7fe700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200ff390 0x7f5520195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551ffff700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55201043d0 0x7f5520199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 -- 172.21.15.31:0/282027215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200ff390 msgr2=0x7f5520195310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200ff390 0x7f5520195310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 -- 172.21.15.31:0/282027215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55201043d0 msgr2=0x7f5520199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55201043d0 0x7f5520199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 -- 172.21.15.31:0/282027215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100095f0 con 0x7f55200ffd30 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551ffff700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55201043d0 0x7f5520199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.392+0000 7f551effd700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 0x7f5520195850 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f551400c9b0 tx=0x7f551400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.393+0000 7f551cff9700 1 -- 172.21.15.31:0/282027215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514007aa0 con 0x7f55200ffd30 2023-12-14T18:35:38.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.393+0000 7f551cff9700 1 -- 172.21.15.31:0/282027215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5514007c00 con 0x7f55200ffd30 2023-12-14T18:35:38.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.393+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552010f1e0 con 0x7f55200ffd30 2023-12-14T18:35:38.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.393+0000 7f551cff9700 1 -- 172.21.15.31:0/282027215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514018660 con 0x7f55200ffd30 2023-12-14T18:35:38.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.393+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552010f050 con 0x7f55200ffd30 2023-12-14T18:35:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.395+0000 7f551cff9700 1 -- 172.21.15.31:0/282027215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f551400f450 con 0x7f55200ffd30 2023-12-14T18:35:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.396+0000 7f551cff9700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55080a1340 0x7f55080a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.396+0000 7f551f7fe700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55080a1340 0x7f55080a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.396+0000 7f551cff9700 1 -- 172.21.15.31:0/282027215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f55140ccf70 con 0x7f55200ffd30 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.396+0000 7f551f7fe700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55080a1340 0x7f55080a37f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5510006010 tx=0x7f5510009f10 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.396+0000 7f5531ed3b80 1 Processor -- start 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 -- start start 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafb8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafbaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fae975e0 con 0x5623fafb8b50 2023-12-14T18:35:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623faf06870 con 0x5623fafadc90 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623faeffd00 con 0x5623fafad380 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafbaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafbaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49942/0 (socket says 172.21.15.31:49942) 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 -- 172.21.15.31:0/279402918 learned_addr learned my addr 172.21.15.31:0/279402918 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551f7fe700 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551effd700 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafb8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 -- 172.21.15.31:0/279402918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 msgr2=0x5623fafad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 -- 172.21.15.31:0/279402918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 msgr2=0x5623fafb8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafb8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 -- 172.21.15.31:0/279402918 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623fad813b0 con 0x5623fafb8b50 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551f7fe700 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.397+0000 7f551ffff700 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafbaf30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5518009cd0 tx=0x7f5518007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f73768b2d3e591 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.398+0000 7f5505ffb700 1 -- 172.21.15.31:0/279402918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518014600 con 0x5623fafb8b50 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.398+0000 7f5505ffb700 1 -- 172.21.15.31:0/279402918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5518007e40 con 0x5623fafb8b50 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.398+0000 7f5505ffb700 1 -- 172.21.15.31:0/279402918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518013aa0 con 0x5623fafb8b50 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.398+0000 7f5531ed3b80 1 -- 172.21.15.31:0/279402918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 msgr2=0x5623fafbaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.398+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafbaf30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5518009cd0 tx=0x7f5518007500 comp rx=0 tx=0).stop 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 -- 172.21.15.31:0/279402918 shutdown_connections 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafbaf30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/279402918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafb8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 -- 172.21.15.31:0/279402918 >> 172.21.15.31:0/279402918 conn(0x5623fafa3ca0 msgr2=0x5623fafa97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 -- 172.21.15.31:0/279402918 shutdown_connections 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 -- 172.21.15.31:0/279402918 wait complete. 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 -- start start 2023-12-14T18:35:38.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.399+0000 7f5531ed3b80 1 Processor -- start 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafc0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafc0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafb79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fae9bf90 con 0x5623fafb8b50 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fae9bc30 con 0x5623fafadc90 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fad836a0 con 0x5623fafad380 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 10 client.0 _ll_register_callbacks cb 0x5623faebf910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafb79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafb79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49944/0 (socket says 172.21.15.31:49944) 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 -- 172.21.15.31:0/2361364088 learned_addr learned my addr 172.21.15.31:0/2361364088 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551effd700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafc0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551f7fe700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafc0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 -- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 msgr2=0x5623fafc0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafc0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 -- 172.21.15.31:0/2361364088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 msgr2=0x5623fafc0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafc0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 -- 172.21.15.31:0/2361364088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5518009900 con 0x5623fafb8b50 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f551ffff700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafb79d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5518017040 tx=0x7f5518004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31204344b5c034fd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f54feffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f54feffd700 1 -- 172.21.15.31:0/2361364088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518013bd0 con 0x5623fafb8b50 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623fafa12b0 con 0x5623fafb8b50 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.400+0000 7f54feffd700 1 -- 172.21.15.31:0/2361364088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f5518013d30 con 0x5623fafb8b50 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f54feffd700 1 -- 172.21.15.31:0/2361364088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5518005040 con 0x5623fafb8b50 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 10 client.17235 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:38.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5623faec6170 con 0x5623fafb8b50 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f54feffd700 1 -- 172.21.15.31:0/2361364088 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623fafa12b0 con 0x5623fafb8b50 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f54feffd700 1 -- 172.21.15.31:0/2361364088 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5518014660 con 0x5623fafb8b50 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 20 client.17235 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fafca5c0 0x5623fafcc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 4 client.17235 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.401+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623faef8ce0 con 0x5623fafca5c0 2023-12-14T18:35:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.402+0000 7f551f7fe700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fafca5c0 0x5623fafcc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.402+0000 7f551f7fe700 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fafca5c0 0x5623fafcc9d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c225f5cb8bc386f6 server_cookie=437ef26d78c167fc in_seq=0 out_seq=0 2023-12-14T18:35:38.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.402+0000 7f54feffd700 10 client.17235 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f54feffd700 1 -- 172.21.15.31:0/2361364088 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5623faef8ce0 con 0x5623fafca5c0 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f54feffd700 10 client.17235 handle_command_reply: tid=0 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 client.17235 shutdown 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 msgr2=0x5623fafb79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafb79d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5518017040 tx=0x7f5518004f70 comp rx=0 tx=0).stop 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 shutdown_connections 2023-12-14T18:35:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fafca5c0 0x5623fafcc9d0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafb8b50 0x5623fafb79d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafad380 0x5623fafc0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f54feffd700 0 client.17235 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/2361364088 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafadc90 0x5623fafc0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 >> 172.21.15.31:0/2361364088 conn(0x5623fafa3ca0 msgr2=0x5623fafa93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 shutdown_connections 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 1 -- 172.21.15.31:0/2361364088 wait complete. 2023-12-14T18:35:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.403+0000 7f5531ed3b80 20 client.17235 trim_cache size 0 max 16384 2023-12-14T18:35:38.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 Processor -- start 2023-12-14T18:35:38.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 -- start start 2023-12-14T18:35:38.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 0x5623fafcc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafadc90 0x5623fafd1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafb8b50 0x5623fafb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fafb1650 con 0x5623fafad380 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fafaed30 con 0x5623fafb8b50 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fafd1870 con 0x5623fafadc90 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f551f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 0x5623fafcc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f551f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 0x5623fafcc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49960/0 (socket says 172.21.15.31:49960) 2023-12-14T18:35:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f551f7fe700 1 -- 172.21.15.31:0/3062158259 learned_addr learned my addr 172.21.15.31:0/3062158259 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f551effd700 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafadc90 0x5623fafd1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.421+0000 7f551ffff700 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafb8b50 0x5623fafb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551f7fe700 1 -- 172.21.15.31:0/3062158259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafadc90 msgr2=0x5623fafd1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafadc90 0x5623fafd1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551f7fe700 1 -- 172.21.15.31:0/3062158259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafb8b50 msgr2=0x5623fafb2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafb8b50 0x5623fafb2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551f7fe700 1 -- 172.21.15.31:0/3062158259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623faec5ec0 con 0x5623fafad380 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551ffff700 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafb8b50 0x5623fafb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:38.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 0x5623fafcc9a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f55100099c0 tx=0x7f5510020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699da196b1426d95 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f55067fc700 1 -- 172.21.15.31:0/3062158259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000be50 con 0x5623fafad380 2023-12-14T18:35:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f55067fc700 1 -- 172.21.15.31:0/3062158259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f551001e040 con 0x5623fafad380 2023-12-14T18:35:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.422+0000 7f55067fc700 1 -- 172.21.15.31:0/3062158259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510034570 con 0x5623fafad380 2023-12-14T18:35:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3062158259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 msgr2=0x5623fafcc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 0x5623fafcc9a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f55100099c0 tx=0x7f5510020000 comp rx=0 tx=0).stop 2023-12-14T18:35:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3062158259 shutdown_connections 2023-12-14T18:35:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafb8b50 0x5623fafb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafad380 0x5623fafcc9a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3062158259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafadc90 0x5623fafd1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3062158259 >> 172.21.15.31:0/3062158259 conn(0x5623fafa3ca0 msgr2=0x5623fafb7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3062158259 shutdown_connections 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3062158259 wait complete. 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 -- start start 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.423+0000 7f5531ed3b80 1 Processor -- start 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafad380 0x5623fafbe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 0x5623fafbe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafb8b50 0x5623fb01f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fb01fdc0 con 0x5623fafadc90 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fb01ff00 con 0x5623fafad380 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623fb020070 con 0x5623fafb8b50 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f5531ed3b80 10 client.0 _ll_register_callbacks cb 0x7f5518009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 0x5623fafbe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 0x5623fafbe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49962/0 (socket says 172.21.15.31:49962) 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 -- 172.21.15.31:0/3419263118 learned_addr learned my addr 172.21.15.31:0/3419263118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafad380 0x5623fafbe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551ffff700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafb8b50 0x5623fb01f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 -- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafb8b50 msgr2=0x5623fb01f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafb8b50 0x5623fb01f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 -- 172.21.15.31:0/3419263118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafad380 msgr2=0x5623fafbe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafad380 0x5623fafbe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 -- 172.21.15.31:0/3419263118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551002c040 con 0x5623fafadc90 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafad380 0x5623fafbe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f551effd700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 0x5623fafbe800 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f551400ffd0 tx=0x7f551400a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99550270d2d4f07d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f54feffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f54feffd700 1 -- 172.21.15.31:0/3419263118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400f700 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.424+0000 7f54feffd700 1 -- 172.21.15.31:0/3419263118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f551400f860 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5518007e40 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f54feffd700 1 -- 172.21.15.31:0/3419263118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5514099eb0 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 10 client.17241 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5623faeac9a0 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f54feffd700 1 -- 172.21.15.31:0/3419263118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5518007e40 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f54feffd700 1 -- 172.21.15.31:0/3419263118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f55140a1c10 con 0x5623fafadc90 2023-12-14T18:35:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 10 client.17241 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 20 client.17241 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fb021850 0x5623fb023c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 4 client.17241 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5623faef8a00 con 0x5623fb021850 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.425+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fb021850 0x5623fb023c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f54feffd700 10 client.17241 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f551f7fe700 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fb021850 0x5623fb023c60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635d8ddd18a4c70d server_cookie=6f11f943cdc193d5 in_seq=0 out_seq=0 2023-12-14T18:35:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f54feffd700 1 -- 172.21.15.31:0/3419263118 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5623faef8a00 con 0x5623fb021850 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f54feffd700 10 client.17241 handle_command_reply: tid=0 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 client.17241 shutdown 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 msgr2=0x5623fafbe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 0x5623fafbe800 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f551400ffd0 tx=0x7f551400a4b0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 shutdown_connections 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5623fb021850 0x5623fb023c60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623fafb8b50 0x5623fb01f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5623fafad380 0x5623fafbe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f54feffd700 0 client.17241 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 --2- 172.21.15.31:0/3419263118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5623fafadc90 0x5623fafbe800 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 >> 172.21.15.31:0/3419263118 conn(0x5623fafa3ca0 msgr2=0x5623fafb3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 shutdown_connections 2023-12-14T18:35:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 1 -- 172.21.15.31:0/3419263118 wait complete. 2023-12-14T18:35:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.426+0000 7f5531ed3b80 20 client.17241 trim_cache size 0 max 16384 2023-12-14T18:35:38.438 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:35:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:35:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:35:38.439 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:35:38.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55080a1340 msgr2=0x7f55080a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55080a1340 0x7f55080a37f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5510006010 tx=0x7f5510009f10 comp rx=0 tx=0).stop 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 msgr2=0x7f5520195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 0x7f5520195850 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f551400c9b0 tx=0x7f551400ccc0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 shutdown_connections 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f55201043d0 0x7f5520199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f55080a1340 0x7f55080a37f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55200ff390 0x7f5520195310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 --2- 172.21.15.31:0/282027215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f55200ffd30 0x7f5520195850 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 >> 172.21.15.31:0/282027215 conn(0x7f5520075fd0 msgr2=0x7f55200fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 shutdown_connections 2023-12-14T18:35:38.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.427+0000 7f5525d45700 1 -- 172.21.15.31:0/282027215 wait complete. 2023-12-14T18:35:38.443 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:35:38.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:35:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 -- 172.21.15.31:0/1457766680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda04101280 msgr2=0x7fda04101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 --2- 172.21.15.31:0/1457766680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda04101280 0x7fda04101650 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009990 tx=0x7fd9f4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 -- 172.21.15.31:0/1457766680 shutdown_connections 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 --2- 172.21.15.31:0/1457766680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda0410a7a0 0x7fda0410cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 --2- 172.21.15.31:0/1457766680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda04101280 0x7fda04101650 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 --2- 172.21.15.31:0/1457766680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 0x7fda0410a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 -- 172.21.15.31:0/1457766680 >> 172.21.15.31:0/1457766680 conn(0x7fda040761b0 msgr2=0x7fda040765b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 -- 172.21.15.31:0/1457766680 shutdown_connections 2023-12-14T18:35:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.579+0000 7fda08aec700 1 -- 172.21.15.31:0/1457766680 wait complete. 2023-12-14T18:35:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 Processor -- start 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 -- start start 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda04101280 0x7fda04196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 0x7fda04196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 0x7fda0419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0419b330 con 0x7fda0410a7a0 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0419b4a0 con 0x7fda04101280 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda08aec700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0419b660 con 0x7fda04101b90 2023-12-14T18:35:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 0x7fda0419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 0x7fda0419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49976/0 (socket says 172.21.15.31:49976) 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 -- 172.21.15.31:0/747559232 learned_addr learned my addr 172.21.15.31:0/747559232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda0259c700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda04101280 0x7fda04196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda01d9b700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 0x7fda04196890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 -- 172.21.15.31:0/747559232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 msgr2=0x7fda04196890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 0x7fda04196890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 -- 172.21.15.31:0/747559232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda04101280 msgr2=0x7fda04196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda04101280 0x7fda04196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda02d9d700 1 -- 172.21.15.31:0/747559232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f40095f0 con 0x7fda0410a7a0 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.580+0000 7fda01d9b700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 0x7fda04196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.581+0000 7fda02d9d700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 0x7fda0419acf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc007ee0 tx=0x7fd9fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.581+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc017070 con 0x7fda0410a7a0 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.581+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9fc00f040 con 0x7fda0410a7a0 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.581+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda04110260 con 0x7fda0410a7a0 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.581+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc005740 con 0x7fda0410a7a0 2023-12-14T18:35:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.581+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda041100d0 con 0x7fda0410a7a0 2023-12-14T18:35:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.582+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda041103f0 con 0x7fda0410a7a0 2023-12-14T18:35:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.583+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd9fc00dc60 con 0x7fda0410a7a0 2023-12-14T18:35:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.584+0000 7fd9f37fe700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9ec0a12c0 0x7fd9ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.584+0000 7fda0259c700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9ec0a12c0 0x7fd9ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.584+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd9fc0ccf30 con 0x7fda0410a7a0 2023-12-14T18:35:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.584+0000 7fda0259c700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9ec0a12c0 0x7fd9ec0a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400b2e0 tx=0x7fd9f4005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.587+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd9fc0f9730 con 0x7fda0410a7a0 2023-12-14T18:35:38.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.843+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda041050c0 con 0x7fda0410a7a0 2023-12-14T18:35:38.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.843+0000 7fd9f37fe700 1 -- 172.21.15.31:0/747559232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10822 (secure 0 0 0) 0x7fd9fc095ef0 con 0x7fda0410a7a0 2023-12-14T18:35:38.846 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:35:38.846 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:31:40.963020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:35:38.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9ec0a12c0 msgr2=0x7fd9ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9ec0a12c0 0x7fd9ec0a3770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400b2e0 tx=0x7fd9f4005fb0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 msgr2=0x7fda0419acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 0x7fda0419acf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc007ee0 tx=0x7fd9fc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 shutdown_connections 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda0410a7a0 0x7fda0419acf0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fda04101280 0x7fda04196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd9ec0a12c0 0x7fd9ec0a3770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 --2- 172.21.15.31:0/747559232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda04101b90 0x7fda04196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 >> 172.21.15.31:0/747559232 conn(0x7fda040761b0 msgr2=0x7fda04102200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 shutdown_connections 2023-12-14T18:35:38.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:38.846+0000 7fda08aec700 1 -- 172.21.15.31:0/747559232 wait complete. 2023-12-14T18:35:38.850 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-12-14T18:35:38.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:35:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.000+0000 7f713d31b700 1 -- 172.21.15.31:0/2917857725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380ff3e0 msgr2=0x7f71380ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.000+0000 7f713d31b700 1 --2- 172.21.15.31:0/2917857725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380ff3e0 0x7f71380ff7b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f71280099c0 tx=0x7f7128009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 -- 172.21.15.31:0/2917857725 shutdown_connections 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 --2- 172.21.15.31:0/2917857725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7138104420 0x7f713810bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 --2- 172.21.15.31:0/2917857725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380ff3e0 0x7f71380ff7b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 --2- 172.21.15.31:0/2917857725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71380ffd80 0x7f7138103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 -- 172.21.15.31:0/2917857725 >> 172.21.15.31:0/2917857725 conn(0x7f7138075fc0 msgr2=0x7f71380763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 -- 172.21.15.31:0/2917857725 shutdown_connections 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 -- 172.21.15.31:0/2917857725 wait complete. 2023-12-14T18:35:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.001+0000 7f713d31b700 1 Processor -- start 2023-12-14T18:35:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 -- start start 2023-12-14T18:35:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71380ff3e0 0x7f71381963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71380ffd80 0x7f7138196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 0x7f713819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713819b3a0 con 0x7f7138104420 2023-12-14T18:35:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713819b510 con 0x7f71380ff3e0 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f713d31b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713819b6d0 con 0x7f71380ffd80 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 0x7f713819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 0x7f713819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49992/0 (socket says 172.21.15.31:49992) 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 -- 172.21.15.31:0/525000697 learned_addr learned my addr 172.21.15.31:0/525000697 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f712e7fc700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71380ffd80 0x7f7138196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f7136ffd700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71380ff3e0 0x7f71381963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 -- 172.21.15.31:0/525000697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71380ffd80 msgr2=0x7f7138196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71380ffd80 0x7f7138196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 -- 172.21.15.31:0/525000697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71380ff3e0 msgr2=0x7f71381963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71380ff3e0 0x7f71381963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 -- 172.21.15.31:0/525000697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71280095f0 con 0x7f7138104420 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.002+0000 7f71377fe700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 0x7f713819ad60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7130007ee0 tx=0x7f713000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.003+0000 7f7134ff9700 1 -- 172.21.15.31:0/525000697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130017070 con 0x7f7138104420 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.003+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7138110350 con 0x7f7138104420 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.003+0000 7f7134ff9700 1 -- 172.21.15.31:0/525000697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f713000f040 con 0x7f7138104420 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.003+0000 7f7134ff9700 1 -- 172.21.15.31:0/525000697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130005740 con 0x7f7138104420 2023-12-14T18:35:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.003+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381101c0 con 0x7f7138104420 2023-12-14T18:35:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.005+0000 7f7134ff9700 1 -- 172.21.15.31:0/525000697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f713000dc60 con 0x7f7138104420 2023-12-14T18:35:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.006+0000 7f7134ff9700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f71240a12c0 0x7f71240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.006+0000 7f7136ffd700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f71240a12c0 0x7f71240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.006+0000 7f7134ff9700 1 -- 172.21.15.31:0/525000697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f71300cc8f0 con 0x7f7138104420 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.006+0000 7f7136ffd700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f71240a12c0 0x7f71240a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7128000c00 tx=0x7f7128005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.006+0000 7f71494a9b80 1 Processor -- start 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.006+0000 7f71494a9b80 1 -- start start 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe60750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe6ff60 0x5612fbe72340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71494a9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbd4a5e0 con 0x5612fbe60c90 2023-12-14T18:35:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71494a9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbdb9870 con 0x5612fbe60380 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71494a9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbdb2d00 con 0x5612fbe6ff60 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50006/0 (socket says 172.21.15.31:50006) 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 -- 172.21.15.31:0/958843158 learned_addr learned my addr 172.21.15.31:0/958843158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f7136ffd700 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe60750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f71377fe700 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe6ff60 0x5612fbe72340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 -- 172.21.15.31:0/958843158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe6ff60 msgr2=0x5612fbe72340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe6ff60 0x5612fbe72340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 -- 172.21.15.31:0/958843158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 msgr2=0x5612fbe60750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe60750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.007+0000 7f712e7fc700 1 -- 172.21.15.31:0/958843158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612fbc343b0 con 0x5612fbe60c90 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.008+0000 7f712e7fc700 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6fa20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f712000bcb0 tx=0x7f71200103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d0a23d7156b315 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.008+0000 7f712d7fa700 1 -- 172.21.15.31:0/958843158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120010c50 con 0x5612fbe60c90 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.008+0000 7f712d7fa700 1 -- 172.21.15.31:0/958843158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7120015070 con 0x5612fbe60c90 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.008+0000 7f712d7fa700 1 -- 172.21.15.31:0/958843158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120005020 con 0x5612fbe60c90 2023-12-14T18:35:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 -- 172.21.15.31:0/958843158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 msgr2=0x5612fbe6fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6fa20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f712000bcb0 tx=0x7f71200103d0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 -- 172.21.15.31:0/958843158 shutdown_connections 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe6ff60 0x5612fbe72340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe60750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 --2- 172.21.15.31:0/958843158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6fa20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 -- 172.21.15.31:0/958843158 >> 172.21.15.31:0/958843158 conn(0x5612fbe56ca0 msgr2=0x5612fbe5c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 -- 172.21.15.31:0/958843158 shutdown_connections 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 -- 172.21.15.31:0/958843158 wait complete. 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 -- start start 2023-12-14T18:35:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.009+0000 7f71494a9b80 1 Processor -- start 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe6f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe6ff60 0x5612fbe6d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbd4ef90 con 0x5612fbe60c90 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbd4ec30 con 0x5612fbe6ff60 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbc366a0 con 0x5612fbe60380 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71494a9b80 10 client.0 _ll_register_callbacks cb 0x5612fbd72910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50016/0 (socket says 172.21.15.31:50016) 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f7136ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe6f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 -- 172.21.15.31:0/3820459163 learned_addr learned my addr 172.21.15.31:0/3820459163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71377fe700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe6ff60 0x5612fbe6d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 -- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 msgr2=0x5612fbe6f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe6f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 -- 172.21.15.31:0/3820459163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe6ff60 msgr2=0x5612fbe6d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe6ff60 0x5612fbe6d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 -- 172.21.15.31:0/3820459163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712000b8e0 con 0x5612fbe60c90 2023-12-14T18:35:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f71377fe700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe6ff60 0x5612fbe6d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f712e7fc700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6cbc0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7120010fe0 tx=0x7f7120010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfceabec33f6cd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.010+0000 7f711e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f711e7fc700 1 -- 172.21.15.31:0/3820459163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120020070 con 0x5612fbe60c90 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f711e7fc700 1 -- 172.21.15.31:0/3820459163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7120017070 con 0x5612fbe60c90 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612fbe542b0 con 0x5612fbe60c90 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f711e7fc700 1 -- 172.21.15.31:0/3820459163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7120004890 con 0x5612fbe60c90 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 10 client.17259 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5612fbd79170 con 0x5612fbe60c90 2023-12-14T18:35:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f711e7fc700 1 -- 172.21.15.31:0/3820459163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612fbe542b0 con 0x5612fbe60c90 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f711e7fc700 1 -- 172.21.15.31:0/3820459163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7120011440 con 0x5612fbe60c90 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 10 client.17259 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 10 client.17259 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 20 client.17259 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbe7f910 0x5612fbe81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 4 client.17259 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.011+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612fbdabce0 con 0x5612fbe7f910 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.012+0000 7f7136ffd700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbe7f910 0x5612fbe81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.012+0000 7f7136ffd700 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbe7f910 0x5612fbe81d20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79e7a5134670776d server_cookie=532036a183753d in_seq=0 out_seq=0 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.012+0000 7f711e7fc700 10 client.17259 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f711e7fc700 1 -- 172.21.15.31:0/3820459163 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5612fbdabce0 con 0x5612fbe7f910 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f711e7fc700 10 client.17259 handle_command_reply: tid=0 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 client.17259 shutdown 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 msgr2=0x5612fbe6cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6cbc0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7120010fe0 tx=0x7f7120010c80 comp rx=0 tx=0).stop 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 shutdown_connections 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbe7f910 0x5612fbe81d20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe6ff60 0x5612fbe6d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f711e7fc700 0 client.17259 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe6f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 --2- 172.21.15.31:0/3820459163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe60c90 0x5612fbe6cbc0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 >> 172.21.15.31:0/3820459163 conn(0x5612fbe56ca0 msgr2=0x5612fbe5c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 shutdown_connections 2023-12-14T18:35:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 1 -- 172.21.15.31:0/3820459163 wait complete. 2023-12-14T18:35:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.013+0000 7f71494a9b80 20 client.17259 trim_cache size 0 max 16384 2023-12-14T18:35:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 Processor -- start 2023-12-14T18:35:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 -- start start 2023-12-14T18:35:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe76aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60c90 0x5612fbe5af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe7bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbe6d860 con 0x5612fbe6ff60 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbe77010 con 0x5612fbe60c90 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.027+0000 7f71494a9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbe6d140 con 0x5612fbe60380 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe7bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe7bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50018/0 (socket says 172.21.15.31:50018) 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 -- 172.21.15.31:0/1078777065 learned_addr learned my addr 172.21.15.31:0/1078777065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f7136ffd700 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe76aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f712e7fc700 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60c90 0x5612fbe5af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 -- 172.21.15.31:0/1078777065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 msgr2=0x5612fbe76aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe76aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 -- 172.21.15.31:0/1078777065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60c90 msgr2=0x5612fbe5af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60c90 0x5612fbe5af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 -- 172.21.15.31:0/1078777065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612fbd78ec0 con 0x5612fbe6ff60 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f712e7fc700 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60c90 0x5612fbe5af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f71377fe700 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe7bc10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7130009510 tx=0x7f713000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1ee6a3faf86d2e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f712dffb700 1 -- 172.21.15.31:0/1078777065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130099570 con 0x5612fbe6ff60 2023-12-14T18:35:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f712dffb700 1 -- 172.21.15.31:0/1078777065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f713009d5a0 con 0x5612fbe6ff60 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.028+0000 7f712dffb700 1 -- 172.21.15.31:0/1078777065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71300058a0 con 0x5612fbe6ff60 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 -- 172.21.15.31:0/1078777065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 msgr2=0x5612fbe7bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe7bc10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7130009510 tx=0x7f713000b040 comp rx=0 tx=0).stop 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 -- 172.21.15.31:0/1078777065 shutdown_connections 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe7bc10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60380 0x5612fbe76aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 --2- 172.21.15.31:0/1078777065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60c90 0x5612fbe5af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 -- 172.21.15.31:0/1078777065 >> 172.21.15.31:0/1078777065 conn(0x5612fbe56ca0 msgr2=0x5612fbe86640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 -- 172.21.15.31:0/1078777065 shutdown_connections 2023-12-14T18:35:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 -- 172.21.15.31:0/1078777065 wait complete. 2023-12-14T18:35:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 -- start start 2023-12-14T18:35:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.029+0000 7f71494a9b80 1 Processor -- start 2023-12-14T18:35:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe76830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60c90 0x5612fbe74b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbe757b0 con 0x5612fbe6ff60 2023-12-14T18:35:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbe75920 con 0x5612fbe60380 2023-12-14T18:35:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612fbed37a0 con 0x5612fbe60c90 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71494a9b80 10 client.0 _ll_register_callbacks cb 0x7f71300951f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50024/0 (socket says 172.21.15.31:50024) 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 -- 172.21.15.31:0/424048122 learned_addr learned my addr 172.21.15.31:0/424048122 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f7136ffd700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe76830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f712e7fc700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60c90 0x5612fbe74b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 -- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60c90 msgr2=0x5612fbe74b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60c90 0x5612fbe74b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 -- 172.21.15.31:0/424048122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 msgr2=0x5612fbe76830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe76830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.030+0000 7f71377fe700 1 -- 172.21.15.31:0/424048122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71381101c0 con 0x5612fbe6ff60 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71377fe700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe750a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7130007fd0 tx=0x7f713000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f5dbf510a1b92a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f711e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f711e7fc700 1 -- 172.21.15.31:0/424048122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71300b7070 con 0x5612fbe6ff60 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f711e7fc700 1 -- 172.21.15.31:0/424048122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f7130005aa0 con 0x5612fbe6ff60 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7120015070 con 0x5612fbe6ff60 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f711e7fc700 1 -- 172.21.15.31:0/424048122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f713009d140 con 0x5612fbe6ff60 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 10 client.17265 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5612fbd5f9a0 con 0x5612fbe6ff60 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f711e7fc700 1 -- 172.21.15.31:0/424048122 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7120015070 con 0x5612fbe6ff60 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f711e7fc700 1 -- 172.21.15.31:0/424048122 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f71300997d0 con 0x5612fbe6ff60 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 10 client.17265 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 10 client.17265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 20 client.17265 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbed49f0 0x5612fbed6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 4 client.17265 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.031+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5612fbdaba00 con 0x5612fbed49f0 2023-12-14T18:35:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f7136ffd700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbed49f0 0x5612fbed6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f711e7fc700 10 client.17265 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f7136ffd700 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbed49f0 0x5612fbed6dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee944caf5a1fc017 server_cookie=7ab121041b1b224 in_seq=0 out_seq=0 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f711e7fc700 1 -- 172.21.15.31:0/424048122 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5612fbdaba00 con 0x5612fbed49f0 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f711e7fc700 10 client.17265 handle_command_reply: tid=0 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 client.17265 shutdown 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 msgr2=0x5612fbe750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe750a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7130007fd0 tx=0x7f713000df60 comp rx=0 tx=0).stop 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 shutdown_connections 2023-12-14T18:35:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x5612fbed49f0 0x5612fbed6dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5612fbe6ff60 0x5612fbe750a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5612fbe60380 0x5612fbe76830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 --2- 172.21.15.31:0/424048122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5612fbe60c90 0x5612fbe74b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f711e7fc700 0 client.17265 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 >> 172.21.15.31:0/424048122 conn(0x5612fbe56ca0 msgr2=0x5612fbe62260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.032+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 shutdown_connections 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f71494a9b80 1 -- 172.21.15.31:0/424048122 wait complete. 2023-12-14T18:35:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f71494a9b80 20 client.17265 trim_cache size 0 max 16384 2023-12-14T18:35:39.044 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:35:39.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f71240a12c0 msgr2=0x7f71240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f71240a12c0 0x7f71240a3770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7128000c00 tx=0x7f7128005fb0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 msgr2=0x7f713819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 0x7f713819ad60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7130007ee0 tx=0x7f713000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 shutdown_connections 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7138104420 0x7f713819ad60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71380ff3e0 0x7f71381963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f71240a12c0 0x7f71240a3770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 --2- 172.21.15.31:0/525000697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71380ffd80 0x7f7138196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 >> 172.21.15.31:0/525000697 conn(0x7f7138075fc0 msgr2=0x7f71380fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 shutdown_connections 2023-12-14T18:35:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.033+0000 7f713d31b700 1 -- 172.21.15.31:0/525000697 wait complete. 2023-12-14T18:35:39.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:35:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 -- 172.21.15.31:0/4294211461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0ffa00 msgr2=0x7f35ac0ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 --2- 172.21.15.31:0/4294211461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0ffa00 0x7f35ac0ffe70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f35a0009990 tx=0x7f35a0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 -- 172.21.15.31:0/4294211461 shutdown_connections 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 --2- 172.21.15.31:0/4294211461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac10a8d0 0x7f35ac10ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 --2- 172.21.15.31:0/4294211461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac068b70 0x7f35ac0ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 --2- 172.21.15.31:0/4294211461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac0ffa00 0x7f35ac0ffe70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 -- 172.21.15.31:0/4294211461 >> 172.21.15.31:0/4294211461 conn(0x7f35ac074a50 msgr2=0x7f35ac074e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 -- 172.21.15.31:0/4294211461 shutdown_connections 2023-12-14T18:35:39.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.186+0000 7f35b1744700 1 -- 172.21.15.31:0/4294211461 wait complete. 2023-12-14T18:35:39.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 Processor -- start 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 -- start start 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 0x7f35ac1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac0ffa00 0x7f35ac1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac10a8d0 0x7f35ac19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac19b390 con 0x7f35ac068b70 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac19b500 con 0x7f35ac10a8d0 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35b1744700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac19b6c0 con 0x7f35ac0ffa00 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.187+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 0x7f35ac1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 0x7f35ac1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50044/0 (socket says 172.21.15.31:50044) 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 -- 172.21.15.31:0/3307313966 learned_addr learned my addr 172.21.15.31:0/3307313966 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac0ffa00 0x7f35ac1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac10a8d0 0x7f35ac19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 -- 172.21.15.31:0/3307313966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac0ffa00 msgr2=0x7f35ac1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac0ffa00 0x7f35ac1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 -- 172.21.15.31:0/3307313966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac10a8d0 msgr2=0x7f35ac19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac10a8d0 0x7f35ac19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 -- 172.21.15.31:0/3307313966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a00095f0 con 0x7f35ac068b70 2023-12-14T18:35:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac10a8d0 0x7f35ac19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac0ffa00 0x7f35ac1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35aaffd700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 0x7f35ac1963b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d950 tx=0x7f359c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f3593fff700 1 -- 172.21.15.31:0/3307313966 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c009b40 con 0x7f35ac068b70 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f3593fff700 1 -- 172.21.15.31:0/3307313966 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c009ca0 con 0x7f35ac068b70 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac110350 con 0x7f35ac068b70 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f3593fff700 1 -- 172.21.15.31:0/3307313966 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00f620 con 0x7f35ac068b70 2023-12-14T18:35:39.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.188+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac1101c0 con 0x7f35ac068b70 2023-12-14T18:35:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.191+0000 7f3593fff700 1 -- 172.21.15.31:0/3307313966 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f359c010460 con 0x7f35ac068b70 2023-12-14T18:35:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.191+0000 7f3593fff700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f35940a12c0 0x7f35940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.191+0000 7f3593fff700 1 -- 172.21.15.31:0/3307313966 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f359c0cd490 con 0x7f35ac068b70 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.191+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f35940a12c0 0x7f35940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f35940a12c0 0x7f35940a3770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f35a0006010 tx=0x7f35a000b540 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 Processor -- start 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 -- start start 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 0x55dccc6d6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d70a0 0x55dccc6e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6e1f60 0x55dccc6e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc5bc5e0 con 0x55dccc6d6790 2023-12-14T18:35:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc62b870 con 0x55dccc6e1f60 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc624d00 con 0x55dccc6d70a0 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 0x55dccc6d6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 0x55dccc6d6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50054/0 (socket says 172.21.15.31:50054) 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.192+0000 7f35aaffd700 1 -- 172.21.15.31:0/3137346531 learned_addr learned my addr 172.21.15.31:0/3137346531 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d70a0 0x55dccc6e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6e1f60 0x55dccc6e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aaffd700 1 -- 172.21.15.31:0/3137346531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d70a0 msgr2=0x55dccc6e1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aaffd700 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d70a0 0x55dccc6e1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aaffd700 1 -- 172.21.15.31:0/3137346531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6e1f60 msgr2=0x55dccc6e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aaffd700 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6e1f60 0x55dccc6e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aaffd700 1 -- 172.21.15.31:0/3137346531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dccc4a63b0 con 0x55dccc6d6790 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6e1f60 0x55dccc6e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35aaffd700 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 0x55dccc6d6b60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f359c096670 tx=0x7f359c099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170436e0cf4d594d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35917fa700 1 -- 172.21.15.31:0/3137346531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c09e1a0 con 0x55dccc6d6790 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35917fa700 1 -- 172.21.15.31:0/3137346531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c0a25c0 con 0x55dccc6d6790 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.193+0000 7f35917fa700 1 -- 172.21.15.31:0/3137346531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c09a630 con 0x55dccc6d6790 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/3137346531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 msgr2=0x55dccc6d6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 0x55dccc6d6b60 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f359c096670 tx=0x7f359c099ad0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/3137346531 shutdown_connections 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6e1f60 0x55dccc6e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6d6790 0x55dccc6d6b60 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/3137346531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d70a0 0x55dccc6e1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/3137346531 >> 172.21.15.31:0/3137346531 conn(0x55dccc6c8ca0 msgr2=0x55dccc6ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/3137346531 shutdown_connections 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/3137346531 wait complete. 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 -- start start 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.194+0000 7f35bd8d2b80 1 Processor -- start 2023-12-14T18:35:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc5c0f90 con 0x55dccc6e1f60 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc5c0c30 con 0x55dccc6d70a0 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc4a86a0 con 0x55dccc6d6790 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35bd8d2b80 10 client.0 _ll_register_callbacks cb 0x55dccc5e4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6df130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50058/0 (socket says 172.21.15.31:50058) 2023-12-14T18:35:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2323719594 learned_addr learned my addr 172.21.15.31:0/2323719594 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35aaffd700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2323719594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 msgr2=0x55dccc6e1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 msgr2=0x55dccc6deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6deb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2323719594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac1101c0 con 0x55dccc6e1f60 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6df130 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f35a40078c0 tx=0x7f35a4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4598544591293a93 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.195+0000 7f358eff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f358eff5700 1 -- 172.21.15.31:0/2323719594 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4010670 con 0x55dccc6e1f60 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f358eff5700 1 -- 172.21.15.31:0/2323719594 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f35a40107d0 con 0x55dccc6e1f60 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dccc6c62b0 con 0x55dccc6e1f60 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f358eff5700 1 -- 172.21.15.31:0/2323719594 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a4010990 con 0x55dccc6e1f60 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 10 client.17277 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dccc5eb170 con 0x55dccc6e1f60 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f358eff5700 1 -- 172.21.15.31:0/2323719594 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dccc6c62b0 con 0x55dccc6e1f60 2023-12-14T18:35:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f358eff5700 1 -- 172.21.15.31:0/2323719594 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f35a4015cc0 con 0x55dccc6e1f60 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 10 client.17277 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 10 client.17277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 20 client.17277 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc6f17c0 0x55dccc6f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 4 client.17277 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.196+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dccc61dce0 con 0x55dccc6f17c0 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.197+0000 7f35aaffd700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc6f17c0 0x55dccc6f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.197+0000 7f358eff5700 10 client.17277 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:39.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.197+0000 7f35aaffd700 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc6f17c0 0x55dccc6f3bd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca5a0eb0938ff092 server_cookie=1e04bdbe1a77b503 in_seq=0 out_seq=0 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f358eff5700 1 -- 172.21.15.31:0/2323719594 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dccc61dce0 con 0x55dccc6f17c0 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f358eff5700 10 client.17277 handle_command_reply: tid=0 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 client.17277 shutdown 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 msgr2=0x55dccc6df130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6df130 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f35a40078c0 tx=0x7f35a4007bd0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 shutdown_connections 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6df130 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc6f17c0 0x55dccc6f3bd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2323719594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f358eff5700 0 client.17277 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 >> 172.21.15.31:0/2323719594 conn(0x55dccc6c8ca0 msgr2=0x55dccc6ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 shutdown_connections 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2323719594 wait complete. 2023-12-14T18:35:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.198+0000 7f35bd8d2b80 20 client.17277 trim_cache size 0 max 16384 2023-12-14T18:35:39.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 Processor -- start 2023-12-14T18:35:39.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 -- start start 2023-12-14T18:35:39.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6de370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc6d7710 con 0x55dccc6e1f60 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc6cf250 con 0x55dccc6d70a0 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc6e1be0 con 0x55dccc6d6790 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6d9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50064/0 (socket says 172.21.15.31:50064) 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2185291742 learned_addr learned my addr 172.21.15.31:0/2185291742 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.212+0000 7f35aaffd700 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6de370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2185291742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 msgr2=0x55dccc6de370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6de370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2185291742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 msgr2=0x55dccc6ed0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6ed0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35ab7fe700 1 -- 172.21.15.31:0/2185291742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dccc5eaec0 con 0x55dccc6e1f60 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6d9ee0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f35a401d4c0 tx=0x7f35a4010220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7d435a6baf2249 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f3591ffb700 1 -- 172.21.15.31:0/2185291742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a401d5e0 con 0x55dccc6e1f60 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f3591ffb700 1 -- 172.21.15.31:0/2185291742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35a401d740 con 0x55dccc6e1f60 2023-12-14T18:35:39.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.213+0000 7f3591ffb700 1 -- 172.21.15.31:0/2185291742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a4015620 con 0x55dccc6e1f60 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2185291742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 msgr2=0x55dccc6d9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6d9ee0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f35a401d4c0 tx=0x7f35a4010220 comp rx=0 tx=0).stop 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2185291742 shutdown_connections 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6d9ee0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6de370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/2185291742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6ed0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2185291742 >> 172.21.15.31:0/2185291742 conn(0x55dccc6c8ca0 msgr2=0x55dccc6e14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2185291742 shutdown_connections 2023-12-14T18:35:39.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/2185291742 wait complete. 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 -- start start 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.214+0000 7f35bd8d2b80 1 Processor -- start 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6e6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc6e6e00 con 0x55dccc6e1f60 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc6e6f70 con 0x55dccc6d70a0 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dccc6e7130 con 0x55dccc6d6790 2023-12-14T18:35:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 10 client.0 _ll_register_callbacks cb 0x55dccc5e4910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6e6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50070/0 (socket says 172.21.15.31:50070) 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 -- 172.21.15.31:0/484431326 learned_addr learned my addr 172.21.15.31:0/484431326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35aa7fc700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35aaffd700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 -- 172.21.15.31:0/484431326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 msgr2=0x55dccc6e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 -- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 msgr2=0x55dccc6e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6e8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 -- 172.21.15.31:0/484431326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dccc5eb170 con 0x55dccc6e1f60 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35aaffd700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35ab7fe700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6e6730 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f35a40102b0 tx=0x7f35a4004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8653ed1246951288 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f358eff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f358eff5700 1 -- 172.21.15.31:0/484431326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a401d5e0 con 0x55dccc6e1f60 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f358eff5700 1 -- 172.21.15.31:0/484431326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f35a401dac0 con 0x55dccc6e1f60 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.215+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f359c0a25c0 con 0x55dccc6e1f60 2023-12-14T18:35:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f358eff5700 1 -- 172.21.15.31:0/484431326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a401dc80 con 0x55dccc6e1f60 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 10 client.17283 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dccc5d19a0 con 0x55dccc6e1f60 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f358eff5700 1 -- 172.21.15.31:0/484431326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f359c0a25c0 con 0x55dccc6e1f60 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f358eff5700 1 -- 172.21.15.31:0/484431326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f35a4010ad0 con 0x55dccc6e1f60 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 10 client.17283 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 10 client.17283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 20 client.17283 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc746270 0x55dccc748650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 4 client.17283 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dccc61da00 con 0x55dccc746270 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.216+0000 7f35aaffd700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc746270 0x55dccc748650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35aaffd700 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc746270 0x55dccc748650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65719616f2a4d7f3 server_cookie=fbb8cc7f54142fb2 in_seq=0 out_seq=0 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f358eff5700 10 client.17283 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f358eff5700 1 -- 172.21.15.31:0/484431326 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dccc61da00 con 0x55dccc746270 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f358eff5700 10 client.17283 handle_command_reply: tid=0 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 client.17283 shutdown 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 msgr2=0x55dccc6e6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6e6730 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f35a40102b0 tx=0x7f35a4004d40 comp rx=0 tx=0).stop 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 shutdown_connections 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dccc6e1f60 0x55dccc6e6730 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55dccc746270 0x55dccc748650 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dccc6d6790 0x55dccc6e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 --2- 172.21.15.31:0/484431326 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dccc6d70a0 0x55dccc6e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f358eff5700 0 client.17283 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 >> 172.21.15.31:0/484431326 conn(0x55dccc6c8ca0 msgr2=0x55dccc6ed0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 shutdown_connections 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 1 -- 172.21.15.31:0/484431326 wait complete. 2023-12-14T18:35:39.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.217+0000 7f35bd8d2b80 20 client.17283 trim_cache size 0 max 16384 2023-12-14T18:35:39.229 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f35940a12c0 msgr2=0x7f35940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f35940a12c0 0x7f35940a3770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f35a0006010 tx=0x7f35a000b540 comp rx=0 tx=0).stop 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 msgr2=0x7f35ac1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 0x7f35ac1963b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d950 tx=0x7f359c00dd10 comp rx=0 tx=0).stop 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 shutdown_connections 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35ac10a8d0 0x7f35ac19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35ac068b70 0x7f35ac1963b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f35940a12c0 0x7f35940a3770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 --2- 172.21.15.31:0/3307313966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35ac0ffa00 0x7f35ac1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 >> 172.21.15.31:0/3307313966 conn(0x7f35ac074a50 msgr2=0x7f35ac1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 shutdown_connections 2023-12-14T18:35:39.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.218+0000 7f35b1744700 1 -- 172.21.15.31:0/3307313966 wait complete. 2023-12-14T18:35:39.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:35:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 -- 172.21.15.31:0/2387597693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a7410cbe0 msgr2=0x7f3a7410efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/2387597693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a7410cbe0 0x7f3a7410efc0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00b0a0 tx=0x7f3a6c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 -- 172.21.15.31:0/2387597693 shutdown_connections 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/2387597693 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a7410cbe0 0x7f3a7410efc0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/2387597693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a740677b0 0x7f3a740ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/2387597693 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a740ff9b0 0x7f3a740ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.373+0000 7f3a7aad4700 1 -- 172.21.15.31:0/2387597693 >> 172.21.15.31:0/2387597693 conn(0x7f3a74074a60 msgr2=0x7f3a74074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 -- 172.21.15.31:0/2387597693 shutdown_connections 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 -- 172.21.15.31:0/2387597693 wait complete. 2023-12-14T18:35:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 Processor -- start 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 -- start start 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a740677b0 0x7f3a74198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 0x7f3a74198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7410cbe0 0x7f3a7419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7419d630 con 0x7f3a740ff9b0 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7419d7a0 con 0x7f3a7410cbe0 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.374+0000 7f3a7aad4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7419d960 con 0x7f3a740677b0 2023-12-14T18:35:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 0x7f3a74198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 0x7f3a74198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50086/0 (socket says 172.21.15.31:50086) 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1770744290 learned_addr learned my addr 172.21.15.31:0/1770744290 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a78870700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a740677b0 0x7f3a74198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a79071700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7410cbe0 0x7f3a7419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1770744290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a740677b0 msgr2=0x7f3a74198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a740677b0 0x7f3a74198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1770744290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7410cbe0 msgr2=0x7f3a7419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7410cbe0 0x7f3a7419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1770744290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c009d00 con 0x7f3a740ff9b0 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 0x7f3a74198b90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400e9a0 tx=0x7f3a6400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a69ffb700 1 -- 172.21.15.31:0/1770744290 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64009790 con 0x7f3a740ff9b0 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a69ffb700 1 -- 172.21.15.31:0/1770744290 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a64010430 con 0x7f3a740ff9b0 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a741124d0 con 0x7f3a740ff9b0 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.375+0000 7f3a69ffb700 1 -- 172.21.15.31:0/1770744290 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64018680 con 0x7f3a740ff9b0 2023-12-14T18:35:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.376+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a74112340 con 0x7f3a740ff9b0 2023-12-14T18:35:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.378+0000 7f3a69ffb700 1 -- 172.21.15.31:0/1770744290 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3a640105a0 con 0x7f3a740ff9b0 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.378+0000 7f3a69ffb700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3a5c0a12c0 0x7f3a5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.379+0000 7f3a78870700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3a5c0a12c0 0x7f3a5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.379+0000 7f3a69ffb700 1 -- 172.21.15.31:0/1770744290 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f3a64014070 con 0x7f3a740ff9b0 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.379+0000 7f3a78870700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3a5c0a12c0 0x7f3a5c0a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a740773e0 tx=0x7f3a7000b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.379+0000 7f3a86c62b80 1 Processor -- start 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 -- start start 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218380 0x55dd4b218750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b225f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b1025e0 con 0x55dd4b218c90 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b171870 con 0x55dd4b218380 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b16ad00 con 0x55dd4b223b50 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50100/0 (socket says 172.21.15.31:50100) 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1031854182 learned_addr learned my addr 172.21.15.31:0/1031854182 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a78870700 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218380 0x55dd4b218750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a79071700 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b225f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1031854182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 msgr2=0x55dd4b225f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b225f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1031854182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218380 msgr2=0x55dd4b218750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218380 0x55dd4b218750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.380+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1031854182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd4afec3b0 con 0x55dd4b218c90 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.381+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3a640a1ec0 tx=0x7f3a64099fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc66f9cca79bf14c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.381+0000 7f3a5affd700 1 -- 172.21.15.31:0/1031854182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640cd160 con 0x55dd4b218c90 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.381+0000 7f3a5affd700 1 -- 172.21.15.31:0/1031854182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a640a23f0 con 0x55dd4b218c90 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.381+0000 7f3a5affd700 1 -- 172.21.15.31:0/1031854182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64099690 con 0x55dd4b218c90 2023-12-14T18:35:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.381+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1031854182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 msgr2=0x55dd4b223610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.381+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3a640a1ec0 tx=0x7f3a64099fa0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1031854182 shutdown_connections 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b225f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218380 0x55dd4b218750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1031854182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223610 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1031854182 >> 172.21.15.31:0/1031854182 conn(0x55dd4b20eca0 msgr2=0x55dd4b2147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1031854182 shutdown_connections 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1031854182 wait complete. 2023-12-14T18:35:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 -- start start 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.382+0000 7f3a86c62b80 1 Processor -- start 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 0x55dd4b2235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218c90 0x55dd4b220880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b220f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b106f90 con 0x55dd4b218380 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b106c30 con 0x55dd4b218c90 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4afee6a0 con 0x55dd4b223b50 2023-12-14T18:35:39.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 10 client.0 _ll_register_callbacks cb 0x55dd4b12a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 0x55dd4b2235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 0x55dd4b2235d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50114/0 (socket says 172.21.15.31:50114) 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 -- 172.21.15.31:0/723152051 learned_addr learned my addr 172.21.15.31:0/723152051 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a79071700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b220f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218c90 0x55dd4b220880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 -- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 msgr2=0x55dd4b220f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b220f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 -- 172.21.15.31:0/723152051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218c90 msgr2=0x55dd4b220880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218c90 0x55dd4b220880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 -- 172.21.15.31:0/723152051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a74112340 con 0x55dd4b218380 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218c90 0x55dd4b220880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a78870700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 0x55dd4b2235d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3a700116c0 tx=0x7f3a700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717c222c53606304 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a53fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a53fff700 1 -- 172.21.15.31:0/723152051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70028670 con 0x55dd4b218380 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a53fff700 1 -- 172.21.15.31:0/723152051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f3a70017070 con 0x55dd4b218380 2023-12-14T18:35:39.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.383+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd4b20c2b0 con 0x55dd4b218380 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a53fff700 1 -- 172.21.15.31:0/723152051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a70011e40 con 0x55dd4b218380 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 10 client.17295 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dd4b131170 con 0x55dd4b218380 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a53fff700 1 -- 172.21.15.31:0/723152051 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd4b20c2b0 con 0x55dd4b218380 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a53fff700 1 -- 172.21.15.31:0/723152051 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3a7001e550 con 0x55dd4b218380 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 10 client.17295 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 20 client.17295 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b233600 0x55dd4b235a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 4 client.17295 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd4b163ce0 con 0x55dd4b233600 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.384+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b233600 0x55dd4b235a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.385+0000 7f3a53fff700 10 client.17295 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.385+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b233600 0x55dd4b235a10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67541cd9e9159740 server_cookie=36d6c78fc2351b2d in_seq=0 out_seq=0 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a53fff700 1 -- 172.21.15.31:0/723152051 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd4b163ce0 con 0x55dd4b233600 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a53fff700 10 client.17295 handle_command_reply: tid=0 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 client.17295 shutdown 2023-12-14T18:35:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 msgr2=0x55dd4b2235d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 0x55dd4b2235d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3a700116c0 tx=0x7f3a700047c0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 shutdown_connections 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b233600 0x55dd4b235a10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b223b50 0x55dd4b220f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a53fff700 0 client.17295 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218380 0x55dd4b2235d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/723152051 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b218c90 0x55dd4b220880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 >> 172.21.15.31:0/723152051 conn(0x55dd4b20eca0 msgr2=0x55dd4b2143d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 shutdown_connections 2023-12-14T18:35:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 1 -- 172.21.15.31:0/723152051 wait complete. 2023-12-14T18:35:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.386+0000 7f3a86c62b80 20 client.17295 trim_cache size 0 max 16384 2023-12-14T18:35:39.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 Processor -- start 2023-12-14T18:35:39.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 -- start start 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b22a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b221550 con 0x55dd4b218c90 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b22f540 con 0x55dd4b223b50 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.400+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b2152d0 con 0x55dd4b218380 2023-12-14T18:35:39.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50130/0 (socket says 172.21.15.31:50130) 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1484783515 learned_addr learned my addr 172.21.15.31:0/1484783515 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a78870700 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b22a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a79071700 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1484783515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 msgr2=0x55dd4b22a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b22a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1484783515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 msgr2=0x55dd4b2235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1484783515 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd4b2104e0 con 0x55dd4b218c90 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a79071700 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3a640ca6d0 tx=0x7f3a64010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc15401abdf62d84 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a5b7fe700 1 -- 172.21.15.31:0/1484783515 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640cd1e0 con 0x55dd4b218c90 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a5b7fe700 1 -- 172.21.15.31:0/1484783515 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a64010880 con 0x55dd4b218c90 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.401+0000 7f3a5b7fe700 1 -- 172.21.15.31:0/1484783515 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6409a460 con 0x55dd4b218c90 2023-12-14T18:35:39.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1484783515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 msgr2=0x55dd4b223090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223090 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3a640ca6d0 tx=0x7f3a64010c80 comp rx=0 tx=0).stop 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1484783515 shutdown_connections 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b22a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1484783515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b223090 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1484783515 >> 172.21.15.31:0/1484783515 conn(0x55dd4b20eca0 msgr2=0x55dd4b22ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1484783515 shutdown_connections 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1484783515 wait complete. 2023-12-14T18:35:39.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 -- start start 2023-12-14T18:35:39.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.402+0000 7f3a86c62b80 1 Processor -- start 2023-12-14T18:35:39.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b2280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b228620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2864c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b228ee0 con 0x55dd4b218c90 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b286a00 con 0x55dd4b223b50 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd4b286b40 con 0x55dd4b218380 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a86c62b80 10 client.0 _ll_register_callbacks cb 0x55dd4b12a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b228620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b228620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50144/0 (socket says 172.21.15.31:50144) 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1520025473 learned_addr learned my addr 172.21.15.31:0/1520025473 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a78870700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b2280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a79071700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 msgr2=0x55dd4b2280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b2280e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1520025473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 msgr2=0x55dd4b2864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.403+0000 7f3a6bfff700 1 -- 172.21.15.31:0/1520025473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd4afec3b0 con 0x55dd4b218c90 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a6bfff700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b228620 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3a64096a10 tx=0x7f3a64010a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2aec43d951760e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a53fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a53fff700 1 -- 172.21.15.31:0/1520025473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6409a400 con 0x55dd4b218c90 2023-12-14T18:35:39.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a53fff700 1 -- 172.21.15.31:0/1520025473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f3a64099490 con 0x55dd4b218c90 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a640a23f0 con 0x55dd4b218c90 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a53fff700 1 -- 172.21.15.31:0/1520025473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a64099670 con 0x55dd4b218c90 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 10 client.17301 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dd4af98b60 con 0x55dd4b218c90 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a53fff700 1 -- 172.21.15.31:0/1520025473 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a640a23f0 con 0x55dd4b218c90 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a53fff700 1 -- 172.21.15.31:0/1520025473 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3a640c6d20 con 0x55dd4b218c90 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 10 client.17301 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:39.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 20 client.17301 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b287d10 0x55dd4b28a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 4 client.17301 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.404+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd4b163a00 con 0x55dd4b287d10 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a78870700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b287d10 0x55dd4b28a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a53fff700 10 client.17301 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a78870700 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b287d10 0x55dd4b28a0f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877795d0da3d5644 server_cookie=b5b26578cf81078b in_seq=0 out_seq=0 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a53fff700 1 -- 172.21.15.31:0/1520025473 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd4b163a00 con 0x55dd4b287d10 2023-12-14T18:35:39.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a53fff700 10 client.17301 handle_command_reply: tid=0 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 client.17301 shutdown 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 msgr2=0x55dd4b228620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b228620 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3a64096a10 tx=0x7f3a64010a70 comp rx=0 tx=0).stop 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 shutdown_connections 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dd4b287d10 0x55dd4b28a0f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dd4b223b50 0x55dd4b2864c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dd4b218380 0x55dd4b2280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 --2- 172.21.15.31:0/1520025473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dd4b218c90 0x55dd4b228620 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a53fff700 0 client.17301 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:39.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.405+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 >> 172.21.15.31:0/1520025473 conn(0x55dd4b20eca0 msgr2=0x55dd4b220120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 shutdown_connections 2023-12-14T18:35:39.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a86c62b80 1 -- 172.21.15.31:0/1520025473 wait complete. 2023-12-14T18:35:39.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a86c62b80 20 client.17301 trim_cache size 0 max 16384 2023-12-14T18:35:39.418 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:35:39.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3a5c0a12c0 msgr2=0x7f3a5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3a5c0a12c0 0x7f3a5c0a3770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3a740773e0 tx=0x7f3a7000b410 comp rx=0 tx=0).stop 2023-12-14T18:35:39.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 msgr2=0x7f3a74198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 0x7f3a74198b90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400e9a0 tx=0x7f3a6400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 shutdown_connections 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3a7410cbe0 0x7f3a7419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3a5c0a12c0 0x7f3a5c0a3770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a740677b0 0x7f3a74198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 --2- 172.21.15.31:0/1770744290 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a740ff9b0 0x7f3a74198b90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 >> 172.21.15.31:0/1770744290 conn(0x7f3a74074a60 msgr2=0x7f3a7410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:39.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 shutdown_connections 2023-12-14T18:35:39.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:39.406+0000 7f3a7aad4700 1 -- 172.21.15.31:0/1770744290 wait complete. 2023-12-14T18:35:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:39 smithi204 ceph-mon[43866]: pgmap v242: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:39 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/747559232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:35:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:39 smithi031 ceph-mon[42085]: pgmap v242: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:39 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/747559232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:35:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:39 smithi176 ceph-mon[43506]: pgmap v242: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:39 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/747559232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:35:40.423 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:35:40.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:35:40.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 -- 172.21.15.31:0/920529712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f80ff3b0 msgr2=0x7f03f80ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 --2- 172.21.15.31:0/920529712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f80ff3b0 0x7f03f80ff780 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f03e80099c0 tx=0x7f03e8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 -- 172.21.15.31:0/920529712 shutdown_connections 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 --2- 172.21.15.31:0/920529712 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03f81043f0 0x7f03f810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 --2- 172.21.15.31:0/920529712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f80ff3b0 0x7f03f80ff780 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 --2- 172.21.15.31:0/920529712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f80ffd50 0x7f03f8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 -- 172.21.15.31:0/920529712 >> 172.21.15.31:0/920529712 conn(0x7f03f80763a0 msgr2=0x7f03f80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 -- 172.21.15.31:0/920529712 shutdown_connections 2023-12-14T18:35:40.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.573+0000 7f03fd16a700 1 -- 172.21.15.31:0/920529712 wait complete. 2023-12-14T18:35:40.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 Processor -- start 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 -- start start 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03f80ff3b0 0x7f03f810c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f80ffd50 0x7f03f810ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 0x7f03f81a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f810d640 con 0x7f03f81043f0 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f81a4ab0 con 0x7f03f80ff3b0 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.574+0000 7f03fd16a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f81a4bf0 con 0x7f03f80ffd50 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 0x7f03f81a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 0x7f03f81a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50166/0 (socket says 172.21.15.31:50166) 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 -- 172.21.15.31:0/342133742 learned_addr learned my addr 172.21.15.31:0/342133742 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03f80ff3b0 0x7f03f810c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f659c700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f80ffd50 0x7f03f810ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 -- 172.21.15.31:0/342133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f80ffd50 msgr2=0x7f03f810ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f80ffd50 0x7f03f810ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 -- 172.21.15.31:0/342133742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03f80ff3b0 msgr2=0x7f03f810c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03f80ff3b0 0x7f03f810c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 -- 172.21.15.31:0/342133742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80095f0 con 0x7f03f81043f0 2023-12-14T18:35:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03f759e700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 0x7f03f81a4570 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f03f000efc0 tx=0x7f03f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03effff700 1 -- 172.21.15.31:0/342133742 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0018070 con 0x7f03f81043f0 2023-12-14T18:35:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03effff700 1 -- 172.21.15.31:0/342133742 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f000f040 con 0x7f03f81043f0 2023-12-14T18:35:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f8110320 con 0x7f03f81043f0 2023-12-14T18:35:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03effff700 1 -- 172.21.15.31:0/342133742 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000cc10 con 0x7f03f81043f0 2023-12-14T18:35:40.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.575+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f8110190 con 0x7f03f81043f0 2023-12-14T18:35:40.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.578+0000 7f03effff700 1 -- 172.21.15.31:0/342133742 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f03f0003a00 con 0x7f03f81043f0 2023-12-14T18:35:40.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.578+0000 7f03effff700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f03e40a12c0 0x7f03e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.578+0000 7f03effff700 1 -- 172.21.15.31:0/342133742 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f03f00ccaf0 con 0x7f03f81043f0 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.578+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f03e40a12c0 0x7f03e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.579+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f03e40a12c0 0x7f03e40a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009990 tx=0x7f03e8005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.579+0000 7f04092f8b80 1 Processor -- start 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.579+0000 7f04092f8b80 1 -- start start 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e5597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e564690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e566fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f04092f8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e443660 con 0x55b65e559400 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f04092f8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e4b28f0 con 0x55b65e559d10 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f04092f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e4abd80 con 0x55b65e564bd0 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e5597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e5597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50182/0 (socket says 172.21.15.31:50182) 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 -- 172.21.15.31:0/1209008919 learned_addr learned my addr 172.21.15.31:0/1209008919 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f759e700 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e566fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f659c700 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e564690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 -- 172.21.15.31:0/1209008919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 msgr2=0x55b65e566fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e566fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 -- 172.21.15.31:0/1209008919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 msgr2=0x55b65e564690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e564690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 -- 172.21.15.31:0/1209008919 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65e32d430 con 0x55b65e559400 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.580+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e5597d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f03e80099c0 tx=0x7f03e801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84d5251977cee6b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f03ed7fa700 1 -- 172.21.15.31:0/1209008919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800b990 con 0x55b65e559400 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f03ed7fa700 1 -- 172.21.15.31:0/1209008919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03e801e040 con 0x55b65e559400 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f03ed7fa700 1 -- 172.21.15.31:0/1209008919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800f660 con 0x55b65e559400 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f04092f8b80 1 -- 172.21.15.31:0/1209008919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 msgr2=0x55b65e5597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f04092f8b80 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e5597d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f03e80099c0 tx=0x7f03e801f000 comp rx=0 tx=0).stop 2023-12-14T18:35:40.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f04092f8b80 1 -- 172.21.15.31:0/1209008919 shutdown_connections 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.581+0000 7f04092f8b80 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e566fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e5597d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 --2- 172.21.15.31:0/1209008919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e564690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- 172.21.15.31:0/1209008919 >> 172.21.15.31:0/1209008919 conn(0x55b65e54fd20 msgr2=0x55b65e555860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- 172.21.15.31:0/1209008919 shutdown_connections 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- 172.21.15.31:0/1209008919 wait complete. 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- start start 2023-12-14T18:35:40.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 Processor -- start 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e564530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e5617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e561d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e448010 con 0x55b65e559400 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e447cb0 con 0x55b65e559d10 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e32f720 con 0x55b65e564bd0 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f04092f8b80 10 client.0 _ll_register_callbacks cb 0x55b65e46b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e564530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e564530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50194/0 (socket says 172.21.15.31:50194) 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.582+0000 7f03f6d9d700 1 -- 172.21.15.31:0/524054507 learned_addr learned my addr 172.21.15.31:0/524054507 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f759e700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e561d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f659c700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e5617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f6d9d700 1 -- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 msgr2=0x55b65e561d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e561d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f6d9d700 1 -- 172.21.15.31:0/524054507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 msgr2=0x55b65e5617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e5617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f6d9d700 1 -- 172.21.15.31:0/524054507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f8068f80 con 0x55b65e559400 2023-12-14T18:35:40.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e564530 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009d10 tx=0x7f03e8003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd43adb3978bccb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03de7fc700 1 -- 172.21.15.31:0/524054507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800fd30 con 0x55b65e559400 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03de7fc700 1 -- 172.21.15.31:0/524054507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f03e800b790 con 0x55b65e559400 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b65e54d330 con 0x55b65e559400 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f03de7fc700 1 -- 172.21.15.31:0/524054507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03e800fe90 con 0x55b65e559400 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f04092f8b80 10 client.17313 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.583+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b65e4721f0 con 0x55b65e559400 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f03de7fc700 1 -- 172.21.15.31:0/524054507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b65e54d330 con 0x55b65e559400 2023-12-14T18:35:40.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f03de7fc700 1 -- 172.21.15.31:0/524054507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f03e803aa40 con 0x55b65e559400 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f04092f8b80 10 client.17313 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f04092f8b80 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f04092f8b80 20 client.17313 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f04092f8b80 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e574060 0x55b65e576470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f04092f8b80 4 client.17313 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b65e4a4d60 con 0x55b65e574060 2023-12-14T18:35:40.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.584+0000 7f03f659c700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e574060 0x55b65e576470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.585+0000 7f03f659c700 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e574060 0x55b65e576470 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f39252bc48aa466 server_cookie=4cde3d2af18c04e0 in_seq=0 out_seq=0 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.585+0000 7f03de7fc700 10 client.17313 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.585+0000 7f03de7fc700 1 -- 172.21.15.31:0/524054507 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b65e4a4d60 con 0x55b65e574060 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.585+0000 7f03de7fc700 10 client.17313 handle_command_reply: tid=0 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.585+0000 7f04092f8b80 1 client.17313 shutdown 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 msgr2=0x55b65e564530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e564530 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009d10 tx=0x7f03e8003f70 comp rx=0 tx=0).stop 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 shutdown_connections 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e574060 0x55b65e576470 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e564bd0 0x55b65e561d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f03de7fc700 0 client.17313 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e564530 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 --2- 172.21.15.31:0/524054507 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e5617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 >> 172.21.15.31:0/524054507 conn(0x55b65e54fd20 msgr2=0x55b65e555450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 shutdown_connections 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 1 -- 172.21.15.31:0/524054507 wait complete. 2023-12-14T18:35:40.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.586+0000 7f04092f8b80 20 client.17313 trim_cache size 0 max 16384 2023-12-14T18:35:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 Processor -- start 2023-12-14T18:35:40.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 -- start start 2023-12-14T18:35:40.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e561790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559d10 0x55b65e56b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e564bd0 0x55b65e563ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e55c630 con 0x55b65e559400 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e55fd60 con 0x55b65e564bd0 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.604+0000 7f04092f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e554630 con 0x55b65e559d10 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e561790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e561790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50206/0 (socket says 172.21.15.31:50206) 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 -- 172.21.15.31:0/3981196258 learned_addr learned my addr 172.21.15.31:0/3981196258 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f659c700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559d10 0x55b65e56b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f759e700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e564bd0 0x55b65e563ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 -- 172.21.15.31:0/3981196258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559d10 msgr2=0x55b65e56b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559d10 0x55b65e56b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 -- 172.21.15.31:0/3981196258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e564bd0 msgr2=0x55b65e563ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e564bd0 0x55b65e563ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 -- 172.21.15.31:0/3981196258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65e471f40 con 0x55b65e559400 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f659c700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559d10 0x55b65e56b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f759e700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e564bd0 0x55b65e563ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:40.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e561790 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f03e8004700 tx=0x7f03e8003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ab0f38203fde5e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03edffb700 1 -- 172.21.15.31:0/3981196258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800f790 con 0x55b65e559400 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03edffb700 1 -- 172.21.15.31:0/3981196258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03e8030520 con 0x55b65e559400 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.605+0000 7f03edffb700 1 -- 172.21.15.31:0/3981196258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800b730 con 0x55b65e559400 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 -- 172.21.15.31:0/3981196258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 msgr2=0x55b65e561790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e561790 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f03e8004700 tx=0x7f03e8003e70 comp rx=0 tx=0).stop 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 -- 172.21.15.31:0/3981196258 shutdown_connections 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e564bd0 0x55b65e563ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e559400 0x55b65e561790 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 --2- 172.21.15.31:0/3981196258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559d10 0x55b65e56b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 -- 172.21.15.31:0/3981196258 >> 172.21.15.31:0/3981196258 conn(0x55b65e54fd20 msgr2=0x55b65e561080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 -- 172.21.15.31:0/3981196258 shutdown_connections 2023-12-14T18:35:40.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 -- 172.21.15.31:0/3981196258 wait complete. 2023-12-14T18:35:40.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.606+0000 7f04092f8b80 1 -- start start 2023-12-14T18:35:40.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 Processor -- start 2023-12-14T18:35:40.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559400 0x55b65e56b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e55ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 0x55b65e55e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e55eb20 con 0x55b65e564bd0 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e55ec90 con 0x55b65e559d10 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65e5c7f20 con 0x55b65e559400 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f04092f8b80 10 client.0 _ll_register_callbacks cb 0x7f03e80039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 0x55b65e55e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 0x55b65e55e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50212/0 (socket says 172.21.15.31:50212) 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 -- 172.21.15.31:0/2631374739 learned_addr learned my addr 172.21.15.31:0/2631374739 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f659c700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e55ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559400 0x55b65e56b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 -- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559400 msgr2=0x55b65e56b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559400 0x55b65e56b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 -- 172.21.15.31:0/2631374739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 msgr2=0x55b65e55ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e55ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f759e700 1 -- 172.21.15.31:0/2631374739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b65e4721f0 con 0x55b65e564bd0 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.607+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559400 0x55b65e56b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03f759e700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 0x55b65e55e410 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f03f00090a0 tx=0x7f03f0099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8675fd88beda786c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03de7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03de7fc700 1 -- 172.21.15.31:0/2631374739 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f009d7f0 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03de7fc700 1 -- 172.21.15.31:0/2631374739 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f03f00a1b80 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03e801e040 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03de7fc700 1 -- 172.21.15.31:0/2631374739 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03f00b9070 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 10 client.17319 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55b65e458a20 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03de7fc700 1 -- 172.21.15.31:0/2631374739 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f00b6070 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f03de7fc700 1 -- 172.21.15.31:0/2631374739 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f03f0098cb0 con 0x55b65e564bd0 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 10 client.17319 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 20 client.17319 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e5c9590 0x55b65e5cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 4 client.17319 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.608+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b65e4a4a80 con 0x55b65e5c9590 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.609+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e5c9590 0x55b65e5cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.609+0000 7f03de7fc700 10 client.17319 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.609+0000 7f03f6d9d700 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e5c9590 0x55b65e5cb9a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e576d8de257b299 server_cookie=b927279c0e991234 in_seq=0 out_seq=0 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.610+0000 7f03de7fc700 1 -- 172.21.15.31:0/2631374739 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b65e4a4a80 con 0x55b65e5c9590 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.610+0000 7f03de7fc700 10 client.17319 handle_command_reply: tid=0 2023-12-14T18:35:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.610+0000 7f04092f8b80 1 client.17319 shutdown 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 msgr2=0x55b65e55e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 0x55b65e55e410 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f03f00090a0 tx=0x7f03f0099380 comp rx=0 tx=0).stop 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 shutdown_connections 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55b65e5c9590 0x55b65e5cb9a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55b65e564bd0 0x55b65e55e410 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65e559400 0x55b65e56b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 --2- 172.21.15.31:0/2631374739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55b65e559d10 0x55b65e55ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 >> 172.21.15.31:0/2631374739 conn(0x55b65e54fd20 msgr2=0x55b65e55ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f03de7fc700 0 client.17319 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 shutdown_connections 2023-12-14T18:35:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 1 -- 172.21.15.31:0/2631374739 wait complete. 2023-12-14T18:35:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f04092f8b80 20 client.17319 trim_cache size 0 max 16384 2023-12-14T18:35:40.621 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:35:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:35:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "55d16bb6-3ad0-490d-ae30-b258415a027b", 2023-12-14T18:35:40.622 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:35:40.622 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:35:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f03e40a12c0 msgr2=0x7f03e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f03fd16a700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f03e40a12c0 0x7f03e40a3770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009990 tx=0x7f03e8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 msgr2=0x7f03f81a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f03fd16a700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 0x7f03f81a4570 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f03f000efc0 tx=0x7f03f000c3a0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.611+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 shutdown_connections 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f81043f0 0x7f03f81a4570 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03f80ff3b0 0x7f03f810c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f03e40a12c0 0x7f03e40a3770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 --2- 172.21.15.31:0/342133742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03f80ffd50 0x7f03f810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 >> 172.21.15.31:0/342133742 conn(0x7f03f80763a0 msgr2=0x7f03f8102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 shutdown_connections 2023-12-14T18:35:40.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.612+0000 7f03fd16a700 1 -- 172.21.15.31:0/342133742 wait complete. 2023-12-14T18:35:40.628 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:35:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 -- 172.21.15.31:0/3984666232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 msgr2=0x7f1718103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 --2- 172.21.15.31:0/3984666232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718103d30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f17000099c0 tx=0x7f1700009cd0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 -- 172.21.15.31:0/3984666232 shutdown_connections 2023-12-14T18:35:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 --2- 172.21.15.31:0/3984666232 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1718104410 0x7f171810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 --2- 172.21.15.31:0/3984666232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17180ff420 0x7f17180ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 --2- 172.21.15.31:0/3984666232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718103d30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 -- 172.21.15.31:0/3984666232 >> 172.21.15.31:0/3984666232 conn(0x7f1718075fc0 msgr2=0x7f17180763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 -- 172.21.15.31:0/3984666232 shutdown_connections 2023-12-14T18:35:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.764+0000 7f171cd3a700 1 -- 172.21.15.31:0/3984666232 wait complete. 2023-12-14T18:35:40.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.765+0000 7f171cd3a700 1 Processor -- start 2023-12-14T18:35:40.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.765+0000 7f171cd3a700 1 -- start start 2023-12-14T18:35:40.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171cd3a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17180ff420 0x7f1718195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171cd3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171cd3a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1718104410 0x7f1718199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171cd3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171819a1e0 con 0x7f17180ffdc0 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171cd3a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171819a350 con 0x7f1718104410 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171cd3a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171819a510 con 0x7f17180ff420 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f171659c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17180ff420 0x7f1718195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50240/0 (socket says 172.21.15.31:50240) 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 -- 172.21.15.31:0/1676737544 learned_addr learned my addr 172.21.15.31:0/1676737544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1716d9d700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1718104410 0x7f1718199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 -- 172.21.15.31:0/1676737544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17180ff420 msgr2=0x7f1718195200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17180ff420 0x7f1718195200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 -- 172.21.15.31:0/1676737544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1718104410 msgr2=0x7f1718199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1718104410 0x7f1718199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 -- 172.21.15.31:0/1676737544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17000095f0 con 0x7f17180ffdc0 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f1715d9b700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718195740 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1700009fd0 tx=0x7f1700009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.766+0000 7f170f7fe700 1 -- 172.21.15.31:0/1676737544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170001d070 con 0x7f17180ffdc0 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.767+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171810f190 con 0x7f17180ffdc0 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.767+0000 7f170f7fe700 1 -- 172.21.15.31:0/1676737544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1700005020 con 0x7f17180ffdc0 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.767+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171810f000 con 0x7f17180ffdc0 2023-12-14T18:35:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.767+0000 7f170f7fe700 1 -- 172.21.15.31:0/1676737544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700004ac0 con 0x7f17180ffdc0 2023-12-14T18:35:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.769+0000 7f170f7fe700 1 -- 172.21.15.31:0/1676737544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f17000054b0 con 0x7f17180ffdc0 2023-12-14T18:35:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.770+0000 7f170f7fe700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17040a12c0 0x7f17040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.770+0000 7f171659c700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17040a12c0 0x7f17040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.770+0000 7f170f7fe700 1 -- 172.21.15.31:0/1676737544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f17000cdc70 con 0x7f17180ffdc0 2023-12-14T18:35:40.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.770+0000 7f1728ec8b80 1 Processor -- start 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.770+0000 7f171659c700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17040a12c0 0x7f17040a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1708009c60 tx=0x7f1708009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.770+0000 7f1728ec8b80 1 -- start start 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a75380 0x55f465a75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 0x55f465a82820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a82d60 0x55f465a85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46595f5e0 con 0x55f465a75c90 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4659ce870 con 0x55f465a75380 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4659c7d00 con 0x55f465a82d60 2023-12-14T18:35:40.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 0x55f465a82820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 0x55f465a82820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50246/0 (socket says 172.21.15.31:50246) 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 -- 172.21.15.31:0/2547568187 learned_addr learned my addr 172.21.15.31:0/2547568187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1716d9d700 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a82d60 0x55f465a85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f171659c700 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a75380 0x55f465a75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 -- 172.21.15.31:0/2547568187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a82d60 msgr2=0x55f465a85140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a82d60 0x55f465a85140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 -- 172.21.15.31:0/2547568187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a75380 msgr2=0x55f465a75750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a75380 0x55f465a75750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 -- 172.21.15.31:0/2547568187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4658493b0 con 0x55f465a75c90 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f171659c700 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a75380 0x55f465a75750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.771+0000 7f1715d9b700 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 0x55f465a82820 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f170009a910 tx=0x7f170009a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0371d2743dcb7ed server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f170cff9700 1 -- 172.21.15.31:0/2547568187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17000a2d40 con 0x55f465a75c90 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f170cff9700 1 -- 172.21.15.31:0/2547568187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1700098d20 con 0x55f465a75c90 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f170cff9700 1 -- 172.21.15.31:0/2547568187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170009a0d0 con 0x55f465a75c90 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 -- 172.21.15.31:0/2547568187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 msgr2=0x55f465a82820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 0x55f465a82820 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f170009a910 tx=0x7f170009a360 comp rx=0 tx=0).stop 2023-12-14T18:35:40.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 -- 172.21.15.31:0/2547568187 shutdown_connections 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a82d60 0x55f465a85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a75380 0x55f465a75750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/2547568187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75c90 0x55f465a82820 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.772+0000 7f1728ec8b80 1 -- 172.21.15.31:0/2547568187 >> 172.21.15.31:0/2547568187 conn(0x55f465a6bca0 msgr2=0x55f465a717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 -- 172.21.15.31:0/2547568187 shutdown_connections 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 -- 172.21.15.31:0/2547568187 wait complete. 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 -- start start 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 Processor -- start 2023-12-14T18:35:40.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a83130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a83670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a8e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465963f90 con 0x55f465a75380 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465963c30 con 0x55f465a82d60 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46584b6a0 con 0x55f465a75c90 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f1728ec8b80 10 client.0 _ll_register_callbacks cb 0x55f465987910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f171659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a83130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f171659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a83130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50250/0 (socket says 172.21.15.31:50250) 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.773+0000 7f171659c700 1 -- 172.21.15.31:0/1518505194 learned_addr learned my addr 172.21.15.31:0/1518505194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f1715d9b700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a83670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f171659c700 1 -- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 msgr2=0x55f465a83670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f171659c700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a83670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f1716d9d700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f171659c700 1 -- 172.21.15.31:0/1518505194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 msgr2=0x55f465a8e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f171659c700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a8e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f171659c700 1 -- 172.21.15.31:0/1518505194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171810f000 con 0x55f465a75380 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f171659c700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a83130 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f170801db70 tx=0x7f17080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa728901e14bccc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f16fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f16fdffb700 1 -- 172.21.15.31:0/1518505194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708009ee0 con 0x55f465a75380 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f16fdffb700 1 -- 172.21.15.31:0/1518505194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1708004260 con 0x55f465a75380 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f465a692b0 con 0x55f465a75380 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f16fdffb700 1 -- 172.21.15.31:0/1518505194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1708004460 con 0x55f465a75380 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f1728ec8b80 10 client.17331 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.774+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f46598e170 con 0x55f465a75380 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f16fdffb700 1 -- 172.21.15.31:0/1518505194 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f465a692b0 con 0x55f465a75380 2023-12-14T18:35:40.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f16fdffb700 1 -- 172.21.15.31:0/1518505194 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1708003c10 con 0x55f465a75380 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1728ec8b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1728ec8b80 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1728ec8b80 20 client.17331 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465a92450 0x55f465a94860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1728ec8b80 4 client.17331 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4659c0ce0 con 0x55f465a92450 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.775+0000 7f1715d9b700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465a92450 0x55f465a94860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f1715d9b700 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465a92450 0x55f465a94860 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e202728295519b30 server_cookie=a01724551e4ab0f9 in_seq=0 out_seq=0 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f16fdffb700 10 client.17331 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f16fdffb700 1 -- 172.21.15.31:0/1518505194 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4659c0ce0 con 0x55f465a92450 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f16fdffb700 10 client.17331 handle_command_reply: tid=0 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f1728ec8b80 1 client.17331 shutdown 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 msgr2=0x55f465a83130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.776+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a83130 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f170801db70 tx=0x7f17080047c0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 shutdown_connections 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465a92450 0x55f465a94860 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a8e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a83130 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f16fdffb700 0 client.17331 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/1518505194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a83670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 >> 172.21.15.31:0/1518505194 conn(0x55f465a6bca0 msgr2=0x55f465a71370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 shutdown_connections 2023-12-14T18:35:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 1 -- 172.21.15.31:0/1518505194 wait complete. 2023-12-14T18:35:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.777+0000 7f1728ec8b80 20 client.17331 trim_cache size 0 max 16384 2023-12-14T18:35:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 Processor -- start 2023-12-14T18:35:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 -- start start 2023-12-14T18:35:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a7d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a7d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465a765b0 con 0x55f465a75380 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465a7cd30 con 0x55f465a82d60 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.794+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465a839e0 con 0x55f465a75c90 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a88140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50256/0 (socket says 172.21.15.31:50256) 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 -- 172.21.15.31:0/3306209762 learned_addr learned my addr 172.21.15.31:0/3306209762 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f1715d9b700 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a7d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 -- 172.21.15.31:0/3306209762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 msgr2=0x55f465a7d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a7d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 -- 172.21.15.31:0/3306209762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 msgr2=0x55f465a7d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f1716d9d700 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a7d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a7d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 -- 172.21.15.31:0/3306209762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f465a82820 con 0x55f465a75380 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f171659c700 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a88140 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1708004020 tx=0x7f1708005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a881567f64a8f964 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f170d7fa700 1 -- 172.21.15.31:0/3306209762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708009ed0 con 0x55f465a75380 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f170d7fa700 1 -- 172.21.15.31:0/3306209762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f170801e6b0 con 0x55f465a75380 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.795+0000 7f170d7fa700 1 -- 172.21.15.31:0/3306209762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708003a40 con 0x55f465a75380 2023-12-14T18:35:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3306209762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 msgr2=0x55f465a88140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a88140 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f1708004020 tx=0x7f1708005c20 comp rx=0 tx=0).stop 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3306209762 shutdown_connections 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465a7d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a88140 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3306209762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a7d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3306209762 >> 172.21.15.31:0/3306209762 conn(0x55f465a6bca0 msgr2=0x55f465a6ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3306209762 shutdown_connections 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3306209762 wait complete. 2023-12-14T18:35:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 -- start start 2023-12-14T18:35:40.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 Processor -- start 2023-12-14T18:35:40.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.796+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a77f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465ae06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465ae0da0 con 0x55f465a75380 2023-12-14T18:35:40.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465ae0f10 con 0x55f465a82d60 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f465ae10d0 con 0x55f465a75c90 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 10 client.0 _ll_register_callbacks cb 0x7f170801d750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a77f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a77f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50270/0 (socket says 172.21.15.31:50270) 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 -- 172.21.15.31:0/3452259899 learned_addr learned my addr 172.21.15.31:0/3452259899 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1715d9b700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 -- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 msgr2=0x55f465a784c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a784c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 -- 172.21.15.31:0/3452259899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 msgr2=0x55f465ae06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:35:40.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465ae06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 -- 172.21.15.31:0/3452259899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f46598e170 con 0x55f465a75380 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1715d9b700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f171659c700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a77f80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1708009410 tx=0x7f170801ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b1e2b81672174e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f16fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f16fdffb700 1 -- 172.21.15.31:0/3452259899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1708003880 con 0x55f465a75380 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f16fdffb700 1 -- 172.21.15.31:0/3452259899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f17080039e0 con 0x55f465a75380 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.797+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1700098d20 con 0x55f465a75380 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f16fdffb700 1 -- 172.21.15.31:0/3452259899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f170801ed10 con 0x55f465a75380 2023-12-14T18:35:40.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 10 client.17337 fetch_fsmap learned FSMap version 28 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f4657f5b60 con 0x55f465a75380 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f16fdffb700 1 -- 172.21.15.31:0/3452259899 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1700098d20 con 0x55f465a75380 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f16fdffb700 1 -- 172.21.15.31:0/3452259899 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1708027470 con 0x55f465a75380 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 10 client.17337 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 20 client.17337 populate_metadata read hostname 'smithi031' 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465ae93d0 0x55f465aeb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 4 client.17337 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:35:40.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4659c0a00 con 0x55f465ae93d0 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.798+0000 7f1715d9b700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465ae93d0 0x55f465aeb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f16fdffb700 10 client.17337 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1715d9b700 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465ae93d0 0x55f465aeb7e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3049bdb3b1976d5 server_cookie=3cf5125a073c1dd7 in_seq=0 out_seq=0 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f16fdffb700 1 -- 172.21.15.31:0/3452259899 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f4659c0a00 con 0x55f465ae93d0 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f16fdffb700 10 client.17337 handle_command_reply: tid=0 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 client.17337 shutdown 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 msgr2=0x55f465a77f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a77f80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f1708009410 tx=0x7f170801ec00 comp rx=0 tx=0).stop 2023-12-14T18:35:40.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 shutdown_connections 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55f465ae93d0 0x55f465aeb7e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f465a82d60 0x55f465ae06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f465a75380 0x55f465a77f80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f16fdffb700 0 client.17337 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 --2- 172.21.15.31:0/3452259899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f465a75c90 0x55f465a784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 >> 172.21.15.31:0/3452259899 conn(0x55f465a6bca0 msgr2=0x55f465a709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 shutdown_connections 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 1 -- 172.21.15.31:0/3452259899 wait complete. 2023-12-14T18:35:40.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.799+0000 7f1728ec8b80 20 client.17337 trim_cache size 0 max 16384 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "55d16bb6-3ad0-490d-ae30-b258415a027b": { 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "55d16bb6-3ad0-490d-ae30-b258415a027b", 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:35:40.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:35:40.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:35:40.812 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:35:40.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17040a12c0 msgr2=0x7f17040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17040a12c0 0x7f17040a3770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1708009c60 tx=0x7f1708009260 comp rx=0 tx=0).stop 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 msgr2=0x7f1718195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718195740 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1700009fd0 tx=0x7f1700009580 comp rx=0 tx=0).stop 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 shutdown_connections 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1718104410 0x7f1718199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f17040a12c0 0x7f17040a3770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17180ff420 0x7f1718195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 --2- 172.21.15.31:0/1676737544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17180ffdc0 0x7f1718195740 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 >> 172.21.15.31:0/1676737544 conn(0x7f1718075fc0 msgr2=0x7f17180fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:35:40.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 shutdown_connections 2023-12-14T18:35:40.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:35:40.800+0000 7f171cd3a700 1 -- 172.21.15.31:0/1676737544 wait complete. 2023-12-14T18:35:40.815 INFO:tasks.cephfs.filesystem:scrub status for tag:55d16bb6-3ad0-490d-ae30-b258415a027b - {'path': '/', 'tag': '55d16bb6-3ad0-490d-ae30-b258415a027b', 'options': 'recursive,force'} 2023-12-14T18:35:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: pgmap v243: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:35:41.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:35:41.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:35:41.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:41 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: pgmap v243: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:41 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: pgmap v243: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:35:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:35:41.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:35:41.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:41.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:41 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:35:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:43 smithi204 ceph-mon[43866]: pgmap v244: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:43.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:43 smithi031 ceph-mon[42085]: pgmap v244: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:43 smithi176 ceph-mon[43506]: pgmap v244: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:45.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:45 smithi204 ceph-mon[43866]: pgmap v245: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:45.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:45 smithi031 ceph-mon[42085]: pgmap v245: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:45.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:45 smithi176 ceph-mon[43506]: pgmap v245: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:47 smithi204 ceph-mon[43866]: pgmap v246: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:47 smithi031 ceph-mon[42085]: pgmap v246: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:47 smithi176 ceph-mon[43506]: pgmap v246: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:49 smithi204 ceph-mon[43866]: pgmap v247: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:49.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:49 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:35:49.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:49 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:35:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:49 smithi031 ceph-mon[42085]: pgmap v247: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:49 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:35:49.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:49 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:35:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:49 smithi176 ceph-mon[43506]: pgmap v247: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:49 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:35:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:49 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:35:50.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:50 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:50.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:50 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:50.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:50 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:51 smithi204 ceph-mon[43866]: pgmap v248: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:51 smithi031 ceph-mon[42085]: pgmap v248: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:51.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:51 smithi176 ceph-mon[43506]: pgmap v248: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:53 smithi204 ceph-mon[43866]: pgmap v249: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:53.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:53 smithi031 ceph-mon[42085]: pgmap v249: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:53 smithi176 ceph-mon[43506]: pgmap v249: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:35:55.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:55 smithi204 ceph-mon[43866]: pgmap v250: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:55.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:55 smithi031 ceph-mon[42085]: pgmap v250: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:55 smithi176 ceph-mon[43506]: pgmap v250: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:35:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:57 smithi204 ceph-mon[43866]: pgmap v251: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:57.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:57 smithi031 ceph-mon[42085]: pgmap v251: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:57 smithi176 ceph-mon[43506]: pgmap v251: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:35:59 smithi204 ceph-mon[43866]: pgmap v252: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:35:59 smithi031 ceph-mon[42085]: pgmap v252: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:35:59 smithi176 ceph-mon[43506]: pgmap v252: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:01 smithi204 ceph-mon[43866]: pgmap v253: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:01.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:01 smithi031 ceph-mon[42085]: pgmap v253: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:01 smithi176 ceph-mon[43506]: pgmap v253: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:03 smithi204 ceph-mon[43866]: pgmap v254: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:03 smithi031 ceph-mon[42085]: pgmap v254: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:03 smithi176 ceph-mon[43506]: pgmap v254: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:05.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:05 smithi204 ceph-mon[43866]: pgmap v255: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:05.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:05 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:05.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:05 smithi031 ceph-mon[42085]: pgmap v255: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:05.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:05 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:05 smithi176 ceph-mon[43506]: pgmap v255: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:05 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:06.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:06 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:06.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:06 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:06.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:06 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:07 smithi204 ceph-mon[43866]: pgmap v256: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:36:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:07 smithi031 ceph-mon[42085]: pgmap v256: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:36:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:07 smithi176 ceph-mon[43506]: pgmap v256: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:36:09.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:09 smithi031 ceph-mon[42085]: pgmap v257: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:09.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:09 smithi176 ceph-mon[43506]: pgmap v257: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:10.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:09 smithi204 ceph-mon[43866]: pgmap v257: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:10.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:36:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:10 smithi031 ceph-mon[42085]: osdmap e95: 12 total, 12 up, 12 in 2023-12-14T18:36:10.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 -- 172.21.15.31:0/1052405861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e78105510 msgr2=0x7f7e7810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:10.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 --2- 172.21.15.31:0/1052405861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e78105510 0x7f7e7810d0e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000b0d0 tx=0x7f7e7000b3e0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 -- 172.21.15.31:0/1052405861 shutdown_connections 2023-12-14T18:36:10.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 --2- 172.21.15.31:0/1052405861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e78105510 0x7f7e7810d0e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 --2- 172.21.15.31:0/1052405861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e781004d0 0x7f7e781008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 --2- 172.21.15.31:0/1052405861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e78100e70 0x7f7e78104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 -- 172.21.15.31:0/1052405861 >> 172.21.15.31:0/1052405861 conn(0x7f7e780fbd40 msgr2=0x7f7e780fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 -- 172.21.15.31:0/1052405861 shutdown_connections 2023-12-14T18:36:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 -- 172.21.15.31:0/1052405861 wait complete. 2023-12-14T18:36:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 Processor -- start 2023-12-14T18:36:10.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.967+0000 7f7e7f860700 1 -- start start 2023-12-14T18:36:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7f860700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 0x7f7e78196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7f860700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e78100e70 0x7f7e781969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7f860700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78105510 0x7f7e7819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7f860700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7819b450 con 0x7f7e781004d0 2023-12-14T18:36:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7f860700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7819b5c0 con 0x7f7e78100e70 2023-12-14T18:36:10.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7f860700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7819b780 con 0x7f7e78105510 2023-12-14T18:36:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 0x7f7e78196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 0x7f7e78196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54314/0 (socket says 172.21.15.31:54314) 2023-12-14T18:36:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 -- 172.21.15.31:0/3020307395 learned_addr learned my addr 172.21.15.31:0/3020307395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e78100e70 0x7f7e781969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78105510 0x7f7e7819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 -- 172.21.15.31:0/3020307395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78105510 msgr2=0x7f7e7819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78105510 0x7f7e7819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 -- 172.21.15.31:0/3020307395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e78100e70 msgr2=0x7f7e781969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e78100e70 0x7f7e781969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 -- 172.21.15.31:0/3020307395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009d00 con 0x7f7e781004d0 2023-12-14T18:36:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 0x7f7e78196470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400d980 tx=0x7f7e7400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:10.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.968+0000 7f7e6e7fc700 1 -- 172.21.15.31:0/3020307395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400bb60 con 0x7f7e781004d0 2023-12-14T18:36:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.969+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e781102e0 con 0x7f7e781004d0 2023-12-14T18:36:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.969+0000 7f7e6e7fc700 1 -- 172.21.15.31:0/3020307395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e74007640 con 0x7f7e781004d0 2023-12-14T18:36:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.969+0000 7f7e6e7fc700 1 -- 172.21.15.31:0/3020307395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74004e90 con 0x7f7e781004d0 2023-12-14T18:36:10.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.969+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e78110150 con 0x7f7e781004d0 2023-12-14T18:36:10.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.971+0000 7f7e6e7fc700 1 -- 172.21.15.31:0/3020307395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7e7400bcc0 con 0x7f7e781004d0 2023-12-14T18:36:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.971+0000 7f7e6e7fc700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7e640a12c0 0x7f7e640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.972+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7e640a12c0 0x7f7e640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.972+0000 7f7e6e7fc700 1 -- 172.21.15.31:0/3020307395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7e740cd010 con 0x7f7e781004d0 2023-12-14T18:36:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.972+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7e640a12c0 0x7f7e640a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e78197a80 tx=0x7f7e6800d4b0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 Processor -- start 2023-12-14T18:36:10.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 -- start start 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8da610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c7b95e0 con 0x55661c8dab50 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c828870 con 0x55661c8cf380 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c821d00 con 0x55661c8cfc90 2023-12-14T18:36:10.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54320/0 (socket says 172.21.15.31:54320) 2023-12-14T18:36:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/3313571536 learned_addr learned my addr 172.21.15.31:0/3313571536 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8da610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/3313571536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 msgr2=0x55661c8da610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:10.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8da610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/3313571536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 msgr2=0x55661c8cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.973+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/3313571536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55661c6a33b0 con 0x55661c8dab50 2023-12-14T18:36:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.974+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dcf30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009c20 tx=0x7f7e70009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96544ca697d53fd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.974+0000 7f7e637fe700 1 -- 172.21.15.31:0/3313571536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000e070 con 0x55661c8dab50 2023-12-14T18:36:10.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.974+0000 7f7e637fe700 1 -- 172.21.15.31:0/3313571536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e70004a50 con 0x55661c8dab50 2023-12-14T18:36:10.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.974+0000 7f7e637fe700 1 -- 172.21.15.31:0/3313571536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70007910 con 0x55661c8dab50 2023-12-14T18:36:10.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/3313571536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 msgr2=0x55661c8dcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:10.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dcf30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009c20 tx=0x7f7e70009c50 comp rx=0 tx=0).stop 2023-12-14T18:36:10.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/3313571536 shutdown_connections 2023-12-14T18:36:10.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dcf30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/3313571536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8da610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/3313571536 >> 172.21.15.31:0/3313571536 conn(0x55661c8c5ca0 msgr2=0x55661c8cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:10.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/3313571536 shutdown_connections 2023-12-14T18:36:10.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/3313571536 wait complete. 2023-12-14T18:36:10.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 -- start start 2023-12-14T18:36:10.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.975+0000 7f7e8b9eeb80 1 Processor -- start 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c7bdf90 con 0x55661c8cfc90 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c7bdc30 con 0x55661c8cf380 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c6a56a0 con 0x55661c8dab50 2023-12-14T18:36:10.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e8b9eeb80 10 client.0 _ll_register_callbacks cb 0x55661c7e1910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:10.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8d7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54328/0 (socket says 172.21.15.31:54328) 2023-12-14T18:36:10.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1736788521 learned_addr learned my addr 172.21.15.31:0/1736788521 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:10.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 msgr2=0x55661c8d7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1736788521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 msgr2=0x55661c8da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1736788521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7810d920 con 0x55661c8cfc90 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8d7730 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d8b0 tx=0x7f7e68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97c49a2b79f429 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.976+0000 7f7e60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:10.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1736788521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68028670 con 0x55661c8cfc90 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1736788521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7e680287d0 con 0x55661c8cfc90 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55661c8c32b0 con 0x55661c8cfc90 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1736788521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e6800feb0 con 0x55661c8cfc90 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 10 client.17349 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55661c7e8170 con 0x55661c8cfc90 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1736788521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55661c8c32b0 con 0x55661c8cfc90 2023-12-14T18:36:10.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1736788521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7e6801fa60 con 0x55661c8cfc90 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 10 client.17349 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 10 client.17349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 20 client.17349 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c8ea4f0 0x55661c8ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 4 client.17349 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55661c81ace0 con 0x55661c8ea4f0 2023-12-14T18:36:10.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.977+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c8ea4f0 0x55661c8ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.978+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c8ea4f0 0x55661c8ec900 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec7cb28e2ba8c54 server_cookie=f4773e242f4870f3 in_seq=0 out_seq=0 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.978+0000 7f7e60ff9700 10 client.17349 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.978+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1736788521 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55661c81ace0 con 0x55661c8ea4f0 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.978+0000 7f7e60ff9700 10 client.17349 handle_command_reply: tid=0 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 client.17349 shutdown 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 msgr2=0x55661c8d7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8d7730 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d8b0 tx=0x7f7e68004750 comp rx=0 tx=0).stop 2023-12-14T18:36:10.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 shutdown_connections 2023-12-14T18:36:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c8ea4f0 0x55661c8ec900 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e60ff9700 0 client.17349 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1736788521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8d7730 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:10.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 >> 172.21.15.31:0/1736788521 conn(0x55661c8c5ca0 msgr2=0x55661c8cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 shutdown_connections 2023-12-14T18:36:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1736788521 wait complete. 2023-12-14T18:36:10.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.979+0000 7f7e8b9eeb80 20 client.17349 trim_cache size 0 max 16384 2023-12-14T18:36:10.999 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:10 smithi176 ceph-mon[43506]: osdmap e95: 12 total, 12 up, 12 in 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 Processor -- start 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 -- start start 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c8d0330 con 0x55661c8cfc90 2023-12-14T18:36:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c8d5c90 con 0x55661c8cf380 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c8d3580 con 0x55661c8dab50 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.997+0000 7f7e7cdfb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8da480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54338/0 (socket says 172.21.15.31:54338) 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1230436388 learned_addr learned my addr 172.21.15.31:0/1230436388 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1230436388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 msgr2=0x55661c8d76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1230436388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 msgr2=0x55661c8d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 -- 172.21.15.31:0/1230436388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55661c7e7ec0 con 0x55661c8cfc90 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8da480 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7e68014040 tx=0x7f7e6800d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1063274579ef5f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e63fff700 1 -- 172.21.15.31:0/1230436388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800d750 con 0x55661c8cfc90 2023-12-14T18:36:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e63fff700 1 -- 172.21.15.31:0/1230436388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e68011e70 con 0x55661c8cfc90 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.998+0000 7f7e63fff700 1 -- 172.21.15.31:0/1230436388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6801f7d0 con 0x55661c8cfc90 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1230436388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 msgr2=0x55661c8da480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8da480 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7e68014040 tx=0x7f7e6800d350 comp rx=0 tx=0).stop 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1230436388 shutdown_connections 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8dab50 0x55661c8d76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1230436388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8cfc90 0x55661c8da480 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1230436388 >> 172.21.15.31:0/1230436388 conn(0x55661c8c5ca0 msgr2=0x55661c8e6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1230436388 shutdown_connections 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1230436388 wait complete. 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 -- start start 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:10.999+0000 7f7e8b9eeb80 1 Processor -- start 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c8e0420 con 0x55661c8dab50 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c8e0560 con 0x55661c8cf380 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55661c8e0720 con 0x55661c8cfc90 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e8b9eeb80 10 client.0 _ll_register_callbacks cb 0x7f7e6800f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54346/0 (socket says 172.21.15.31:54346) 2023-12-14T18:36:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/1753975328 learned_addr learned my addr 172.21.15.31:0/1753975328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7cdfb700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 msgr2=0x55661c8df8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8df8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/1753975328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 msgr2=0x55661c8e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.000+0000 7f7e7ddfd700 1 -- 172.21.15.31:0/1753975328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6802d040 con 0x55661c8dab50 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e7ddfd700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dfe00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000cf90 tx=0x7f7e700077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9943e33d8aea9108 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e60ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1753975328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000e040 con 0x55661c8dab50 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1753975328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7e7001b640 con 0x55661c8dab50 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e70004a50 con 0x55661c8dab50 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1753975328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e70006000 con 0x55661c8dab50 2023-12-14T18:36:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 10 client.17355 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55661c7ce9a0 con 0x55661c8dab50 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1753975328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e70004a50 con 0x55661c8dab50 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1753975328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7e7002c070 con 0x55661c8dab50 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 10 client.17355 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 20 client.17355 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c93f6c0 0x55661c941ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 4 client.17355 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.001+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55661c81aa00 con 0x55661c93f6c0 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.002+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c93f6c0 0x55661c941ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.002+0000 7f7e60ff9700 10 client.17355 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.002+0000 7f7e7d5fc700 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c93f6c0 0x55661c941ad0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ceedc7797df8f35 server_cookie=bf9624ceb63f628c in_seq=0 out_seq=0 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.002+0000 7f7e60ff9700 1 -- 172.21.15.31:0/1753975328 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55661c81aa00 con 0x55661c93f6c0 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.002+0000 7f7e60ff9700 10 client.17355 handle_command_reply: tid=0 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.002+0000 7f7e8b9eeb80 1 client.17355 shutdown 2023-12-14T18:36:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 msgr2=0x55661c8dfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dfe00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000cf90 tx=0x7f7e700077c0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 shutdown_connections 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55661c93f6c0 0x55661c941ad0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55661c8dab50 0x55661c8dfe00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55661c8cf380 0x55661c8e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 --2- 172.21.15.31:0/1753975328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55661c8cfc90 0x55661c8df8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e60ff9700 0 client.17355 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:11.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 >> 172.21.15.31:0/1753975328 conn(0x55661c8c5ca0 msgr2=0x55661c8d76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 shutdown_connections 2023-12-14T18:36:11.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 1 -- 172.21.15.31:0/1753975328 wait complete. 2023-12-14T18:36:11.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e8b9eeb80 20 client.17355 trim_cache size 0 max 16384 2023-12-14T18:36:11.017 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:36:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:36:11.018 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:36:11.018 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:36:11.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7e640a12c0 msgr2=0x7f7e640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7e640a12c0 0x7f7e640a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7e78197a80 tx=0x7f7e6800d4b0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 msgr2=0x7f7e78196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 0x7f7e78196470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400d980 tx=0x7f7e7400dd40 comp rx=0 tx=0).stop 2023-12-14T18:36:11.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 shutdown_connections 2023-12-14T18:36:11.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7e640a12c0 0x7f7e640a3770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e78105510 0x7f7e7819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e781004d0 0x7f7e78196470 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 --2- 172.21.15.31:0/3020307395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7e78100e70 0x7f7e781969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.003+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 >> 172.21.15.31:0/3020307395 conn(0x7f7e780fbd40 msgr2=0x7f7e780fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.004+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 shutdown_connections 2023-12-14T18:36:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.004+0000 7f7e7f860700 1 -- 172.21.15.31:0/3020307395 wait complete. 2023-12-14T18:36:11.021 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:36:11.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:36:11.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:10 smithi204 ceph-mon[43866]: osdmap e95: 12 total, 12 up, 12 in 2023-12-14T18:36:11.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 -- 172.21.15.31:0/2157922666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3500ffd30 msgr2=0x7fb350103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 --2- 172.21.15.31:0/2157922666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3500ffd30 0x7fb350103cf0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb3400099c0 tx=0x7fb340009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 -- 172.21.15.31:0/2157922666 shutdown_connections 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 --2- 172.21.15.31:0/2157922666 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3501043d0 0x7fb35010bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 --2- 172.21.15.31:0/2157922666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3500ff390 0x7fb3500ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 --2- 172.21.15.31:0/2157922666 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3500ffd30 0x7fb350103cf0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 -- 172.21.15.31:0/2157922666 >> 172.21.15.31:0/2157922666 conn(0x7fb350076370 msgr2=0x7fb350076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 -- 172.21.15.31:0/2157922666 shutdown_connections 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.157+0000 7fb357b5c700 1 -- 172.21.15.31:0/2157922666 wait complete. 2023-12-14T18:36:11.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 Processor -- start 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 -- start start 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3500ff390 0x7fb350100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3500ffd30 0x7fb3501012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 0x7fb3501a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb350101b00 con 0x7fb3501043d0 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3501a4ad0 con 0x7fb3500ff390 2023-12-14T18:36:11.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.158+0000 7fb357b5c700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3501a4c10 con 0x7fb3500ffd30 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 0x7fb3501a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 0x7fb3501a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54354/0 (socket says 172.21.15.31:54354) 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 -- 172.21.15.31:0/55099750 learned_addr learned my addr 172.21.15.31:0/55099750 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3550f7700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3500ffd30 0x7fb3501012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3558f8700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3500ff390 0x7fb350100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 -- 172.21.15.31:0/55099750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3500ffd30 msgr2=0x7fb3501012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3500ffd30 0x7fb3501012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 -- 172.21.15.31:0/55099750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3500ff390 msgr2=0x7fb350100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3500ff390 0x7fb350100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 -- 172.21.15.31:0/55099750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400095f0 con 0x7fb3501043d0 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb3560f9700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 0x7fb3501a4590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb348007ee0 tx=0x7fb34800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348017070 con 0x7fb3501043d0 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35010f150 con 0x7fb3501043d0 2023-12-14T18:36:11.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb34800f040 con 0x7fb3501043d0 2023-12-14T18:36:11.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348005580 con 0x7fb3501043d0 2023-12-14T18:36:11.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.159+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35010efc0 con 0x7fb3501043d0 2023-12-14T18:36:11.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.160+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35010f2e0 con 0x7fb3501043d0 2023-12-14T18:36:11.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.162+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb34800db20 con 0x7fb3501043d0 2023-12-14T18:36:11.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.162+0000 7fb346ffd700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb33c0a12c0 0x7fb33c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.162+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb3480cce80 con 0x7fb3501043d0 2023-12-14T18:36:11.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.162+0000 7fb3558f8700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb33c0a12c0 0x7fb33c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.163+0000 7fb3558f8700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb33c0a12c0 0x7fb33c0a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009c60 tx=0x7fb34c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.166+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb348095e50 con 0x7fb3501043d0 2023-12-14T18:36:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.425+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3500679f0 con 0x7fb3501043d0 2023-12-14T18:36:11.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.425+0000 7fb346ffd700 1 -- 172.21.15.31:0/55099750 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10817 (secure 0 0 0) 0x7fb348099470 con 0x7fb3501043d0 2023-12-14T18:36:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:36:11.427 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:05.569769+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:36:11.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb33c0a12c0 msgr2=0x7fb33c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb33c0a12c0 0x7fb33c0a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009c60 tx=0x7fb34c009260 comp rx=0 tx=0).stop 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 msgr2=0x7fb3501a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 0x7fb3501a4590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb348007ee0 tx=0x7fb34800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 shutdown_connections 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3501043d0 0x7fb3501a4590 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3500ff390 0x7fb350100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb33c0a12c0 0x7fb33c0a3770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 --2- 172.21.15.31:0/55099750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3500ffd30 0x7fb3501012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 >> 172.21.15.31:0/55099750 conn(0x7fb350076370 msgr2=0x7fb350067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 shutdown_connections 2023-12-14T18:36:11.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.428+0000 7fb357b5c700 1 -- 172.21.15.31:0/55099750 wait complete. 2023-12-14T18:36:11.431 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 31 2023-12-14T18:36:11.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:36:11.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 -- 172.21.15.31:0/3066444594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5241043d0 msgr2=0x7fb52410bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 --2- 172.21.15.31:0/3066444594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5241043d0 0x7fb52410bfa0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00b0d0 tx=0x7fb51c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 -- 172.21.15.31:0/3066444594 shutdown_connections 2023-12-14T18:36:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 --2- 172.21.15.31:0/3066444594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5241043d0 0x7fb52410bfa0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 --2- 172.21.15.31:0/3066444594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5240ff390 0x7fb5240ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 --2- 172.21.15.31:0/3066444594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5240ffd30 0x7fb524103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 -- 172.21.15.31:0/3066444594 >> 172.21.15.31:0/3066444594 conn(0x7fb524075fd0 msgr2=0x7fb5240763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 -- 172.21.15.31:0/3066444594 shutdown_connections 2023-12-14T18:36:11.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.582+0000 7fb52a165700 1 -- 172.21.15.31:0/3066444594 wait complete. 2023-12-14T18:36:11.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 Processor -- start 2023-12-14T18:36:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 -- start start 2023-12-14T18:36:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5240ff390 0x7fb524196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 0x7fb524196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5241043d0 0x7fb52419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419b500 con 0x7fb5240ffd30 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419b670 con 0x7fb5240ff390 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb52a165700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52419b830 con 0x7fb5241043d0 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 0x7fb524196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 0x7fb524196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54372/0 (socket says 172.21.15.31:54372) 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb522ffd700 1 -- 172.21.15.31:0/806002130 learned_addr learned my addr 172.21.15.31:0/806002130 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb5237fe700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5240ff390 0x7fb524196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.583+0000 7fb523fff700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5241043d0 0x7fb52419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb522ffd700 1 -- 172.21.15.31:0/806002130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5241043d0 msgr2=0x7fb52419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb522ffd700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5241043d0 0x7fb52419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb522ffd700 1 -- 172.21.15.31:0/806002130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5240ff390 msgr2=0x7fb524196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb522ffd700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5240ff390 0x7fb524196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb522ffd700 1 -- 172.21.15.31:0/806002130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009520 con 0x7fb5240ffd30 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb522ffd700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 0x7fb524196a60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb51400e940 tx=0x7fb51400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb520ff9700 1 -- 172.21.15.31:0/806002130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514009730 con 0x7fb5240ffd30 2023-12-14T18:36:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c009d00 con 0x7fb5240ffd30 2023-12-14T18:36:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb520ff9700 1 -- 172.21.15.31:0/806002130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb514010430 con 0x7fb5240ffd30 2023-12-14T18:36:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb520ff9700 1 -- 172.21.15.31:0/806002130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51400cea0 con 0x7fb5240ffd30 2023-12-14T18:36:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.584+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb524110300 con 0x7fb5240ffd30 2023-12-14T18:36:11.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.587+0000 7fb520ff9700 1 -- 172.21.15.31:0/806002130 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb514010810 con 0x7fb5240ffd30 2023-12-14T18:36:11.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.587+0000 7fb520ff9700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb5100a12c0 0x7fb5100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb520ff9700 1 -- 172.21.15.31:0/806002130 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb514014070 con 0x7fb5240ffd30 2023-12-14T18:36:11.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5237fe700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb5100a12c0 0x7fb5100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5237fe700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb5100a12c0 0x7fb5100a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009df0 tx=0x7fb50c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 Processor -- start 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 -- start start 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 0x557c9aa58750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58c90 0x557c9aa67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9a9425e0 con 0x557c9aa58380 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9a9b1870 con 0x557c9aa58c90 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.588+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9a9aad00 con 0x557c9aa67f60 2023-12-14T18:36:11.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 0x557c9aa58750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 0x557c9aa58750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54382/0 (socket says 172.21.15.31:54382) 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 -- 172.21.15.31:0/1877713725 learned_addr learned my addr 172.21.15.31:0/1877713725 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb523fff700 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb522ffd700 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58c90 0x557c9aa67a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 -- 172.21.15.31:0/1877713725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 msgr2=0x557c9aa6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 -- 172.21.15.31:0/1877713725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58c90 msgr2=0x557c9aa67a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58c90 0x557c9aa67a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 -- 172.21.15.31:0/1877713725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c9a82c3b0 con 0x557c9aa58380 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb522ffd700 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58c90 0x557c9aa67a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 0x557c9aa58750 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0094f0 tx=0x7fb50c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3c1f223952c5c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb519ffb700 1 -- 172.21.15.31:0/1877713725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c026630 con 0x557c9aa58380 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb519ffb700 1 -- 172.21.15.31:0/1877713725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50c021070 con 0x557c9aa58380 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.589+0000 7fb519ffb700 1 -- 172.21.15.31:0/1877713725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c01d720 con 0x557c9aa58380 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1877713725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 msgr2=0x557c9aa58750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 0x557c9aa58750 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0094f0 tx=0x7fb50c014000 comp rx=0 tx=0).stop 2023-12-14T18:36:11.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1877713725 shutdown_connections 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58380 0x557c9aa58750 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1877713725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58c90 0x557c9aa67a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1877713725 >> 172.21.15.31:0/1877713725 conn(0x557c9aa4eca0 msgr2=0x557c9aa547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1877713725 shutdown_connections 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.590+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1877713725 wait complete. 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 -- start start 2023-12-14T18:36:11.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 Processor -- start 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa678f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9a946f90 con 0x557c9aa58c90 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9a946c30 con 0x557c9aa58380 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9a82e6a0 con 0x557c9aa67f60 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb5362f3b80 10 client.0 _ll_register_callbacks cb 0x557c9a96a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa64c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa64c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54392/0 (socket says 172.21.15.31:54392) 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb522ffd700 1 -- 172.21.15.31:0/3463048872 learned_addr learned my addr 172.21.15.31:0/3463048872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.591+0000 7fb523fff700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb5237fe700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa678f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb522ffd700 1 -- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 msgr2=0x557c9aa651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb522ffd700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb522ffd700 1 -- 172.21.15.31:0/3463048872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 msgr2=0x557c9aa678f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb522ffd700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa678f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb522ffd700 1 -- 172.21.15.31:0/3463048872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c017070 con 0x557c9aa58c90 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb522ffd700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa64c10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb514099c10 tx=0x7fb51409a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e9f33521160ebe server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb502ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb502ffd700 1 -- 172.21.15.31:0/3463048872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514099600 con 0x557c9aa58c90 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb502ffd700 1 -- 172.21.15.31:0/3463048872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb5140a2360 con 0x557c9aa58c90 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c9aa4c2b0 con 0x557c9aa58c90 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb502ffd700 1 -- 172.21.15.31:0/3463048872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5140a1ed0 con 0x557c9aa58c90 2023-12-14T18:36:11.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb5362f3b80 10 client.17373 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557c9a971170 con 0x557c9aa58c90 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb502ffd700 1 -- 172.21.15.31:0/3463048872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c9aa4c2b0 con 0x557c9aa58c90 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.592+0000 7fb502ffd700 1 -- 172.21.15.31:0/3463048872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb514095d00 con 0x557c9aa58c90 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5362f3b80 10 client.17373 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5362f3b80 10 client.17373 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5362f3b80 20 client.17373 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aa778c0 0x557c9aa79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5362f3b80 4 client.17373 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c9a9a3ce0 con 0x557c9aa778c0 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5237fe700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aa778c0 0x557c9aa79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb502ffd700 10 client.17373 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.593+0000 7fb5237fe700 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aa778c0 0x557c9aa79cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f94e67732fadba0 server_cookie=8fbc3472db406139 in_seq=0 out_seq=0 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb502ffd700 1 -- 172.21.15.31:0/3463048872 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c9a9a3ce0 con 0x557c9aa778c0 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb502ffd700 10 client.17373 handle_command_reply: tid=0 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 client.17373 shutdown 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 msgr2=0x557c9aa64c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa64c10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb514099c10 tx=0x7fb51409a000 comp rx=0 tx=0).stop 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 shutdown_connections 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aa778c0 0x557c9aa79cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa67f60 0x557c9aa651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa678f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/3463048872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa64c10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb502ffd700 0 client.17373 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 >> 172.21.15.31:0/3463048872 conn(0x557c9aa4eca0 msgr2=0x557c9aa543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 shutdown_connections 2023-12-14T18:36:11.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.594+0000 7fb5362f3b80 1 -- 172.21.15.31:0/3463048872 wait complete. 2023-12-14T18:36:11.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.595+0000 7fb5362f3b80 20 client.17373 trim_cache size 0 max 16384 2023-12-14T18:36:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 Processor -- start 2023-12-14T18:36:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 -- start start 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58c90 0x557c9aa73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 0x557c9aa673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9aa595e0 con 0x557c9aa67f60 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9aa55590 con 0x557c9aa58380 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9aa5b3d0 con 0x557c9aa58c90 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb523fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 0x557c9aa673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb523fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 0x557c9aa673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54398/0 (socket says 172.21.15.31:54398) 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb523fff700 1 -- 172.21.15.31:0/415528933 learned_addr learned my addr 172.21.15.31:0/415528933 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb522ffd700 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58c90 0x557c9aa73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.609+0000 7fb5237fe700 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb523fff700 1 -- 172.21.15.31:0/415528933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58c90 msgr2=0x557c9aa73270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb523fff700 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58c90 0x557c9aa73270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb523fff700 1 -- 172.21.15.31:0/415528933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 msgr2=0x557c9aa6ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb523fff700 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa6ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb523fff700 1 -- 172.21.15.31:0/415528933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c9a970ec0 con 0x557c9aa67f60 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb522ffd700 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58c90 0x557c9aa73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb523fff700 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 0x557c9aa673b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009420 tx=0x7fb51c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5179322611b34b27 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb51a7fc700 1 -- 172.21.15.31:0/415528933 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00e040 con 0x557c9aa67f60 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb51a7fc700 1 -- 172.21.15.31:0/415528933 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb51c00bca0 con 0x557c9aa67f60 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.610+0000 7fb51a7fc700 1 -- 172.21.15.31:0/415528933 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c005310 con 0x557c9aa67f60 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 -- 172.21.15.31:0/415528933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 msgr2=0x557c9aa673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 0x557c9aa673b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009420 tx=0x7fb51c00b040 comp rx=0 tx=0).stop 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 -- 172.21.15.31:0/415528933 shutdown_connections 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa67f60 0x557c9aa673b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa58380 0x557c9aa6ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/415528933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58c90 0x557c9aa73270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 -- 172.21.15.31:0/415528933 >> 172.21.15.31:0/415528933 conn(0x557c9aa4eca0 msgr2=0x557c9aa52f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 -- 172.21.15.31:0/415528933 shutdown_connections 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 -- 172.21.15.31:0/415528933 wait complete. 2023-12-14T18:36:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.611+0000 7fb5362f3b80 1 -- start start 2023-12-14T18:36:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 Processor -- start 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58380 0x557c9aa6e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa6ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa67f60 0x557c9aa6b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9aa6b700 con 0x557c9aa58c90 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9aa6b870 con 0x557c9aa67f60 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9aa6ba30 con 0x557c9aa58380 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5362f3b80 10 client.0 _ll_register_callbacks cb 0x7fb514099b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa6ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa6ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54406/0 (socket says 172.21.15.31:54406) 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 -- 172.21.15.31:0/1695680436 learned_addr learned my addr 172.21.15.31:0/1695680436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb523fff700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa67f60 0x557c9aa6b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58380 0x557c9aa6e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 -- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58380 msgr2=0x557c9aa6e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58380 0x557c9aa6e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 -- 172.21.15.31:0/1695680436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa67f60 msgr2=0x557c9aa6b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa67f60 0x557c9aa6b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb522ffd700 1 -- 172.21.15.31:0/1695680436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c9a971170 con 0x557c9aa58c90 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58380 0x557c9aa6e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.612+0000 7fb523fff700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa67f60 0x557c9aa6b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb522ffd700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa6ac80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb5140ca640 tx=0x7fb514099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4488e043da820eb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb502ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb502ffd700 1 -- 172.21.15.31:0/1695680436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514099ce0 con 0x557c9aa58c90 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb502ffd700 1 -- 172.21.15.31:0/1695680436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fb5140a2360 con 0x557c9aa58c90 2023-12-14T18:36:11.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb50c021070 con 0x557c9aa58c90 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb502ffd700 1 -- 172.21.15.31:0/1695680436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb514095d00 con 0x557c9aa58c90 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 10 client.17379 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fb524067dd0 con 0x557c9aa58c90 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb502ffd700 1 -- 172.21.15.31:0/1695680436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50c021070 con 0x557c9aa58c90 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb502ffd700 1 -- 172.21.15.31:0/1695680436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb514099400 con 0x557c9aa58c90 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 10 client.17379 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 10 client.17379 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 20 client.17379 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aacc7e0 0x557c9aacebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 4 client.17379 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.613+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557c9a9a3a00 con 0x557c9aacc7e0 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.614+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aacc7e0 0x557c9aacebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.614+0000 7fb5237fe700 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aacc7e0 0x557c9aacebf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cd9edf1fbc24f42 server_cookie=2718fef697b28228 in_seq=0 out_seq=0 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.614+0000 7fb502ffd700 10 client.17379 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.614+0000 7fb502ffd700 1 -- 172.21.15.31:0/1695680436 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557c9a9a3a00 con 0x557c9aacc7e0 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.614+0000 7fb502ffd700 10 client.17379 handle_command_reply: tid=0 2023-12-14T18:36:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.614+0000 7fb5362f3b80 1 client.17379 shutdown 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 msgr2=0x557c9aa6ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa6ac80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb5140ca640 tx=0x7fb514099f10 comp rx=0 tx=0).stop 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 shutdown_connections 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x557c9aacc7e0 0x557c9aacebf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557c9aa67f60 0x557c9aa6b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557c9aa58380 0x557c9aa6e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 --2- 172.21.15.31:0/1695680436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557c9aa58c90 0x557c9aa6ac80 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb502ffd700 0 client.17379 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 >> 172.21.15.31:0/1695680436 conn(0x557c9aa4eca0 msgr2=0x557c9aa5ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 shutdown_connections 2023-12-14T18:36:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 1 -- 172.21.15.31:0/1695680436 wait complete. 2023-12-14T18:36:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb5362f3b80 20 client.17379 trim_cache size 0 max 16384 2023-12-14T18:36:11.626 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:36:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb5100a12c0 msgr2=0x7fb5100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb5100a12c0 0x7fb5100a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009df0 tx=0x7fb50c009260 comp rx=0 tx=0).stop 2023-12-14T18:36:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 msgr2=0x7fb524196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 0x7fb524196a60 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb51400e940 tx=0x7fb51400ec50 comp rx=0 tx=0).stop 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 shutdown_connections 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb5100a12c0 0x7fb5100a3770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5241043d0 0x7fb52419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb5240ff390 0x7fb524196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 --2- 172.21.15.31:0/806002130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5240ffd30 0x7fb524196a60 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.615+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 >> 172.21.15.31:0/806002130 conn(0x7fb524075fd0 msgr2=0x7fb5240fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.616+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 shutdown_connections 2023-12-14T18:36:11.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.616+0000 7fb52a165700 1 -- 172.21.15.31:0/806002130 wait complete. 2023-12-14T18:36:11.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:36:11.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:11 smithi031 ceph-mon[42085]: pgmap v259: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:11.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:11 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/55099750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:36:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 -- 172.21.15.31:0/597874567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e280ffd70 msgr2=0x7f3e28103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/597874567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e280ffd70 0x7f3e28103d20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c0099c0 tx=0x7f3e1c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 -- 172.21.15.31:0/597874567 shutdown_connections 2023-12-14T18:36:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/597874567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e28104400 0x7f3e2810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/597874567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e280ff3d0 0x7f3e280ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/597874567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e280ffd70 0x7f3e28103d20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 -- 172.21.15.31:0/597874567 >> 172.21.15.31:0/597874567 conn(0x7f3e28075fa0 msgr2=0x7f3e280763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 -- 172.21.15.31:0/597874567 shutdown_connections 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.768+0000 7f3e2d26d700 1 -- 172.21.15.31:0/597874567 wait complete. 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 Processor -- start 2023-12-14T18:36:11.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 -- start start 2023-12-14T18:36:11.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e280ff3d0 0x7f3e281971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e280ffd70 0x7f3e28197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 0x7f3e281a2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e28198040 con 0x7f3e28104400 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e281a2780 con 0x7f3e280ff3d0 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.769+0000 7f3e2d26d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e281a2940 con 0x7f3e280ffd70 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 0x7f3e281a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 0x7f3e281a2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54422/0 (socket says 172.21.15.31:54422) 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 -- 172.21.15.31:0/2540437619 learned_addr learned my addr 172.21.15.31:0/2540437619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e267fc700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e280ffd70 0x7f3e28197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e280ff3d0 0x7f3e281971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 -- 172.21.15.31:0/2540437619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e280ffd70 msgr2=0x7f3e28197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e280ffd70 0x7f3e28197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 -- 172.21.15.31:0/2540437619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e280ff3d0 msgr2=0x7f3e281971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e280ff3d0 0x7f3e281971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 -- 172.21.15.31:0/2540437619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c0095f0 con 0x7f3e28104400 2023-12-14T18:36:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e277fe700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 0x7f3e281a2240 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000ef30 tx=0x7f3e2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e0ffff700 1 -- 172.21.15.31:0/2540437619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20018070 con 0x7f3e28104400 2023-12-14T18:36:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2810f180 con 0x7f3e28104400 2023-12-14T18:36:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e0ffff700 1 -- 172.21.15.31:0/2540437619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2000f040 con 0x7f3e28104400 2023-12-14T18:36:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e0ffff700 1 -- 172.21.15.31:0/2540437619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000cd00 con 0x7f3e28104400 2023-12-14T18:36:11.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.770+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2810eff0 con 0x7f3e28104400 2023-12-14T18:36:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.773+0000 7f3e0ffff700 1 -- 172.21.15.31:0/2540437619 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3e20003a00 con 0x7f3e28104400 2023-12-14T18:36:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.774+0000 7f3e0ffff700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3e100a12c0 0x7f3e100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.774+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3e100a12c0 0x7f3e100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.774+0000 7f3e0ffff700 1 -- 172.21.15.31:0/2540437619 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3e200ccc70 con 0x7f3e28104400 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.774+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3e100a12c0 0x7f3e100a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3e18005950 tx=0x7f3e1800b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 Processor -- start 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 -- start start 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298e4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627297ce5e0 con 0x5627298e4380 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56272983d870 con 0x5627298e4c90 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562729836d00 con 0x5627298efb50 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298e4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54428/0 (socket says 172.21.15.31:54428) 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e26ffd700 1 -- 172.21.15.31:0/299551330 learned_addr learned my addr 172.21.15.31:0/299551330 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e267fc700 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.775+0000 7f3e277fe700 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e26ffd700 1 -- 172.21.15.31:0/299551330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 msgr2=0x5627298f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e26ffd700 1 -- 172.21.15.31:0/299551330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 msgr2=0x5627298ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e26ffd700 1 -- 172.21.15.31:0/299551330 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627296b83b0 con 0x5627298e4380 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298e4750 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3e18010fa0 tx=0x7f3e18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a0c953ffa3b45 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e0d7fa700 1 -- 172.21.15.31:0/299551330 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1801e620 con 0x5627298e4380 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e0d7fa700 1 -- 172.21.15.31:0/299551330 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e18015070 con 0x5627298e4380 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.776+0000 7f3e0d7fa700 1 -- 172.21.15.31:0/299551330 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1801ca20 con 0x5627298e4380 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 -- 172.21.15.31:0/299551330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 msgr2=0x5627298e4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298e4750 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f3e18010fa0 tx=0x7f3e18013000 comp rx=0 tx=0).stop 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 -- 172.21.15.31:0/299551330 shutdown_connections 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298e4750 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/299551330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 -- 172.21.15.31:0/299551330 >> 172.21.15.31:0/299551330 conn(0x5627298daca0 msgr2=0x5627298e07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 -- 172.21.15.31:0/299551330 shutdown_connections 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 -- 172.21.15.31:0/299551330 wait complete. 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 -- start start 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.777+0000 7f3e393fbb80 1 Processor -- start 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ec7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627297d2f90 con 0x5627298e4380 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627297d2c30 con 0x5627298e4c90 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627296ba6a0 con 0x5627298efb50 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e393fbb80 10 client.0 _ll_register_callbacks cb 0x5627297f6910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298ef490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54434/0 (socket says 172.21.15.31:54434) 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1034456998 learned_addr learned my addr 172.21.15.31:0/1034456998 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e277fe700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ec7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1034456998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 msgr2=0x5627298ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298ecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 msgr2=0x5627298ec7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ec7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1034456998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e18010c50 con 0x5627298e4380 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298ef490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3e18010a40 tx=0x7f3e18005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d74f9a63f917208 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.778+0000 7f3e0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1034456998 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1802d070 con 0x5627298e4380 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5627298d82b0 con 0x5627298e4380 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1034456998 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3e18017070 con 0x5627298e4380 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1034456998 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e1800ba30 con 0x5627298e4380 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 10 client.17391 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5627297fd170 con 0x5627298e4380 2023-12-14T18:36:11.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1034456998 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5627298d82b0 con 0x5627298e4380 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1034456998 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3e1801e790 con 0x5627298e4380 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 10 client.17391 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 10 client.17391 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 20 client.17391 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5627298ff390 0x5627299017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 4 client.17391 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.779+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56272982fce0 con 0x5627298ff390 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.780+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5627298ff390 0x5627299017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.780+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5627298ff390 0x5627299017a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b35a4cbd8d61ec70 server_cookie=ad9be0cc731a5fe2 in_seq=0 out_seq=0 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.780+0000 7f3e0aff5700 10 client.17391 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1034456998 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56272982fce0 con 0x5627298ff390 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e0aff5700 10 client.17391 handle_command_reply: tid=0 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 client.17391 shutdown 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 msgr2=0x5627298ef490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298ef490 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f3e18010a40 tx=0x7f3e18005270 comp rx=0 tx=0).stop 2023-12-14T18:36:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 shutdown_connections 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5627298ff390 0x5627299017a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298ef490 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1034456998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298ec7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e0aff5700 0 client.17391 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 >> 172.21.15.31:0/1034456998 conn(0x5627298daca0 msgr2=0x5627298e03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 shutdown_connections 2023-12-14T18:36:11.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1034456998 wait complete. 2023-12-14T18:36:11.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.781+0000 7f3e393fbb80 20 client.17391 trim_cache size 0 max 16384 2023-12-14T18:36:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 Processor -- start 2023-12-14T18:36:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 -- start start 2023-12-14T18:36:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298def30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627298e8450 con 0x5627298e4380 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627298e8600 con 0x5627298e4c90 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627298f6bf0 con 0x5627298efb50 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54438/0 (socket says 172.21.15.31:54438) 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1009851925 learned_addr learned my addr 172.21.15.31:0/1009851925 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298def30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e277fe700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1009851925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 msgr2=0x5627298fb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298fb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1009851925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 msgr2=0x5627298def30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298def30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e26ffd700 1 -- 172.21.15.31:0/1009851925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627297fcec0 con 0x5627298e4380 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298def30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.795+0000 7f3e277fe700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298f66b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800b510 tx=0x7f3e1801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae1d7bb76adfdfc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e0dffb700 1 -- 172.21.15.31:0/1009851925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1802d030 con 0x5627298e4380 2023-12-14T18:36:11.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e0dffb700 1 -- 172.21.15.31:0/1009851925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e18005390 con 0x5627298e4380 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e0dffb700 1 -- 172.21.15.31:0/1009851925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18005720 con 0x5627298e4380 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1009851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 msgr2=0x5627298f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298f66b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800b510 tx=0x7f3e1801ec90 comp rx=0 tx=0).stop 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1009851925 shutdown_connections 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298fb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4380 0x5627298f66b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1009851925 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4c90 0x5627298def30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1009851925 >> 172.21.15.31:0/1009851925 conn(0x5627298daca0 msgr2=0x562729906100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1009851925 shutdown_connections 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.796+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1009851925 wait complete. 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 -- start start 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 Processor -- start 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4380 0x5627298f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 0x5627298f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x562729952320 con 0x5627298e4c90 2023-12-14T18:36:11.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x562729952460 con 0x5627298e4380 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562729952620 con 0x5627298efb50 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e393fbb80 10 client.0 _ll_register_callbacks cb 0x7f3e18010e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e267fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 0x5627298f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e267fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 0x5627298f2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54452/0 (socket says 172.21.15.31:54452) 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e267fc700 1 -- 172.21.15.31:0/1294602590 learned_addr learned my addr 172.21.15.31:0/1294602590 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4380 0x5627298f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.797+0000 7f3e277fe700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e267fc700 1 -- 172.21.15.31:0/1294602590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 msgr2=0x5627298f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e267fc700 1 -- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4380 msgr2=0x5627298f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4380 0x5627298f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e267fc700 1 -- 172.21.15.31:0/1294602590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627297fd170 con 0x5627298e4c90 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e267fc700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 0x5627298f2bd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009990 tx=0x7f3e1c00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b719bd711ad41b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1294602590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01d040 con 0x5627298e4c90 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56272982fa00 con 0x5627298e4c90 2023-12-14T18:36:11.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1294602590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f3e1c029d30 con 0x5627298e4c90 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1294602590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e1c029160 con 0x5627298e4c90 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e393fbb80 10 client.17397 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5627297e39a0 con 0x5627298e4c90 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.798+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1294602590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56272982fa00 con 0x5627298e4c90 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1294602590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f3e1c0037d0 con 0x5627298e4c90 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e393fbb80 10 client.17397 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e393fbb80 10 client.17397 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e393fbb80 20 client.17397 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562729953d00 0x562729956110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e393fbb80 4 client.17397 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562729685cf0 con 0x562729953d00 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562729953d00 0x562729956110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e0aff5700 10 client.17397 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e26ffd700 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562729953d00 0x562729956110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7f5c3509000045e server_cookie=4c672ea262b1f771 in_seq=0 out_seq=0 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e0aff5700 1 -- 172.21.15.31:0/1294602590 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562729685cf0 con 0x562729953d00 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.799+0000 7f3e0aff5700 10 client.17397 handle_command_reply: tid=0 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 client.17397 shutdown 2023-12-14T18:36:11.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 msgr2=0x5627298f2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 0x5627298f2bd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009990 tx=0x7f3e1c00bb80 comp rx=0 tx=0).stop 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 shutdown_connections 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x562729953d00 0x562729956110 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5627298efb50 0x5627298f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e0aff5700 0 client.17397 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5627298e4380 0x5627298f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 --2- 172.21.15.31:0/1294602590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5627298e4c90 0x5627298f2bd0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 >> 172.21.15.31:0/1294602590 conn(0x5627298daca0 msgr2=0x5627298def30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 shutdown_connections 2023-12-14T18:36:11.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 1 -- 172.21.15.31:0/1294602590 wait complete. 2023-12-14T18:36:11.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e393fbb80 20 client.17397 trim_cache size 0 max 16384 2023-12-14T18:36:11.812 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:36:11.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3e100a12c0 msgr2=0x7f3e100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3e100a12c0 0x7f3e100a3770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3e18005950 tx=0x7f3e1800b410 comp rx=0 tx=0).stop 2023-12-14T18:36:11.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 msgr2=0x7f3e281a2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 0x7f3e281a2240 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000ef30 tx=0x7f3e2000c3a0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 shutdown_connections 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e28104400 0x7f3e281a2240 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3e280ff3d0 0x7f3e281971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f3e100a12c0 0x7f3e100a3770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 --2- 172.21.15.31:0/2540437619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e280ffd70 0x7f3e28197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.800+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 >> 172.21.15.31:0/2540437619 conn(0x7f3e28075fa0 msgr2=0x7f3e280fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.801+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 shutdown_connections 2023-12-14T18:36:11.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.801+0000 7f3e2d26d700 1 -- 172.21.15.31:0/2540437619 wait complete. 2023-12-14T18:36:11.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:36:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 -- 172.21.15.31:0/2623379156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 msgr2=0x7fbbe4101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2623379156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4101020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0099c0 tx=0x7fbbcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 -- 172.21.15.31:0/2623379156 shutdown_connections 2023-12-14T18:36:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2623379156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbe4105550 0x7fbbe410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2623379156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4101020 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2623379156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbe41015f0 0x7fbbe4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 -- 172.21.15.31:0/2623379156 >> 172.21.15.31:0/2623379156 conn(0x7fbbe4077570 msgr2=0x7fbbe4077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 -- 172.21.15.31:0/2623379156 shutdown_connections 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.953+0000 7fbbe988d700 1 -- 172.21.15.31:0/2623379156 wait complete. 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 Processor -- start 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 -- start start 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbe41015f0 0x7fbbe4196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbe4105550 0x7fbbe419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe419b430 con 0x7fbbe4100c50 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe419b5a0 con 0x7fbbe41015f0 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe988d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe419b760 con 0x7fbbe4105550 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54474/0 (socket says 172.21.15.31:54474) 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2404917721 learned_addr learned my addr 172.21.15.31:0/2404917721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.954+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbe4105550 0x7fbbe419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbe41015f0 0x7fbbe4196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2404917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbe4105550 msgr2=0x7fbbe419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbe4105550 0x7fbbe419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2404917721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbe41015f0 msgr2=0x7fbbe4196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbe41015f0 0x7fbbe4196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2404917721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0095f0 con 0x7fbbe4100c50 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4196450 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00b580 tx=0x7fbbcc00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe888b700 1 -- 172.21.15.31:0/2404917721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc01d070 con 0x7fbbe4100c50 2023-12-14T18:36:11.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe888b700 1 -- 172.21.15.31:0/2404917721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbcc022470 con 0x7fbbe4100c50 2023-12-14T18:36:11.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe41102c0 con 0x7fbbe4100c50 2023-12-14T18:36:11.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe888b700 1 -- 172.21.15.31:0/2404917721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc017650 con 0x7fbbe4100c50 2023-12-14T18:36:11.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.955+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4110130 con 0x7fbbe4100c50 2023-12-14T18:36:11.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.957+0000 7fbbe888b700 1 -- 172.21.15.31:0/2404917721 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbbcc0225e0 con 0x7fbbe4100c50 2023-12-14T18:36:11.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.958+0000 7fbbe888b700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbd00a12c0 0x7fbbd00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.958+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbd00a12c0 0x7fbbd00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.958+0000 7fbbe888b700 1 -- 172.21.15.31:0/2404917721 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fbbcc0ce150 con 0x7fbbe4100c50 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbd00a12c0 0x7fbbd00a3770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4197a60 tx=0x7fbbd4009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 Processor -- start 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 -- start start 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7380 0x55dc21cf7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d02610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21d04f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21be15e0 con 0x55dc21d02b50 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21c50870 con 0x55dc21cf7380 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21c49d00 con 0x55dc21cf7c90 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21d04f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21d04f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54490/0 (socket says 172.21.15.31:54490) 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbe37fe700 1 -- 172.21.15.31:0/2585548532 learned_addr learned my addr 172.21.15.31:0/2585548532 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d02610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.959+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7380 0x55dc21cf7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe37fe700 1 -- 172.21.15.31:0/2585548532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 msgr2=0x55dc21d02610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d02610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe37fe700 1 -- 172.21.15.31:0/2585548532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7380 msgr2=0x55dc21cf7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7380 0x55dc21cf7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe37fe700 1 -- 172.21.15.31:0/2585548532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc21acb3b0 con 0x55dc21d02b50 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7380 0x55dc21cf7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21d04f30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc011680 tx=0x7fbbdc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1305ca14f6157e3c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbd9ffb700 1 -- 172.21.15.31:0/2585548532 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00e040 con 0x55dc21d02b50 2023-12-14T18:36:11.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbd9ffb700 1 -- 172.21.15.31:0/2585548532 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbdc00f040 con 0x55dc21d02b50 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.960+0000 7fbbd9ffb700 1 -- 172.21.15.31:0/2585548532 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc004860 con 0x55dc21d02b50 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2585548532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 msgr2=0x55dc21d04f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21d04f30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc011680 tx=0x7fbbdc009920 comp rx=0 tx=0).stop 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2585548532 shutdown_connections 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21d04f30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7380 0x55dc21cf7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2585548532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d02610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2585548532 >> 172.21.15.31:0/2585548532 conn(0x55dc21cedca0 msgr2=0x55dc21cf37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2585548532 shutdown_connections 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2585548532 wait complete. 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.961+0000 7fbbf5a1bb80 1 -- start start 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 Processor -- start 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7380 0x55dc21d024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21cff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21cffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21be5f90 con 0x55dc21d02b50 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21be5c30 con 0x55dc21cf7c90 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21acd6a0 con 0x55dc21cf7380 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbf5a1bb80 10 client.0 _ll_register_callbacks cb 0x55dc21c09910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21cffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21cffd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54504/0 (socket says 172.21.15.31:54504) 2023-12-14T18:36:11.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 -- 172.21.15.31:0/608850117 learned_addr learned my addr 172.21.15.31:0/608850117 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7380 0x55dc21d024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21cff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 -- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7380 msgr2=0x55dc21d024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7380 0x55dc21d024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 -- 172.21.15.31:0/608850117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 msgr2=0x55dc21cff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21cff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe37fe700 1 -- 172.21.15.31:0/608850117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc011330 con 0x55dc21d02b50 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.962+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7380 0x55dc21d024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21cffd20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc012010 tx=0x7fbbdc0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f3e061a453260a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/608850117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00e070 con 0x55dc21d02b50 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/608850117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fbbdc003c10 con 0x55dc21d02b50 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc21ceb2b0 con 0x55dc21d02b50 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/608850117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbdc003dd0 con 0x55dc21d02b50 2023-12-14T18:36:11.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 10 client.17409 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55dc21c10170 con 0x55dc21d02b50 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/608850117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc21ceb2b0 con 0x55dc21d02b50 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/608850117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbbdc015450 con 0x55dc21d02b50 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 10 client.17409 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 20 client.17409 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d16880 0x55dc21d18c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 4 client.17409 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.963+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc21c42ce0 con 0x55dc21d16880 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.964+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d16880 0x55dc21d18c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.964+0000 7fbbc2ffd700 10 client.17409 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.964+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d16880 0x55dc21d18c90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6dae109bc70a82e server_cookie=ca9b81d32465e04e in_seq=0 out_seq=0 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/608850117 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dc21c42ce0 con 0x55dc21d16880 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbc2ffd700 10 client.17409 handle_command_reply: tid=0 2023-12-14T18:36:11.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 client.17409 shutdown 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 msgr2=0x55dc21cffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21cffd20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc012010 tx=0x7fbbdc0095e0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 shutdown_connections 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d16880 0x55dc21d18c90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21d02b50 0x55dc21cffd20 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7380 0x55dc21d024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/608850117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21cff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 >> 172.21.15.31:0/608850117 conn(0x55dc21cedca0 msgr2=0x55dc21cf33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbc2ffd700 0 client.17409 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 shutdown_connections 2023-12-14T18:36:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/608850117 wait complete. 2023-12-14T18:36:11.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.965+0000 7fbbf5a1bb80 20 client.17409 trim_cache size 0 max 16384 2023-12-14T18:36:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 Processor -- start 2023-12-14T18:36:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 -- start start 2023-12-14T18:36:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cf1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d01f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21d02b50 0x55dc21d024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21d00300 con 0x55dc21cf7380 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21cfdd00 con 0x55dc21d02b50 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.979+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21cfb610 con 0x55dc21cf7c90 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cf1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cf1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54518/0 (socket says 172.21.15.31:54518) 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2675322611 learned_addr learned my addr 172.21.15.31:0/2675322611 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d01f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21d02b50 0x55dc21d024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2675322611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 msgr2=0x55dc21d01f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d01f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2675322611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21d02b50 msgr2=0x55dc21d024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21d02b50 0x55dc21d024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/2675322611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc21cff2f0 con 0x55dc21cf7380 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d01f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21d02b50 0x55dc21d024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:11.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cf1f30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc000c00 tx=0x7fbbcc01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bd3aedc5dd3c0a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbda7fc700 1 -- 172.21.15.31:0/2675322611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc09eea0 con 0x55dc21cf7380 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbda7fc700 1 -- 172.21.15.31:0/2675322611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbcc0a3230 con 0x55dc21cf7380 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.980+0000 7fbbda7fc700 1 -- 172.21.15.31:0/2675322611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc0a2da0 con 0x55dc21cf7380 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2675322611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 msgr2=0x55dc21cf1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cf1f30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc000c00 tx=0x7fbbcc01a040 comp rx=0 tx=0).stop 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2675322611 shutdown_connections 2023-12-14T18:36:11.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21d02b50 0x55dc21d024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cf1f30 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/2675322611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21cf7c90 0x55dc21d01f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2675322611 >> 172.21.15.31:0/2675322611 conn(0x55dc21cedca0 msgr2=0x55dc21d12c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2675322611 shutdown_connections 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/2675322611 wait complete. 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.981+0000 7fbbf5a1bb80 1 -- start start 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 Processor -- start 2023-12-14T18:36:11.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cfd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21d0bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21d02b50 0x55dc21d0c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21d0c580 con 0x55dc21cf7380 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21d0c6f0 con 0x55dc21cf7c90 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc21d0c8b0 con 0x55dc21d02b50 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbf5a1bb80 10 client.0 _ll_register_callbacks cb 0x7fbbd40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:11.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cfd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cfd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54526/0 (socket says 172.21.15.31:54526) 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/319232049 learned_addr learned my addr 172.21.15.31:0/319232049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21d0bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe37fe700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21d02b50 0x55dc21d0c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21d02b50 msgr2=0x55dc21d0c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21d02b50 0x55dc21d0c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/319232049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 msgr2=0x55dc21d0bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21d0bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.982+0000 7fbbe2ffd700 1 -- 172.21.15.31:0/319232049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe4110130 con 0x55dc21cf7380 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbe2ffd700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cfd280 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009fd0 tx=0x7fbbcc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a5432d5342828 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/319232049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc09a360 con 0x55dc21cf7380 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/319232049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fbbcc09a4c0 con 0x55dc21cf7380 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbdc00f040 con 0x55dc21cf7380 2023-12-14T18:36:11.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/319232049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbcc0a2da0 con 0x55dc21cf7380 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 10 client.17415 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55dc21a77b60 con 0x55dc21cf7380 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/319232049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbbdc00f040 con 0x55dc21cf7380 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/319232049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbbcc0cbcd0 con 0x55dc21cf7380 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 10 client.17415 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 20 client.17415 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d6a5a0 0x55dc21d6c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:11.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 4 client.17415 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.983+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc21c42a00 con 0x55dc21d6a5a0 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d6a5a0 0x55dc21d6c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbe27fc700 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d6a5a0 0x55dc21d6c980 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37a4f2d4fdf1ecf2 server_cookie=a38d8d623646b948 in_seq=0 out_seq=0 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbc2ffd700 10 client.17415 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbc2ffd700 1 -- 172.21.15.31:0/319232049 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc21c42a00 con 0x55dc21d6a5a0 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbc2ffd700 10 client.17415 handle_command_reply: tid=0 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbf5a1bb80 1 client.17415 shutdown 2023-12-14T18:36:11.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 msgr2=0x55dc21cfd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.984+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cfd280 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009fd0 tx=0x7fbbcc005e80 comp rx=0 tx=0).stop 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 shutdown_connections 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55dc21d6a5a0 0x55dc21d6c980 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc21d02b50 0x55dc21d0c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55dc21cf7380 0x55dc21cfd280 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbc2ffd700 0 client.17415 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 --2- 172.21.15.31:0/319232049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55dc21cf7c90 0x55dc21d0bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 >> 172.21.15.31:0/319232049 conn(0x55dc21cedca0 msgr2=0x55dc21cf1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 shutdown_connections 2023-12-14T18:36:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 1 -- 172.21.15.31:0/319232049 wait complete. 2023-12-14T18:36:11.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbf5a1bb80 20 client.17415 trim_cache size 0 max 16384 2023-12-14T18:36:11.998 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:36:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbd00a12c0 msgr2=0x7fbbd00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbd00a12c0 0x7fbbd00a3770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4197a60 tx=0x7fbbd4009260 comp rx=0 tx=0).stop 2023-12-14T18:36:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 msgr2=0x7fbbe4196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4196450 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00b580 tx=0x7fbbcc00bfd0 comp rx=0 tx=0).stop 2023-12-14T18:36:11.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 shutdown_connections 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fbbd00a12c0 0x7fbbd00a3770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbe4105550 0x7fbbe419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbe4100c50 0x7fbbe4196450 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 --2- 172.21.15.31:0/2404917721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbbe41015f0 0x7fbbe4196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 >> 172.21.15.31:0/2404917721 conn(0x7fbbe4077570 msgr2=0x7fbbe40feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.985+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 shutdown_connections 2023-12-14T18:36:12.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:11.986+0000 7fbbe988d700 1 -- 172.21.15.31:0/2404917721 wait complete. 2023-12-14T18:36:12.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:11 smithi176 ceph-mon[43506]: pgmap v259: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:12.001 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:11 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/55099750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:36:12.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:11 smithi204 ceph-mon[43866]: pgmap v259: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:12.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:11 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/55099750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:36:13.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:36:13.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:36:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.152+0000 7f12adfd1700 1 -- 172.21.15.31:0/3274214476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074740 msgr2=0x7f12a8072d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.152+0000 7f12adfd1700 1 --2- 172.21.15.31:0/3274214476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074740 0x7f12a8072d20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1298009990 tx=0x7f1298009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.152+0000 7f12adfd1700 1 -- 172.21.15.31:0/3274214476 shutdown_connections 2023-12-14T18:36:13.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.152+0000 7f12adfd1700 1 --2- 172.21.15.31:0/3274214476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12a8073940 0x7f12a810ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.153+0000 7f12adfd1700 1 --2- 172.21.15.31:0/3274214476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074740 0x7f12a8072d20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.153+0000 7f12adfd1700 1 --2- 172.21.15.31:0/3274214476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8074b10 0x7f12a8073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.153+0000 7f12adfd1700 1 -- 172.21.15.31:0/3274214476 >> 172.21.15.31:0/3274214476 conn(0x7f12a8077570 msgr2=0x7f12a8077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.153+0000 7f12adfd1700 1 -- 172.21.15.31:0/3274214476 shutdown_connections 2023-12-14T18:36:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.153+0000 7f12adfd1700 1 -- 172.21.15.31:0/3274214476 wait complete. 2023-12-14T18:36:13.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.153+0000 7f12adfd1700 1 Processor -- start 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 -- start start 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 0x7f12a8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12a8074740 0x7f12a8198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 0x7f12a819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a819d640 con 0x7f12a8074b10 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a819d7b0 con 0x7f12a8073940 2023-12-14T18:36:13.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12adfd1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a819d970 con 0x7f12a8074740 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 0x7f12a819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 0x7f12a8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 0x7f12a8198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:53874/0 (socket says 172.21.15.31:53874) 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 -- 172.21.15.31:0/1159619607 learned_addr learned my addr 172.21.15.31:0/1159619607 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 0x7f12a819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54546/0 (socket says 172.21.15.31:54546) 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12a8074740 0x7f12a8198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 -- 172.21.15.31:0/1159619607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12a8074740 msgr2=0x7f12a8198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12a8074740 0x7f12a8198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 -- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 msgr2=0x7f12a819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 0x7f12a819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a77fe700 1 -- 172.21.15.31:0/1159619607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980095f0 con 0x7f12a8073940 2023-12-14T18:36:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.154+0000 7f12a7fff700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 0x7f12a819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.155+0000 7f12a77fe700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 0x7f12a8198660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1298005910 tx=0x7f1298004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.155+0000 7f12a4ff9700 1 -- 172.21.15.31:0/1159619607 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129801d070 con 0x7f12a8073940 2023-12-14T18:36:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.155+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a81125c0 con 0x7f12a8073940 2023-12-14T18:36:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.155+0000 7f12a4ff9700 1 -- 172.21.15.31:0/1159619607 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1298022470 con 0x7f12a8073940 2023-12-14T18:36:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.155+0000 7f12a4ff9700 1 -- 172.21.15.31:0/1159619607 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298017650 con 0x7f12a8073940 2023-12-14T18:36:13.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.155+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a8112430 con 0x7f12a8073940 2023-12-14T18:36:13.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.158+0000 7f12a4ff9700 1 -- 172.21.15.31:0/1159619607 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f129800fad0 con 0x7f12a8073940 2023-12-14T18:36:13.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.158+0000 7f12a4ff9700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12900a1340 0x7f12900a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a4ff9700 1 -- 172.21.15.31:0/1159619607 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f12980ce670 con 0x7f12a8073940 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12900a1340 0x7f12900a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 Processor -- start 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 -- start start 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328dc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f328e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f327c6660 con 0x556f328e7bd0 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f328358f0 con 0x556f328dc400 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f3282ed80 con 0x556f328dcd10 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f328e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12900a1340 0x7f12900a37f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12a8199c70 tx=0x7f129c009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f328e9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54562/0 (socket says 172.21.15.31:54562) 2023-12-14T18:36:13.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a7fff700 1 -- 172.21.15.31:0/445617463 learned_addr learned my addr 172.21.15.31:0/445617463 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a77fe700 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328dc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.159+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a7fff700 1 -- 172.21.15.31:0/445617463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 msgr2=0x556f328e7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a7fff700 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a7fff700 1 -- 172.21.15.31:0/445617463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 msgr2=0x556f328dc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a7fff700 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328dc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a7fff700 1 -- 172.21.15.31:0/445617463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f326b0430 con 0x556f328e7bd0 2023-12-14T18:36:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a77fe700 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328dc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f12a7fff700 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f328e9fb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f12a00078d0 tx=0x7f12a0007be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4517a647132286a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f128dffb700 1 -- 172.21.15.31:0/445617463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0004bb0 con 0x556f328e7bd0 2023-12-14T18:36:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f128dffb700 1 -- 172.21.15.31:0/445617463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a000f040 con 0x556f328e7bd0 2023-12-14T18:36:13.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.160+0000 7f128dffb700 1 -- 172.21.15.31:0/445617463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a0005370 con 0x556f328e7bd0 2023-12-14T18:36:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 -- 172.21.15.31:0/445617463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 msgr2=0x556f328e9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f328e9fb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f12a00078d0 tx=0x7f12a0007be0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 -- 172.21.15.31:0/445617463 shutdown_connections 2023-12-14T18:36:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f328e9fb0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328dc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/445617463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 -- 172.21.15.31:0/445617463 >> 172.21.15.31:0/445617463 conn(0x556f328d2d20 msgr2=0x556f328d8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 -- 172.21.15.31:0/445617463 shutdown_connections 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 -- 172.21.15.31:0/445617463 wait complete. 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 -- start start 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.161+0000 7f12ba15fb80 1 Processor -- start 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 0x556f328e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328e7bd0 0x556f328e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f327cb010 con 0x556f328dc400 2023-12-14T18:36:13.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f327cacb0 con 0x556f328e7bd0 2023-12-14T18:36:13.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f326b2720 con 0x556f328dcd10 2023-12-14T18:36:13.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12ba15fb80 10 client.0 _ll_register_callbacks cb 0x556f327ee990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:13.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 0x556f328e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 0x556f328e7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54578/0 (socket says 172.21.15.31:54578) 2023-12-14T18:36:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 -- 172.21.15.31:0/1992481914 learned_addr learned my addr 172.21.15.31:0/1992481914 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a7fff700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328e7bd0 0x556f328e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 -- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 msgr2=0x556f328e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 -- 172.21.15.31:0/1992481914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328e7bd0 msgr2=0x556f328e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328e7bd0 0x556f328e4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 -- 172.21.15.31:0/1992481914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a0007500 con 0x556f328dc400 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a7fff700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328e7bd0 0x556f328e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f12a77fe700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 0x556f328e7550 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f129809b2c0 tx=0x7f129809afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cff785a9c6fbab4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f1286ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:13.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f1286ffd700 1 -- 172.21.15.31:0/1992481914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129809f360 con 0x556f328dc400 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.162+0000 7f1286ffd700 1 -- 172.21.15.31:0/1992481914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f12980a36f0 con 0x556f328dc400 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f328d0330 con 0x556f328dc400 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f1286ffd700 1 -- 172.21.15.31:0/1992481914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f129809f070 con 0x556f328dc400 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 10 client.17427 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556f327f51f0 con 0x556f328dc400 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f1286ffd700 1 -- 172.21.15.31:0/1992481914 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556f328d0330 con 0x556f328dc400 2023-12-14T18:36:13.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f1286ffd700 1 -- 172.21.15.31:0/1992481914 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f129809a820 con 0x556f328dc400 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 10 client.17427 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 20 client.17427 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f328f7180 0x556f328f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 4 client.17427 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f32827d60 con 0x556f328f7180 2023-12-14T18:36:13.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.163+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f328f7180 0x556f328f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.164+0000 7f1286ffd700 10 client.17427 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.164+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f328f7180 0x556f328f9590 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5eebf64a2f77e8a server_cookie=cd7c3eb87a05ee19 in_seq=0 out_seq=0 2023-12-14T18:36:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.164+0000 7f1286ffd700 1 -- 172.21.15.31:0/1992481914 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556f32827d60 con 0x556f328f7180 2023-12-14T18:36:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.164+0000 7f1286ffd700 10 client.17427 handle_command_reply: tid=0 2023-12-14T18:36:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.164+0000 7f12ba15fb80 1 client.17427 shutdown 2023-12-14T18:36:13.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 msgr2=0x556f328e7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 0x556f328e7550 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f129809b2c0 tx=0x7f129809afa0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 shutdown_connections 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f328f7180 0x556f328f9590 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328e7bd0 0x556f328e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dc400 0x556f328e7550 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f1286ffd700 0 client.17427 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/1992481914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 >> 172.21.15.31:0/1992481914 conn(0x556f328d2d20 msgr2=0x556f328d8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 shutdown_connections 2023-12-14T18:36:13.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 1 -- 172.21.15.31:0/1992481914 wait complete. 2023-12-14T18:36:13.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.165+0000 7f12ba15fb80 20 client.17427 trim_cache size 0 max 16384 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 Processor -- start 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 -- start start 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328e7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 0x556f328e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328e7bd0 0x556f328e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f328dd410 con 0x556f328dcd10 2023-12-14T18:36:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f328dd630 con 0x556f328dc400 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.183+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f328d9c40 con 0x556f328e7bd0 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 0x556f328e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 0x556f328e7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54588/0 (socket says 172.21.15.31:54588) 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 -- 172.21.15.31:0/2184574943 learned_addr learned my addr 172.21.15.31:0/2184574943 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a7fff700 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328e7bd0 0x556f328e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a77fe700 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 -- 172.21.15.31:0/2184574943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328e7bd0 msgr2=0x556f328e1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328e7bd0 0x556f328e1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 -- 172.21.15.31:0/2184574943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 msgr2=0x556f328e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328e7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 -- 172.21.15.31:0/2184574943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f328d4560 con 0x556f328dcd10 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 0x556f328e7550 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f129c005950 tx=0x7f129c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09fd27036e5dceb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f128e7fc700 1 -- 172.21.15.31:0/2184574943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c024070 con 0x556f328dcd10 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f128e7fc700 1 -- 172.21.15.31:0/2184574943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f129c025040 con 0x556f328dcd10 2023-12-14T18:36:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.184+0000 7f128e7fc700 1 -- 172.21.15.31:0/2184574943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c028780 con 0x556f328dcd10 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 -- 172.21.15.31:0/2184574943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 msgr2=0x556f328e7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 0x556f328e7550 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f129c005950 tx=0x7f129c014000 comp rx=0 tx=0).stop 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 -- 172.21.15.31:0/2184574943 shutdown_connections 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328e7bd0 0x556f328e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/2184574943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328dcd10 0x556f328e7550 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 -- 172.21.15.31:0/2184574943 >> 172.21.15.31:0/2184574943 conn(0x556f328d2d20 msgr2=0x556f328e3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 -- 172.21.15.31:0/2184574943 shutdown_connections 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 -- 172.21.15.31:0/2184574943 wait complete. 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 -- start start 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.185+0000 7f12ba15fb80 1 Processor -- start 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f32945020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f32945560 con 0x556f328e7bd0 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f329456a0 con 0x556f328dc400 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f32945860 con 0x556f328dcd10 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12ba15fb80 10 client.0 _ll_register_callbacks cb 0x556f327ee990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f32945020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f32945020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54590/0 (socket says 172.21.15.31:54590) 2023-12-14T18:36:13.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 -- 172.21.15.31:0/3430492348 learned_addr learned my addr 172.21.15.31:0/3430492348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a6ffd700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a77fe700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 -- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 msgr2=0x556f328edaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328edaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 -- 172.21.15.31:0/3430492348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 msgr2=0x556f328ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328ed5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 -- 172.21.15.31:0/3430492348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c017070 con 0x556f328e7bd0 2023-12-14T18:36:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f12a7fff700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f32945020 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f12a0015ed0 tx=0x7f12a0015880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155acf9224780dc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.186+0000 7f1286ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f1286ffd700 1 -- 172.21.15.31:0/3430492348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00095e0 con 0x556f328e7bd0 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f1286ffd700 1 -- 172.21.15.31:0/3430492348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f12a0015c00 con 0x556f328e7bd0 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12a000f040 con 0x556f328e7bd0 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f1286ffd700 1 -- 172.21.15.31:0/3430492348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a00050c0 con 0x556f328e7bd0 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 10 client.17433 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556f3265cbe0 con 0x556f328e7bd0 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f1286ffd700 1 -- 172.21.15.31:0/3430492348 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12a000f040 con 0x556f328e7bd0 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f1286ffd700 1 -- 172.21.15.31:0/3430492348 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f12a0027700 con 0x556f328e7bd0 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 10 client.17433 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 20 client.17433 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f3294cfa0 0x556f3294f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 4 client.17433 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:36:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.187+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556f32827a80 con 0x556f3294cfa0 2023-12-14T18:36:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.188+0000 7f12a77fe700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f3294cfa0 0x556f3294f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.188+0000 7f12a77fe700 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f3294cfa0 0x556f3294f3b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3daa6fae0e6839 server_cookie=efb0a431ad1892ce in_seq=0 out_seq=0 2023-12-14T18:36:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.188+0000 7f1286ffd700 10 client.17433 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f1286ffd700 1 -- 172.21.15.31:0/3430492348 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556f32827a80 con 0x556f3294cfa0 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f1286ffd700 10 client.17433 handle_command_reply: tid=0 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 client.17433 shutdown 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 msgr2=0x556f32945020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f32945020 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f12a0015ed0 tx=0x7f12a0015880 comp rx=0 tx=0).stop 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 shutdown_connections 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x556f3294cfa0 0x556f3294f3b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556f328e7bd0 0x556f32945020 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f1286ffd700 0 client.17433 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x556f328dc400 0x556f328ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 --2- 172.21.15.31:0/3430492348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556f328dcd10 0x556f328edaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.189+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 >> 172.21.15.31:0/3430492348 conn(0x556f328d2d20 msgr2=0x556f328de2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 shutdown_connections 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12ba15fb80 1 -- 172.21.15.31:0/3430492348 wait complete. 2023-12-14T18:36:13.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12ba15fb80 20 client.17433 trim_cache size 0 max 16384 2023-12-14T18:36:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:36:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:36:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "938b64f0-48b3-4d20-bca1-0a36d272fb4f", 2023-12-14T18:36:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:36:13.203 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:36:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12900a1340 msgr2=0x7f12900a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12900a1340 0x7f12900a37f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12a8199c70 tx=0x7f129c009260 comp rx=0 tx=0).stop 2023-12-14T18:36:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 msgr2=0x7f12a8198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 0x7f12a8198660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1298005910 tx=0x7f1298004730 comp rx=0 tx=0).stop 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 shutdown_connections 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12a8074b10 0x7f12a819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f12a8073940 0x7f12a8198660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12900a1340 0x7f12900a37f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 --2- 172.21.15.31:0/1159619607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12a8074740 0x7f12a8198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 >> 172.21.15.31:0/1159619607 conn(0x7f12a8077570 msgr2=0x7f12a8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 shutdown_connections 2023-12-14T18:36:13.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.190+0000 7f12adfd1700 1 -- 172.21.15.31:0/1159619607 wait complete. 2023-12-14T18:36:13.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:36:13.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 -- 172.21.15.31:0/1685549596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c101d50 msgr2=0x7f026c10a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 --2- 172.21.15.31:0/1685549596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c101d50 0x7f026c10a240 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f025c0098f0 tx=0x7f025c009c00 comp rx=0 tx=0).stop 2023-12-14T18:36:13.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 -- 172.21.15.31:0/1685549596 shutdown_connections 2023-12-14T18:36:13.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 --2- 172.21.15.31:0/1685549596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c10a810 0x7f026c10cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 --2- 172.21.15.31:0/1685549596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f026c101440 0x7f026c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 --2- 172.21.15.31:0/1685549596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c101d50 0x7f026c10a240 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 -- 172.21.15.31:0/1685549596 >> 172.21.15.31:0/1685549596 conn(0x7f026c076230 msgr2=0x7f026c076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 -- 172.21.15.31:0/1685549596 shutdown_connections 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.347+0000 7f0273381700 1 -- 172.21.15.31:0/1685549596 wait complete. 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 Processor -- start 2023-12-14T18:36:13.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 -- start start 2023-12-14T18:36:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 0x7f026c0ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f026c101d50 0x7f026c0ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c10a810 0x7f026c1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c0ffe00 con 0x7f026c101440 2023-12-14T18:36:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c1a4aa0 con 0x7f026c101d50 2023-12-14T18:36:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f0273381700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026c1a4be0 con 0x7f026c10a810 2023-12-14T18:36:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f027111d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 0x7f026c0ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f027111d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 0x7f026c0ff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54602/0 (socket says 172.21.15.31:54602) 2023-12-14T18:36:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f027111d700 1 -- 172.21.15.31:0/59391144 learned_addr learned my addr 172.21.15.31:0/59391144 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f027091c700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f026c101d50 0x7f026c0ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.348+0000 7f027191e700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c10a810 0x7f026c1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f027111d700 1 -- 172.21.15.31:0/59391144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c10a810 msgr2=0x7f026c1a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f027111d700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c10a810 0x7f026c1a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f027111d700 1 -- 172.21.15.31:0/59391144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f026c101d50 msgr2=0x7f026c0ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f027111d700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f026c101d50 0x7f026c0ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f027111d700 1 -- 172.21.15.31:0/59391144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009520 con 0x7f026c101440 2023-12-14T18:36:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f027111d700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 0x7f026c0ff090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f026800e9a0 tx=0x7f026800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f02627fc700 1 -- 172.21.15.31:0/59391144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268009790 con 0x7f026c101440 2023-12-14T18:36:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f02627fc700 1 -- 172.21.15.31:0/59391144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0268010430 con 0x7f026c101440 2023-12-14T18:36:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026c110290 con 0x7f026c101440 2023-12-14T18:36:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f02627fc700 1 -- 172.21.15.31:0/59391144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268018680 con 0x7f026c101440 2023-12-14T18:36:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.349+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c110100 con 0x7f026c101440 2023-12-14T18:36:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.352+0000 7f02627fc700 1 -- 172.21.15.31:0/59391144 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f02680105a0 con 0x7f026c101440 2023-12-14T18:36:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.353+0000 7f02627fc700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f02580a12c0 0x7f02580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.353+0000 7f027091c700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f02580a12c0 0x7f02580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.353+0000 7f02627fc700 1 -- 172.21.15.31:0/59391144 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f0268014070 con 0x7f026c101440 2023-12-14T18:36:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.353+0000 7f027091c700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f02580a12c0 0x7f02580a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f025c009fd0 tx=0x7f025c019040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.353+0000 7f027f50fb80 1 Processor -- start 2023-12-14T18:36:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 -- start start 2023-12-14T18:36:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bff380 0x55aef5bff750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5ae95e0 con 0x55aef5c0ab50 2023-12-14T18:36:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5b58870 con 0x55aef5bff380 2023-12-14T18:36:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027f50fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5b51d00 con 0x55aef5bffc90 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54608/0 (socket says 172.21.15.31:54608) 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 -- 172.21.15.31:0/46221358 learned_addr learned my addr 172.21.15.31:0/46221358 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027091c700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027111d700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bff380 0x55aef5bff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 -- 172.21.15.31:0/46221358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 msgr2=0x55aef5c0a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c0a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 -- 172.21.15.31:0/46221358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bff380 msgr2=0x55aef5bff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bff380 0x55aef5bff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 -- 172.21.15.31:0/46221358 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef59d33b0 con 0x55aef5c0ab50 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027111d700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bff380 0x55aef5bff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027091c700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.354+0000 7f027191e700 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0cf30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0264011600 tx=0x7f02640099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7900a0f24cddef server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f02577fe700 1 -- 172.21.15.31:0/46221358 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026400e040 con 0x55aef5c0ab50 2023-12-14T18:36:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f02577fe700 1 -- 172.21.15.31:0/46221358 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026400f040 con 0x55aef5c0ab50 2023-12-14T18:36:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f02577fe700 1 -- 172.21.15.31:0/46221358 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02640046c0 con 0x55aef5c0ab50 2023-12-14T18:36:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 -- 172.21.15.31:0/46221358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 msgr2=0x55aef5c0cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0cf30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0264011600 tx=0x7f02640099e0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 -- 172.21.15.31:0/46221358 shutdown_connections 2023-12-14T18:36:13.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0cf30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bff380 0x55aef5bff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 --2- 172.21.15.31:0/46221358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 -- 172.21.15.31:0/46221358 >> 172.21.15.31:0/46221358 conn(0x55aef5bf5ca0 msgr2=0x55aef5bfb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.355+0000 7f027f50fb80 1 -- 172.21.15.31:0/46221358 shutdown_connections 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.356+0000 7f027f50fb80 1 -- 172.21.15.31:0/46221358 wait complete. 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.356+0000 7f027f50fb80 1 -- start start 2023-12-14T18:36:13.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.356+0000 7f027f50fb80 1 Processor -- start 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.356+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 0x55aef5c0a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c07730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c07c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5aedf90 con 0x55aef5bff380 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5aedc30 con 0x55aef5c0ab50 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef59d56a0 con 0x55aef5bffc90 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 10 client.0 _ll_register_callbacks cb 0x55aef5b11910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 0x55aef5c0a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 0x55aef5c0a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54622/0 (socket says 172.21.15.31:54622) 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 -- 172.21.15.31:0/3420235651 learned_addr learned my addr 172.21.15.31:0/3420235651 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027091c700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c07730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027191e700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c07c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 -- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 msgr2=0x55aef5c07730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c07730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 -- 172.21.15.31:0/3420235651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 msgr2=0x55aef5c07c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c07c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 -- 172.21.15.31:0/3420235651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02640112b0 con 0x55aef5bff380 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027111d700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 0x55aef5c0a480 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0268099cb0 tx=0x7f026809dfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c12215e647d7ad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f0254ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f0254ff9700 1 -- 172.21.15.31:0/3420235651 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680996a0 con 0x55aef5bff380 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f0254ff9700 1 -- 172.21.15.31:0/3420235651 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f02680a2400 con 0x55aef5bff380 2023-12-14T18:36:13.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.357+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aef5bf32b0 con 0x55aef5bff380 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f0254ff9700 1 -- 172.21.15.31:0/3420235651 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02680a1f70 con 0x55aef5bff380 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 10 client.17445 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55aef5b18170 con 0x55aef5bff380 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f0254ff9700 1 -- 172.21.15.31:0/3420235651 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aef5bf32b0 con 0x55aef5bff380 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f0254ff9700 1 -- 172.21.15.31:0/3420235651 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0268099030 con 0x55aef5bff380 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 10 client.17445 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:13.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 20 client.17445 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c1a3d0 0x55aef5c1c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 4 client.17445 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aef5b4ace0 con 0x55aef5c1a3d0 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.358+0000 7f027091c700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c1a3d0 0x55aef5c1c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.359+0000 7f027091c700 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c1a3d0 0x55aef5c1c7b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d83db740172a1e server_cookie=137d93ff6508dc00 in_seq=0 out_seq=0 2023-12-14T18:36:13.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.359+0000 7f0254ff9700 10 client.17445 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.359+0000 7f0254ff9700 1 -- 172.21.15.31:0/3420235651 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aef5b4ace0 con 0x55aef5c1a3d0 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.359+0000 7f0254ff9700 10 client.17445 handle_command_reply: tid=0 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.359+0000 7f027f50fb80 1 client.17445 shutdown 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.359+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 msgr2=0x55aef5c0a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 0x55aef5c0a480 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f0268099cb0 tx=0x7f026809dfe0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 shutdown_connections 2023-12-14T18:36:13.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c1a3d0 0x55aef5c1c7b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c07c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bff380 0x55aef5c0a480 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f0254ff9700 0 client.17445 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3420235651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bffc90 0x55aef5c07730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 >> 172.21.15.31:0/3420235651 conn(0x55aef5bf5ca0 msgr2=0x55aef5bfb370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 shutdown_connections 2023-12-14T18:36:13.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 1 -- 172.21.15.31:0/3420235651 wait complete. 2023-12-14T18:36:13.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.360+0000 7f027f50fb80 20 client.17445 trim_cache size 0 max 16384 2023-12-14T18:36:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 Processor -- start 2023-12-14T18:36:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 -- start start 2023-12-14T18:36:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 0x55aef5c15e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5c005b0 con 0x55aef5bffc90 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5bff1c0 con 0x55aef5c0ab50 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027f50fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5c05c90 con 0x55aef5bff380 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 0x55aef5c15e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 0x55aef5c15e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54630/0 (socket says 172.21.15.31:54630) 2023-12-14T18:36:13.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 -- 172.21.15.31:0/3222816761 learned_addr learned my addr 172.21.15.31:0/3222816761 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027111d700 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027191e700 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 -- 172.21.15.31:0/3222816761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 msgr2=0x55aef5c116c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c116c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 -- 172.21.15.31:0/3222816761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 msgr2=0x55aef5c09f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c09f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 -- 172.21.15.31:0/3222816761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aef5b17ec0 con 0x55aef5bffc90 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027191e700 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.378+0000 7f027091c700 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 0x55aef5c15e60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8de2a4f1ff888 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f0257fff700 1 -- 172.21.15.31:0/3222816761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c031960 con 0x55aef5bffc90 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f0257fff700 1 -- 172.21.15.31:0/3222816761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f025c02c040 con 0x55aef5bffc90 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f0257fff700 1 -- 172.21.15.31:0/3222816761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c005300 con 0x55aef5bffc90 2023-12-14T18:36:13.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 -- 172.21.15.31:0/3222816761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 msgr2=0x55aef5c15e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 0x55aef5c15e60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f025c000c00 tx=0x7f025c021000 comp rx=0 tx=0).stop 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 -- 172.21.15.31:0/3222816761 shutdown_connections 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5c0ab50 0x55aef5c09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 --2- 172.21.15.31:0/3222816761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5bffc90 0x55aef5c15e60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 -- 172.21.15.31:0/3222816761 >> 172.21.15.31:0/3222816761 conn(0x55aef5bf5ca0 msgr2=0x55aef5c167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 -- 172.21.15.31:0/3222816761 shutdown_connections 2023-12-14T18:36:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.379+0000 7f027f50fb80 1 -- 172.21.15.31:0/3222816761 wait complete. 2023-12-14T18:36:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 -- start start 2023-12-14T18:36:13.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 Processor -- start 2023-12-14T18:36:13.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c11640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bffc90 0x55aef5c0f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5c10480 con 0x55aef5c0ab50 2023-12-14T18:36:13.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5c105f0 con 0x55aef5bffc90 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aef5c107b0 con 0x55aef5bff380 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027f50fb80 10 client.0 _ll_register_callbacks cb 0x7f0268099910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.380+0000 7f027191e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54644/0 (socket says 172.21.15.31:54644) 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 -- 172.21.15.31:0/7933954 learned_addr learned my addr 172.21.15.31:0/7933954 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027091c700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bffc90 0x55aef5c0f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027111d700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c11640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 -- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 msgr2=0x55aef5c11640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c11640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 -- 172.21.15.31:0/7933954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bffc90 msgr2=0x55aef5c0f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bffc90 0x55aef5c0f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 -- 172.21.15.31:0/7933954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026c10d960 con 0x55aef5c0ab50 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027091c700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bffc90 0x55aef5c0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027191e700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0feb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f02640042e0 tx=0x7f02640055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9260cdc5d9747ef server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f0254ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:13.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f0254ff9700 1 -- 172.21.15.31:0/7933954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026400e040 con 0x55aef5c0ab50 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f0254ff9700 1 -- 172.21.15.31:0/7933954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f0264003a00 con 0x55aef5c0ab50 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f026400f040 con 0x55aef5c0ab50 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f0254ff9700 1 -- 172.21.15.31:0/7933954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0264005a70 con 0x55aef5c0ab50 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027f50fb80 10 client.17451 fetch_fsmap learned FSMap version 31 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.381+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55aef5afe9a0 con 0x55aef5c0ab50 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f0254ff9700 1 -- 172.21.15.31:0/7933954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f026400f040 con 0x55aef5c0ab50 2023-12-14T18:36:13.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f0254ff9700 1 -- 172.21.15.31:0/7933954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0264024020 con 0x55aef5c0ab50 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027f50fb80 10 client.17451 fetch_fsmap finished waiting for FSMap version 31 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027f50fb80 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027f50fb80 20 client.17451 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027f50fb80 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c6f6f0 0x55aef5c71b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027f50fb80 4 client.17451 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aef5b4aa00 con 0x55aef5c6f6f0 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.382+0000 7f027111d700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c6f6f0 0x55aef5c71b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:13.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027111d700 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c6f6f0 0x55aef5c71b00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb41ef6d5eac12a server_cookie=3f9c9ad986831591 in_seq=0 out_seq=0 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f0254ff9700 10 client.17451 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f0254ff9700 1 -- 172.21.15.31:0/7933954 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aef5b4aa00 con 0x55aef5c6f6f0 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f0254ff9700 10 client.17451 handle_command_reply: tid=0 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 client.17451 shutdown 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 msgr2=0x55aef5c0feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0feb0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f02640042e0 tx=0x7f02640055b0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 shutdown_connections 2023-12-14T18:36:13.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55aef5c6f6f0 0x55aef5c71b00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55aef5c0ab50 0x55aef5c0feb0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aef5bff380 0x55aef5c11640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f0254ff9700 0 client.17451 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 --2- 172.21.15.31:0/7933954 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55aef5bffc90 0x55aef5c0f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 >> 172.21.15.31:0/7933954 conn(0x55aef5bf5ca0 msgr2=0x55aef5c15e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 shutdown_connections 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 1 -- 172.21.15.31:0/7933954 wait complete. 2023-12-14T18:36:13.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.383+0000 7f027f50fb80 20 client.17451 trim_cache size 0 max 16384 2023-12-14T18:36:13.397 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:36:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:36:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:36:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "938b64f0-48b3-4d20-bca1-0a36d272fb4f": { 2023-12-14T18:36:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:36:13.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "938b64f0-48b3-4d20-bca1-0a36d272fb4f", 2023-12-14T18:36:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:36:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:36:13.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:36:13.398 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:36:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f02580a12c0 msgr2=0x7f02580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f02580a12c0 0x7f02580a3770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f025c009fd0 tx=0x7f025c019040 comp rx=0 tx=0).stop 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 msgr2=0x7f026c0ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 0x7f026c0ff090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f026800e9a0 tx=0x7f026800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 shutdown_connections 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f02580a12c0 0x7f02580a3770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f026c10a810 0x7f026c1a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f026c101440 0x7f026c0ff090 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 --2- 172.21.15.31:0/59391144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f026c101d50 0x7f026c0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 >> 172.21.15.31:0/59391144 conn(0x7f026c076230 msgr2=0x7f026c0fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 shutdown_connections 2023-12-14T18:36:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:13.384+0000 7f0273381700 1 -- 172.21.15.31:0/59391144 wait complete. 2023-12-14T18:36:13.401 INFO:tasks.cephfs.filesystem:scrub status for tag:938b64f0-48b3-4d20-bca1-0a36d272fb4f - {'path': '/', 'tag': '938b64f0-48b3-4d20-bca1-0a36d272fb4f', 'options': 'recursive,force'} 2023-12-14T18:36:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:13 smithi031 ceph-mon[42085]: pgmap v260: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:13 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:36:13.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:13 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:13 smithi176 ceph-mon[43506]: pgmap v260: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:13.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:13 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:36:13.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:13 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:14.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:13 smithi204 ceph-mon[43866]: pgmap v260: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:14.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:13 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:36:14.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:13 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:14 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:36:14.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:14 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:14.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:14 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:36:14.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:14 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:14 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:36:14.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:14 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:14.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:14 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:36:14.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:14 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:15.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:14 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:36:15.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:14 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:15.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:14 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:36:15.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:14 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:15.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:15 smithi031 ceph-mon[42085]: pgmap v261: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:15.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:15 smithi176 ceph-mon[43506]: pgmap v261: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:16.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:15 smithi204 ceph-mon[43866]: pgmap v261: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:16 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:16 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:36:16.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:16 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:36:16.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:16 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:36:16.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:16 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:36:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:16 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:16 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:36:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:16 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:36:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:16 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:36:16.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:16 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:36:17.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:16 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:17.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:16 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:36:17.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:16 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:36:17.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:16 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:36:17.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:16 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:36:17.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:17 smithi031 ceph-mon[42085]: pgmap v262: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:17.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:17 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:36:17.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:17 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:36:17.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:17 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:36:17.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:17 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:36:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:17 smithi176 ceph-mon[43506]: pgmap v262: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:17 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:36:17.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:17 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:36:17.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:17 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:36:17.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:17 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:36:18.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:17 smithi204 ceph-mon[43866]: pgmap v262: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:18.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:17 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:36:18.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:17 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:36:18.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:17 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:36:18.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:17 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:36:19.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:19 smithi204 ceph-mon[43866]: pgmap v263: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:19.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:19 smithi204 ceph-mon[43866]: osdmap e96: 12 total, 12 up, 12 in 2023-12-14T18:36:19.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:19 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:36:19.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:19 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:36:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:19 smithi031 ceph-mon[42085]: pgmap v263: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:19 smithi031 ceph-mon[42085]: osdmap e96: 12 total, 12 up, 12 in 2023-12-14T18:36:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:19 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:36:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:19 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:36:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:19 smithi176 ceph-mon[43506]: pgmap v263: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:19 smithi176 ceph-mon[43506]: osdmap e96: 12 total, 12 up, 12 in 2023-12-14T18:36:19.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:19 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:36:19.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:19 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:36:21.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:21 smithi204 ceph-mon[43866]: pgmap v265: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:21.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:21 smithi031 ceph-mon[42085]: pgmap v265: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:21.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:21 smithi176 ceph-mon[43506]: pgmap v265: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:23.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:23 smithi204 ceph-mon[43866]: pgmap v266: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:23.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:23 smithi031 ceph-mon[42085]: pgmap v266: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:23.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:23 smithi176 ceph-mon[43506]: pgmap v266: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 204 B/s wr, 7 op/s 2023-12-14T18:36:25.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:25 smithi204 ceph-mon[43866]: pgmap v267: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:36:25.841 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:25 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:25.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:25 smithi031 ceph-mon[42085]: pgmap v267: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:36:25.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:25 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:25.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:25 smithi176 ceph-mon[43506]: pgmap v267: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:36:25.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:25 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:27.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:27 smithi204 ceph-mon[43866]: pgmap v268: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:36:27.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:27 smithi031 ceph-mon[42085]: pgmap v268: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:36:27.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:27 smithi176 ceph-mon[43506]: pgmap v268: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:36:29.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:29 smithi204 ceph-mon[43866]: pgmap v269: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:29.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:29 smithi031 ceph-mon[42085]: pgmap v269: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:29.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:29 smithi176 ceph-mon[43506]: pgmap v269: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:30.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:30 smithi204 ceph-mon[43866]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:30.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:30 smithi031 ceph-mon[42085]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:30.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:30 smithi176 ceph-mon[43506]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:31.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:31 smithi204 ceph-mon[43866]: pgmap v270: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:31.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:31 smithi031 ceph-mon[42085]: pgmap v270: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:31.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:31 smithi176 ceph-mon[43506]: pgmap v270: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:33.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:33 smithi204 ceph-mon[43866]: pgmap v271: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:33.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:33 smithi031 ceph-mon[42085]: pgmap v271: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:33.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:33 smithi176 ceph-mon[43506]: pgmap v271: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:35.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:35 smithi204 ceph-mon[43866]: pgmap v272: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:35.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:35 smithi031 ceph-mon[42085]: pgmap v272: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:35.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:35 smithi176 ceph-mon[43506]: pgmap v272: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:37.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:37 smithi204 ceph-mon[43866]: pgmap v273: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:37.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:37 smithi031 ceph-mon[42085]: pgmap v273: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:37.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:37 smithi176 ceph-mon[43506]: pgmap v273: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:39.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:39 smithi204 ceph-mon[43866]: pgmap v274: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:39.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:39 smithi031 ceph-mon[42085]: pgmap v274: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:39.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:39 smithi176 ceph-mon[43506]: pgmap v274: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:41.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:41 smithi204 ceph-mon[43866]: pgmap v275: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:41.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:41 smithi031 ceph-mon[42085]: pgmap v275: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:41.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:41 smithi176 ceph-mon[43506]: pgmap v275: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:43.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:36:43.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 -- 172.21.15.31:0/2225288344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb810ad10 msgr2=0x7effb810d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 --2- 172.21.15.31:0/2225288344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb810ad10 0x7effb810d0f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7effb000b0a0 tx=0x7effb000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 -- 172.21.15.31:0/2225288344 shutdown_connections 2023-12-14T18:36:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 --2- 172.21.15.31:0/2225288344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb810ad10 0x7effb810d0f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 --2- 172.21.15.31:0/2225288344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb80ff450 0x7effb80ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 --2- 172.21.15.31:0/2225288344 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb80ffd60 0x7effb810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 -- 172.21.15.31:0/2225288344 >> 172.21.15.31:0/2225288344 conn(0x7effb80749c0 msgr2=0x7effb8074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 -- 172.21.15.31:0/2225288344 shutdown_connections 2023-12-14T18:36:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.550+0000 7effbff97700 1 -- 172.21.15.31:0/2225288344 wait complete. 2023-12-14T18:36:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 Processor -- start 2023-12-14T18:36:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 -- start start 2023-12-14T18:36:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 0x7effb8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb80ffd60 0x7effb8196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb810ad10 0x7effb819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb819b430 con 0x7effb80ff450 2023-12-14T18:36:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb819b5a0 con 0x7effb810ad10 2023-12-14T18:36:43.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbff97700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb819b760 con 0x7effb80ffd60 2023-12-14T18:36:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbdd33700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 0x7effb8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbdd33700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 0x7effb8196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36814/0 (socket says 172.21.15.31:36814) 2023-12-14T18:36:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbdd33700 1 -- 172.21.15.31:0/1741377531 learned_addr learned my addr 172.21.15.31:0/1741377531 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbd532700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb80ffd60 0x7effb8196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbe534700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb810ad10 0x7effb819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.551+0000 7effbdd33700 1 -- 172.21.15.31:0/1741377531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb80ffd60 msgr2=0x7effb8196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbdd33700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb80ffd60 0x7effb8196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbdd33700 1 -- 172.21.15.31:0/1741377531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb810ad10 msgr2=0x7effb819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbdd33700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb810ad10 0x7effb819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbdd33700 1 -- 172.21.15.31:0/1741377531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0009d00 con 0x7effb80ff450 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbe534700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb810ad10 0x7effb819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbdd33700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 0x7effb8196450 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7effb400d950 tx=0x7effb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effaeffd700 1 -- 172.21.15.31:0/1741377531 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4009940 con 0x7effb80ff450 2023-12-14T18:36:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effaeffd700 1 -- 172.21.15.31:0/1741377531 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb4010460 con 0x7effb80ff450 2023-12-14T18:36:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effaeffd700 1 -- 172.21.15.31:0/1741377531 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb400f620 con 0x7effb80ff450 2023-12-14T18:36:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb81102f0 con 0x7effb80ff450 2023-12-14T18:36:43.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.552+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb8110160 con 0x7effb80ff450 2023-12-14T18:36:43.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.554+0000 7effaeffd700 1 -- 172.21.15.31:0/1741377531 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7effb40108d0 con 0x7effb80ff450 2023-12-14T18:36:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.555+0000 7effaeffd700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7effa40a12c0 0x7effa40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.555+0000 7effaeffd700 1 -- 172.21.15.31:0/1741377531 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7effb40ccf80 con 0x7effb80ff450 2023-12-14T18:36:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.555+0000 7effbd532700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7effa40a12c0 0x7effa40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.555+0000 7effbd532700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7effa40a12c0 0x7effa40a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7effb8197a60 tx=0x7effa800b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 Processor -- start 2023-12-14T18:36:43.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 -- start start 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f195a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1967f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f18445e0 con 0x5562f1965b50 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f18b3870 con 0x5562f195ac90 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.556+0000 7effcc125b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f18acd00 con 0x5562f195a380 2023-12-14T18:36:43.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1967f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1967f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36828/0 (socket says 172.21.15.31:36828) 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 -- 172.21.15.31:0/466783638 learned_addr learned my addr 172.21.15.31:0/466783638 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbdd33700 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f195a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbd532700 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 -- 172.21.15.31:0/466783638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 msgr2=0x5562f195a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f195a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 -- 172.21.15.31:0/466783638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 msgr2=0x5562f1965610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 -- 172.21.15.31:0/466783638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f172e3b0 con 0x5562f1965b50 2023-12-14T18:36:43.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effbe534700 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1967f30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7effb000b070 tx=0x7effb0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ab1c10dcd7641 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effa3fff700 1 -- 172.21.15.31:0/466783638 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000e050 con 0x5562f1965b50 2023-12-14T18:36:43.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effa3fff700 1 -- 172.21.15.31:0/466783638 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb0004a50 con 0x5562f1965b50 2023-12-14T18:36:43.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.557+0000 7effa3fff700 1 -- 172.21.15.31:0/466783638 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00052a0 con 0x5562f1965b50 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 -- 172.21.15.31:0/466783638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 msgr2=0x5562f1967f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1967f30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7effb000b070 tx=0x7effb0009c60 comp rx=0 tx=0).stop 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 -- 172.21.15.31:0/466783638 shutdown_connections 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1967f30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f195a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 --2- 172.21.15.31:0/466783638 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 -- 172.21.15.31:0/466783638 >> 172.21.15.31:0/466783638 conn(0x5562f1950ca0 msgr2=0x5562f19567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 -- 172.21.15.31:0/466783638 shutdown_connections 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.558+0000 7effcc125b80 1 -- 172.21.15.31:0/466783638 wait complete. 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 -- start start 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 Processor -- start 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 0x5562f1965500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195ac90 0x5562f19627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f1965b50 0x5562f1962d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f1848f90 con 0x5562f195a380 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f1848c30 con 0x5562f1965b50 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f17306a0 con 0x5562f195ac90 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effcc125b80 10 client.0 _ll_register_callbacks cb 0x5562f186c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effbdd33700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 0x5562f1965500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effbdd33700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 0x5562f1965500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36838/0 (socket says 172.21.15.31:36838) 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effbdd33700 1 -- 172.21.15.31:0/1362504210 learned_addr learned my addr 172.21.15.31:0/1362504210 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:43.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effbd532700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195ac90 0x5562f19627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.559+0000 7effbe534700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f1965b50 0x5562f1962d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effbdd33700 1 -- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195ac90 msgr2=0x5562f19627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effbdd33700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195ac90 0x5562f19627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effbdd33700 1 -- 172.21.15.31:0/1362504210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f1965b50 msgr2=0x5562f1962d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effbdd33700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f1965b50 0x5562f1962d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effbdd33700 1 -- 172.21.15.31:0/1362504210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb810d980 con 0x5562f195a380 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effbdd33700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 0x5562f1965500 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7effb4096160 tx=0x7effb409dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd77750165a3547 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effa17fa700 1 -- 172.21.15.31:0/1362504210 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb40964f0 con 0x5562f195a380 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effa17fa700 1 -- 172.21.15.31:0/1362504210 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7effb40b2200 con 0x5562f195a380 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562f194e2b0 con 0x5562f195a380 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effa17fa700 1 -- 172.21.15.31:0/1362504210 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7effb40b23c0 con 0x5562f195a380 2023-12-14T18:36:43.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effcc125b80 10 client.17463 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5562f1873170 con 0x5562f195a380 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.560+0000 7effa17fa700 1 -- 172.21.15.31:0/1362504210 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562f194e2b0 con 0x5562f195a380 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effa17fa700 1 -- 172.21.15.31:0/1362504210 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7effb4098bc0 con 0x5562f195a380 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effcc125b80 10 client.17463 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effcc125b80 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effcc125b80 20 client.17463 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effcc125b80 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f1975400 0x5562f1977810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effcc125b80 4 client.17463 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:43.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562f18a5ce0 con 0x5562f1975400 2023-12-14T18:36:43.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effbd532700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f1975400 0x5562f1977810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effa17fa700 10 client.17463 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:43.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.561+0000 7effbd532700 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f1975400 0x5562f1977810 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3961a3dc0d13afc0 server_cookie=26682392644ea8e5 in_seq=0 out_seq=0 2023-12-14T18:36:43.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effa17fa700 1 -- 172.21.15.31:0/1362504210 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5562f18a5ce0 con 0x5562f1975400 2023-12-14T18:36:43.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effa17fa700 10 client.17463 handle_command_reply: tid=0 2023-12-14T18:36:43.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 client.17463 shutdown 2023-12-14T18:36:43.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 msgr2=0x5562f1965500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 0x5562f1965500 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7effb4096160 tx=0x7effb409dd80 comp rx=0 tx=0).stop 2023-12-14T18:36:43.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 shutdown_connections 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f1975400 0x5562f1977810 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f1965b50 0x5562f1962d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195a380 0x5562f1965500 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effa17fa700 0 client.17463 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 --2- 172.21.15.31:0/1362504210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195ac90 0x5562f19627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 >> 172.21.15.31:0/1362504210 conn(0x5562f1950ca0 msgr2=0x5562f19563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 shutdown_connections 2023-12-14T18:36:43.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.562+0000 7effcc125b80 1 -- 172.21.15.31:0/1362504210 wait complete. 2023-12-14T18:36:43.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.563+0000 7effcc125b80 20 client.17463 trim_cache size 0 max 16384 2023-12-14T18:36:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 Processor -- start 2023-12-14T18:36:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 -- start start 2023-12-14T18:36:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f1964fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1962050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f195b360 con 0x5562f1965b50 2023-12-14T18:36:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f1960d10 con 0x5562f195ac90 2023-12-14T18:36:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.580+0000 7effcc125b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f195e600 con 0x5562f195a380 2023-12-14T18:36:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1962050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1962050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36852/0 (socket says 172.21.15.31:36852) 2023-12-14T18:36:43.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 -- 172.21.15.31:0/2218535283 learned_addr learned my addr 172.21.15.31:0/2218535283 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbd532700 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbdd33700 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f1964fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 -- 172.21.15.31:0/2218535283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 msgr2=0x5562f1964fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f1964fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 -- 172.21.15.31:0/2218535283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 msgr2=0x5562f1965500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 -- 172.21.15.31:0/2218535283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562f1872ec0 con 0x5562f1965b50 2023-12-14T18:36:43.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effbe534700 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1962050 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7effb00062a0 tx=0x7effb0008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b485cc46a8c821 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effacff9700 1 -- 172.21.15.31:0/2218535283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0007760 con 0x5562f1965b50 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effacff9700 1 -- 172.21.15.31:0/2218535283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb00078c0 con 0x5562f1965b50 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.581+0000 7effacff9700 1 -- 172.21.15.31:0/2218535283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb001bcc0 con 0x5562f1965b50 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 -- 172.21.15.31:0/2218535283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 msgr2=0x5562f1962050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1962050 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7effb00062a0 tx=0x7effb0008ec0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 -- 172.21.15.31:0/2218535283 shutdown_connections 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f1965b50 0x5562f1962050 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f195a380 0x5562f1964fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 --2- 172.21.15.31:0/2218535283 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195ac90 0x5562f1965500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 -- 172.21.15.31:0/2218535283 >> 172.21.15.31:0/2218535283 conn(0x5562f1950ca0 msgr2=0x5562f195dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 -- 172.21.15.31:0/2218535283 shutdown_connections 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 -- 172.21.15.31:0/2218535283 wait complete. 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 -- start start 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.582+0000 7effcc125b80 1 Processor -- start 2023-12-14T18:36:43.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195a380 0x5562f196c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 0x5562f196c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f1965b50 0x5562f196a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f196aba0 con 0x5562f195ac90 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f196ad10 con 0x5562f195a380 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562f196aed0 con 0x5562f1965b50 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effcc125b80 10 client.0 _ll_register_callbacks cb 0x7effb4099670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:43.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 0x5562f196c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 0x5562f196c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36854/0 (socket says 172.21.15.31:36854) 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 -- 172.21.15.31:0/4219580794 learned_addr learned my addr 172.21.15.31:0/4219580794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbe534700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f1965b50 0x5562f196a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbdd33700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195a380 0x5562f196c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 -- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f1965b50 msgr2=0x5562f196a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f1965b50 0x5562f196a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 -- 172.21.15.31:0/4219580794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195a380 msgr2=0x5562f196c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:43.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195a380 0x5562f196c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.583+0000 7effbd532700 1 -- 172.21.15.31:0/4219580794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562f172e3b0 con 0x5562f195ac90 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effbd532700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 0x5562f196c610 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x5562f1957b90 tx=0x7effa80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e2bbc92da63527 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effa17fa700 1 -- 172.21.15.31:0/4219580794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8025070 con 0x5562f195ac90 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effa17fa700 1 -- 172.21.15.31:0/4219580794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7effa8017070 con 0x5562f195ac90 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effb0004a50 con 0x5562f195ac90 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effa17fa700 1 -- 172.21.15.31:0/4219580794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa8011e10 con 0x5562f195ac90 2023-12-14T18:36:43.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 10 client.17469 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5562f18599a0 con 0x5562f195ac90 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effa17fa700 1 -- 172.21.15.31:0/4219580794 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb0004a50 con 0x5562f195ac90 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effa17fa700 1 -- 172.21.15.31:0/4219580794 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7effa801f5a0 con 0x5562f195ac90 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 10 client.17469 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 20 client.17469 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:43.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f19ce920 0x5562f19d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 4 client.17469 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.584+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5562f18a5a00 con 0x5562f19ce920 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effbdd33700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f19ce920 0x5562f19d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effbdd33700 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f19ce920 0x5562f19d0d30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b661dda4ccb848f7 server_cookie=f1472302257dee34 in_seq=0 out_seq=0 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effa17fa700 10 client.17469 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effa17fa700 1 -- 172.21.15.31:0/4219580794 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5562f18a5a00 con 0x5562f19ce920 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effa17fa700 10 client.17469 handle_command_reply: tid=0 2023-12-14T18:36:43.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 client.17469 shutdown 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 msgr2=0x5562f196c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 0x5562f196c610 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x5562f1957b90 tx=0x7effa80047c0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 shutdown_connections 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5562f19ce920 0x5562f19d0d30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5562f1965b50 0x5562f196a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5562f195a380 0x5562f196c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 --2- 172.21.15.31:0/4219580794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5562f195ac90 0x5562f196c610 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effa17fa700 0 client.17469 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:43.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.585+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 >> 172.21.15.31:0/4219580794 conn(0x5562f1950ca0 msgr2=0x5562f1962020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 shutdown_connections 2023-12-14T18:36:43.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effcc125b80 1 -- 172.21.15.31:0/4219580794 wait complete. 2023-12-14T18:36:43.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effcc125b80 20 client.17469 trim_cache size 0 max 16384 2023-12-14T18:36:43.604 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:36:43.604 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:36:43.605 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:36:43.605 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:36:43.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7effa40a12c0 msgr2=0x7effa40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7effa40a12c0 0x7effa40a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7effb8197a60 tx=0x7effa800b410 comp rx=0 tx=0).stop 2023-12-14T18:36:43.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 msgr2=0x7effb8196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 0x7effb8196450 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7effb400d950 tx=0x7effb400dc60 comp rx=0 tx=0).stop 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 shutdown_connections 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7effb810ad10 0x7effb819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effb80ff450 0x7effb8196450 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7effa40a12c0 0x7effa40a3770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 --2- 172.21.15.31:0/1741377531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effb80ffd60 0x7effb8196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 >> 172.21.15.31:0/1741377531 conn(0x7effb80749c0 msgr2=0x7effb80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 shutdown_connections 2023-12-14T18:36:43.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.586+0000 7effbff97700 1 -- 172.21.15.31:0/1741377531 wait complete. 2023-12-14T18:36:43.608 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:36:43.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:36:43.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:43 smithi031 ceph-mon[42085]: pgmap v276: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 -- 172.21.15.31:0/86301945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c0ff390 msgr2=0x7fab6c0ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 --2- 172.21.15.31:0/86301945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c0ff390 0x7fab6c0ff760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fab680099c0 tx=0x7fab68009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 -- 172.21.15.31:0/86301945 shutdown_connections 2023-12-14T18:36:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 --2- 172.21.15.31:0/86301945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c1043d0 0x7fab6c10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 --2- 172.21.15.31:0/86301945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c0ff390 0x7fab6c0ff760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 --2- 172.21.15.31:0/86301945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab6c0ffd30 0x7fab6c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 -- 172.21.15.31:0/86301945 >> 172.21.15.31:0/86301945 conn(0x7fab6c076370 msgr2=0x7fab6c076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 -- 172.21.15.31:0/86301945 shutdown_connections 2023-12-14T18:36:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.744+0000 7fab73605700 1 -- 172.21.15.31:0/86301945 wait complete. 2023-12-14T18:36:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 Processor -- start 2023-12-14T18:36:43.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 -- start start 2023-12-14T18:36:43.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab6c0ff390 0x7fab6c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 0x7fab6c196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c1043d0 0x7fab6c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c19b3e0 con 0x7fab6c0ffd30 2023-12-14T18:36:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c19b550 con 0x7fab6c0ff390 2023-12-14T18:36:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab73605700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c19b710 con 0x7fab6c1043d0 2023-12-14T18:36:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 0x7fab6c196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 0x7fab6c196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36872/0 (socket says 172.21.15.31:36872) 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 -- 172.21.15.31:0/2485358284 learned_addr learned my addr 172.21.15.31:0/2485358284 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab71ba2700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c1043d0 0x7fab6c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab713a1700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab6c0ff390 0x7fab6c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 -- 172.21.15.31:0/2485358284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c1043d0 msgr2=0x7fab6c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c1043d0 0x7fab6c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 -- 172.21.15.31:0/2485358284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab6c0ff390 msgr2=0x7fab6c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab6c0ff390 0x7fab6c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab70ba0700 1 -- 172.21.15.31:0/2485358284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64009c40 con 0x7fab6c0ffd30 2023-12-14T18:36:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.745+0000 7fab71ba2700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c1043d0 0x7fab6c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.746+0000 7fab70ba0700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 0x7fab6c196940 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00d950 tx=0x7fab5c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.746+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c004220 con 0x7fab6c0ffd30 2023-12-14T18:36:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.746+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab680095f0 con 0x7fab6c0ffd30 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.746+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5c004cf0 con 0x7fab6c0ffd30 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.746+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0056c0 con 0x7fab6c0ffd30 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.746+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c10f190 con 0x7fab6c0ffd30 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.747+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c10f000 con 0x7fab6c0ffd30 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.748+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fab5c004380 con 0x7fab6c0ffd30 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.748+0000 7fab627fc700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fab580a12c0 0x7fab580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:43.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.749+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fab5c0cd430 con 0x7fab6c0ffd30 2023-12-14T18:36:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.749+0000 7fab713a1700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fab580a12c0 0x7fab580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.749+0000 7fab713a1700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fab580a12c0 0x7fab580a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab68009fd0 tx=0x7fab68005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:43.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:43.752+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fab5c096450 con 0x7fab6c0ffd30 2023-12-14T18:36:43.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:43 smithi204 ceph-mon[43866]: pgmap v276: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:43.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:43 smithi176 ceph-mon[43506]: pgmap v276: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:44.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.008+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab6c10f320 con 0x7fab6c0ffd30 2023-12-14T18:36:44.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.009+0000 7fab627fc700 1 -- 172.21.15.31:0/2485358284 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10813 (secure 0 0 0) 0x7fab5c016090 con 0x7fab6c0ffd30 2023-12-14T18:36:44.012 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:36:44.012 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:29.509045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:36:44.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fab580a12c0 msgr2=0x7fab580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fab580a12c0 0x7fab580a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab68009fd0 tx=0x7fab68005fb0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 msgr2=0x7fab6c196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 0x7fab6c196940 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00d950 tx=0x7fab5c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:36:44.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 shutdown_connections 2023-12-14T18:36:44.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fab580a12c0 0x7fab580a3770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab6c1043d0 0x7fab6c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fab6c0ff390 0x7fab6c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 --2- 172.21.15.31:0/2485358284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab6c0ffd30 0x7fab6c196940 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.011+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 >> 172.21.15.31:0/2485358284 conn(0x7fab6c076370 msgr2=0x7fab6c067e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.012+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 shutdown_connections 2023-12-14T18:36:44.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.012+0000 7fab73605700 1 -- 172.21.15.31:0/2485358284 wait complete. 2023-12-14T18:36:44.016 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2023-12-14T18:36:44.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:36:44.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 -- 172.21.15.31:0/2889932269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 msgr2=0x7fb280104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 --2- 172.21.15.31:0/2889932269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb280104e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb270009920 tx=0x7fb270009c30 comp rx=0 tx=0).stop 2023-12-14T18:36:44.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 -- 172.21.15.31:0/2889932269 shutdown_connections 2023-12-14T18:36:44.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 --2- 172.21.15.31:0/2889932269 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280105550 0x7fb28010d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 --2- 172.21.15.31:0/2889932269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280100c50 0x7fb280101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 --2- 172.21.15.31:0/2889932269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb280104e20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 -- 172.21.15.31:0/2889932269 >> 172.21.15.31:0/2889932269 conn(0x7fb280077570 msgr2=0x7fb280077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 -- 172.21.15.31:0/2889932269 shutdown_connections 2023-12-14T18:36:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.165+0000 7fb2873ee700 1 -- 172.21.15.31:0/2889932269 wait complete. 2023-12-14T18:36:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 Processor -- start 2023-12-14T18:36:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 -- start start 2023-12-14T18:36:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280100c50 0x7fb2801964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb2801969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 0x7fb28019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28019b480 con 0x7fb280105550 2023-12-14T18:36:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28019b5f0 con 0x7fb280100c50 2023-12-14T18:36:44.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb2873ee700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28019b7b0 con 0x7fb2801015f0 2023-12-14T18:36:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb28598b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 0x7fb28019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb284989700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb2801969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb284989700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb2801969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47534/0 (socket says 172.21.15.31:47534) 2023-12-14T18:36:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb28598b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 0x7fb28019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36894/0 (socket says 172.21.15.31:36894) 2023-12-14T18:36:44.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb284989700 1 -- 172.21.15.31:0/3567172050 learned_addr learned my addr 172.21.15.31:0/3567172050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.166+0000 7fb28518a700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280100c50 0x7fb2801964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28598b700 1 -- 172.21.15.31:0/3567172050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 msgr2=0x7fb2801969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28598b700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb2801969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28598b700 1 -- 172.21.15.31:0/3567172050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280100c50 msgr2=0x7fb2801964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28598b700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280100c50 0x7fb2801964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28598b700 1 -- 172.21.15.31:0/3567172050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0095f0 con 0x7fb280105550 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb284989700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb2801969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28518a700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280100c50 0x7fb2801964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb28598b700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 0x7fb28019ae40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb27800efc0 tx=0x7fb27800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb2767fc700 1 -- 172.21.15.31:0/3567172050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278018070 con 0x7fb280105550 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb2767fc700 1 -- 172.21.15.31:0/3567172050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb27800f040 con 0x7fb280105550 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb2767fc700 1 -- 172.21.15.31:0/3567172050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27800cd40 con 0x7fb280105550 2023-12-14T18:36:44.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb270009520 con 0x7fb280105550 2023-12-14T18:36:44.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.167+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2801102d0 con 0x7fb280105550 2023-12-14T18:36:44.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.170+0000 7fb2767fc700 1 -- 172.21.15.31:0/3567172050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb278003a00 con 0x7fb280105550 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.170+0000 7fb2767fc700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb26c0a12c0 0x7fb26c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.170+0000 7fb28518a700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb26c0a12c0 0x7fb26c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.170+0000 7fb2767fc700 1 -- 172.21.15.31:0/3567172050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb2780ccc00 con 0x7fb280105550 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb28518a700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb26c0a12c0 0x7fb26c0a3770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb27c005fd0 tx=0x7fb27c009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 Processor -- start 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 -- start start 2023-12-14T18:36:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754f1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f753d05e0 con 0x564f754e6c90 2023-12-14T18:36:44.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f7543f870 con 0x564f754e6380 2023-12-14T18:36:44.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.171+0000 7fb29357cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f75438d00 con 0x564f754f1b50 2023-12-14T18:36:44.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754f1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754f1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36896/0 (socket says 172.21.15.31:36896) 2023-12-14T18:36:44.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 -- 172.21.15.31:0/4013403734 learned_addr learned my addr 172.21.15.31:0/4013403734 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb28598b700 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb28518a700 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 -- 172.21.15.31:0/4013403734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 msgr2=0x564f754f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 -- 172.21.15.31:0/4013403734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 msgr2=0x564f754e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754e6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 -- 172.21.15.31:0/4013403734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f752ba3b0 con 0x564f754e6c90 2023-12-14T18:36:44.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb28598b700 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb284989700 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754f1610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb2700094a0 tx=0x7fb270005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3995073a67c7c4c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb26b7fe700 1 -- 172.21.15.31:0/4013403734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27001c050 con 0x564f754e6c90 2023-12-14T18:36:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb26b7fe700 1 -- 172.21.15.31:0/4013403734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb27001d040 con 0x564f754e6c90 2023-12-14T18:36:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.172+0000 7fb26b7fe700 1 -- 172.21.15.31:0/4013403734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700053a0 con 0x564f754e6c90 2023-12-14T18:36:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 -- 172.21.15.31:0/4013403734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 msgr2=0x564f754f1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754f1610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb2700094a0 tx=0x7fb270005b40 comp rx=0 tx=0).stop 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 -- 172.21.15.31:0/4013403734 shutdown_connections 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 --2- 172.21.15.31:0/4013403734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754f1610 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 -- 172.21.15.31:0/4013403734 >> 172.21.15.31:0/4013403734 conn(0x564f754dcca0 msgr2=0x564f754e27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 -- 172.21.15.31:0/4013403734 shutdown_connections 2023-12-14T18:36:44.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 -- 172.21.15.31:0/4013403734 wait complete. 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.173+0000 7fb29357cb80 1 -- start start 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 Processor -- start 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754ee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f753d4f90 con 0x564f754e6c90 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f753d4c30 con 0x564f754e6380 2023-12-14T18:36:44.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f752bc6a0 con 0x564f754f1b50 2023-12-14T18:36:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb29357cb80 10 client.0 _ll_register_callbacks cb 0x564f753f8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb284989700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb284989700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754ee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36900/0 (socket says 172.21.15.31:36900) 2023-12-14T18:36:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb284989700 1 -- 172.21.15.31:0/989035544 learned_addr learned my addr 172.21.15.31:0/989035544 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb28598b700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.174+0000 7fb28518a700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb284989700 1 -- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 msgr2=0x564f754eee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb284989700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754eee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb284989700 1 -- 172.21.15.31:0/989035544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 msgr2=0x564f754f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb284989700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb284989700 1 -- 172.21.15.31:0/989035544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28010d910 con 0x564f754e6c90 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb28598b700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:44.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb284989700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754ee840 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb270009c70 tx=0x7fb27000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9700761ce42d61 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb268ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb268ff9700 1 -- 172.21.15.31:0/989035544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27001c040 con 0x564f754e6c90 2023-12-14T18:36:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb268ff9700 1 -- 172.21.15.31:0/989035544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb270005240 con 0x564f754e6c90 2023-12-14T18:36:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f754da2b0 con 0x564f754e6c90 2023-12-14T18:36:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb268ff9700 1 -- 172.21.15.31:0/989035544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb270005400 con 0x564f754e6c90 2023-12-14T18:36:44.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb29357cb80 10 client.17487 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.175+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564f753ff170 con 0x564f754e6c90 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb268ff9700 1 -- 172.21.15.31:0/989035544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f754da2b0 con 0x564f754e6c90 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb268ff9700 1 -- 172.21.15.31:0/989035544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb270015020 con 0x564f754e6c90 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb29357cb80 10 client.17487 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb29357cb80 10 client.17487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb29357cb80 20 client.17487 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:44.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb29357cb80 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75500f90 0x564f755033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb29357cb80 4 client.17487 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f75431ce0 con 0x564f75500f90 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.176+0000 7fb28518a700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75500f90 0x564f755033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb268ff9700 10 client.17487 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb28518a700 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75500f90 0x564f755033a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df8c0cba49029334 server_cookie=5ff5c38b001c1035 in_seq=0 out_seq=0 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb268ff9700 1 -- 172.21.15.31:0/989035544 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f75431ce0 con 0x564f75500f90 2023-12-14T18:36:44.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb268ff9700 10 client.17487 handle_command_reply: tid=0 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 client.17487 shutdown 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 msgr2=0x564f754ee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754ee840 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb270009c70 tx=0x7fb27000b790 comp rx=0 tx=0).stop 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 shutdown_connections 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75500f90 0x564f755033a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754eee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb268ff9700 0 client.17487 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6380 0x564f754f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 --2- 172.21.15.31:0/989035544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6c90 0x564f754ee840 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 >> 172.21.15.31:0/989035544 conn(0x564f754dcca0 msgr2=0x564f754e23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.177+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 shutdown_connections 2023-12-14T18:36:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.178+0000 7fb29357cb80 1 -- 172.21.15.31:0/989035544 wait complete. 2023-12-14T18:36:44.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.178+0000 7fb29357cb80 20 client.17487 trim_cache size 0 max 16384 2023-12-14T18:36:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.191+0000 7fb29357cb80 1 Processor -- start 2023-12-14T18:36:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.191+0000 7fb29357cb80 1 -- start start 2023-12-14T18:36:44.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f754ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f754e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f75507cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb29357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f754ecde0 con 0x564f754e6380 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb29357cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f754e1470 con 0x564f754e6c90 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb29357cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f754ea690 con 0x564f754f1b50 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f754ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f754ee800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36912/0 (socket says 172.21.15.31:36912) 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 -- 172.21.15.31:0/3036085458 learned_addr learned my addr 172.21.15.31:0/3036085458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 -- 172.21.15.31:0/3036085458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 msgr2=0x564f75507cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f75507cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 -- 172.21.15.31:0/3036085458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 msgr2=0x564f754e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f754e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 -- 172.21.15.31:0/3036085458 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f753feec0 con 0x564f754e6380 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb28518a700 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f754ee800 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x564f75503c40 tx=0x7fb27c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b4a407ca2af010 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.192+0000 7fb26bfff700 1 -- 172.21.15.31:0/3036085458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c026070 con 0x564f754e6380 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb26bfff700 1 -- 172.21.15.31:0/3036085458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb27c0041f0 con 0x564f754e6380 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb26bfff700 1 -- 172.21.15.31:0/3036085458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0114a0 con 0x564f754e6380 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 -- 172.21.15.31:0/3036085458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 msgr2=0x564f754ee800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f754ee800 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x564f75503c40 tx=0x7fb27c014000 comp rx=0 tx=0).stop 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 -- 172.21.15.31:0/3036085458 shutdown_connections 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f75507cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f754ee800 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 --2- 172.21.15.31:0/3036085458 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f754e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 -- 172.21.15.31:0/3036085458 >> 172.21.15.31:0/3036085458 conn(0x564f754dcca0 msgr2=0x564f754de210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 -- 172.21.15.31:0/3036085458 shutdown_connections 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 -- 172.21.15.31:0/3036085458 wait complete. 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.193+0000 7fb29357cb80 1 -- start start 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 Processor -- start 2023-12-14T18:36:44.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f7554e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f7554e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f754f4b40 con 0x564f754e6380 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f754f4cb0 con 0x564f754e6c90 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f754f4e70 con 0x564f754f1b50 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb29357cb80 10 client.0 _ll_register_callbacks cb 0x7fb270009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb28518a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f7554e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb28518a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f7554e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36928/0 (socket says 172.21.15.31:36928) 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb28518a700 1 -- 172.21.15.31:0/2019463674 learned_addr learned my addr 172.21.15.31:0/2019463674 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb28598b700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.194+0000 7fb284989700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f7554e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb28518a700 1 -- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 msgr2=0x564f754f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb28518a700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb28518a700 1 -- 172.21.15.31:0/2019463674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 msgr2=0x564f7554e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb28518a700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f7554e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb28518a700 1 -- 172.21.15.31:0/2019463674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c017070 con 0x564f754e6380 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb28518a700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f7554e030 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb27c003eb0 tx=0x7fb27c027a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d7e31f77a128f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb268ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb268ff9700 1 -- 172.21.15.31:0/2019463674 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c026050 con 0x564f754e6380 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb268ff9700 1 -- 172.21.15.31:0/2019463674 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb27c027250 con 0x564f754e6380 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb27001d040 con 0x564f754e6380 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb268ff9700 1 -- 172.21.15.31:0/2019463674 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb27c027410 con 0x564f754e6380 2023-12-14T18:36:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb29357cb80 10 client.17493 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564f753e59a0 con 0x564f754e6380 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.195+0000 7fb268ff9700 1 -- 172.21.15.31:0/2019463674 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb27001d040 con 0x564f754e6380 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb268ff9700 1 -- 172.21.15.31:0/2019463674 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb27c02e030 con 0x564f754e6380 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb29357cb80 10 client.17493 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb29357cb80 10 client.17493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb29357cb80 20 client.17493 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb29357cb80 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75555d50 0x564f75558160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb29357cb80 4 client.17493 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f75431a00 con 0x564f75555d50 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb284989700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75555d50 0x564f75558160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb268ff9700 10 client.17493 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.196+0000 7fb284989700 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75555d50 0x564f75558160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfd7e9e3111c2a4d server_cookie=6902841ea4578046 in_seq=0 out_seq=0 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb268ff9700 1 -- 172.21.15.31:0/2019463674 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f75431a00 con 0x564f75555d50 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb268ff9700 10 client.17493 handle_command_reply: tid=0 2023-12-14T18:36:44.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 client.17493 shutdown 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 msgr2=0x564f7554e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f7554e030 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb27c003eb0 tx=0x7fb27c027a10 comp rx=0 tx=0).stop 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 shutdown_connections 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x564f75555d50 0x564f75558160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f754f1b50 0x564f754f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x564f754e6380 0x564f7554e030 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb268ff9700 0 client.17493 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:36:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 --2- 172.21.15.31:0/2019463674 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x564f754e6c90 0x564f7554e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 >> 172.21.15.31:0/2019463674 conn(0x564f754dcca0 msgr2=0x564f754fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 shutdown_connections 2023-12-14T18:36:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 1 -- 172.21.15.31:0/2019463674 wait complete. 2023-12-14T18:36:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.197+0000 7fb29357cb80 20 client.17493 trim_cache size 0 max 16384 2023-12-14T18:36:44.216 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:36:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb26c0a12c0 msgr2=0x7fb26c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb26c0a12c0 0x7fb26c0a3770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb27c005fd0 tx=0x7fb27c009310 comp rx=0 tx=0).stop 2023-12-14T18:36:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 msgr2=0x7fb28019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 0x7fb28019ae40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb27800efc0 tx=0x7fb27800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 shutdown_connections 2023-12-14T18:36:44.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb280105550 0x7fb28019ae40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb280100c50 0x7fb2801964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb26c0a12c0 0x7fb26c0a3770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 --2- 172.21.15.31:0/3567172050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2801015f0 0x7fb2801969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 >> 172.21.15.31:0/3567172050 conn(0x7fb280077570 msgr2=0x7fb2800feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 shutdown_connections 2023-12-14T18:36:44.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.198+0000 7fb2873ee700 1 -- 172.21.15.31:0/3567172050 wait complete. 2023-12-14T18:36:44.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:36:44.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1559357063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 msgr2=0x7f9ba4104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1559357063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4104ea0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9b980099c0 tx=0x7f9b98009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1559357063 shutdown_connections 2023-12-14T18:36:44.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1559357063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ba4105580 0x7f9ba410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1559357063 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ba40692f0 0x7f9ba40696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1559357063 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4104ea0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1559357063 >> 172.21.15.31:0/1559357063 conn(0x7f9ba4075fa0 msgr2=0x7f9ba40763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1559357063 shutdown_connections 2023-12-14T18:36:44.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.357+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1559357063 wait complete. 2023-12-14T18:36:44.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 Processor -- start 2023-12-14T18:36:44.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 -- start start 2023-12-14T18:36:44.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ba40692f0 0x7f9ba4196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ba4105580 0x7f9ba419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419b2e0 con 0x7f9ba4069c90 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419b450 con 0x7f9ba4105580 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba99c0700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba419b610 con 0x7f9ba40692f0 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.358+0000 7f9ba27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36946/0 (socket says 172.21.15.31:36946) 2023-12-14T18:36:44.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 -- 172.21.15.31:0/1012718498 learned_addr learned my addr 172.21.15.31:0/1012718498 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ba40692f0 0x7f9ba4196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ba4105580 0x7f9ba419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 -- 172.21.15.31:0/1012718498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ba40692f0 msgr2=0x7f9ba4196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ba40692f0 0x7f9ba4196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 -- 172.21.15.31:0/1012718498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ba4105580 msgr2=0x7f9ba419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ba4105580 0x7f9ba419aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 -- 172.21.15.31:0/1012718498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980095f0 con 0x7f9ba4069c90 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4196840 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009990 tx=0x7f9b98005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba89be700 1 -- 172.21.15.31:0/1012718498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9801d070 con 0x7f9ba4069c90 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba4110290 con 0x7f9ba4069c90 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba89be700 1 -- 172.21.15.31:0/1012718498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b98004090 con 0x7f9ba4069c90 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba89be700 1 -- 172.21.15.31:0/1012718498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98022d00 con 0x7f9ba4069c90 2023-12-14T18:36:44.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.359+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba4110100 con 0x7f9ba4069c90 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.361+0000 7f9ba89be700 1 -- 172.21.15.31:0/1012718498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9b98022470 con 0x7f9ba4069c90 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.362+0000 7f9ba89be700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9b8c0a12c0 0x7f9b8c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.362+0000 7f9ba89be700 1 -- 172.21.15.31:0/1012718498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9b980ce3f0 con 0x7f9ba4069c90 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.362+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9b8c0a12c0 0x7f9b8c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9b8c0a12c0 0x7f9b8c0a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b94005950 tx=0x7f9b940058e0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 Processor -- start 2023-12-14T18:36:44.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 -- start start 2023-12-14T18:36:44.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf07750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 0x55de1bf16a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf16f60 0x55de1bf19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bdf15e0 con 0x55de1bf07c90 2023-12-14T18:36:44.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1be60870 con 0x55de1bf16f60 2023-12-14T18:36:44.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1be59d00 con 0x55de1bf07380 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9ba27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 0x55de1bf16a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9ba27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 0x55de1bf16a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36950/0 (socket says 172.21.15.31:36950) 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9ba27fc700 1 -- 172.21.15.31:0/4222459085 learned_addr learned my addr 172.21.15.31:0/4222459085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf07750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.363+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf16f60 0x55de1bf19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9ba27fc700 1 -- 172.21.15.31:0/4222459085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 msgr2=0x55de1bf07750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf07750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9ba27fc700 1 -- 172.21.15.31:0/4222459085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf16f60 msgr2=0x55de1bf19340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf16f60 0x55de1bf19340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9ba27fc700 1 -- 172.21.15.31:0/4222459085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1bcdb3b0 con 0x55de1bf07c90 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 0x55de1bf16a20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9b98018fd0 tx=0x7f9b98009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4f83456b2c4512 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9b89ffb700 1 -- 172.21.15.31:0/4222459085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b980cbea0 con 0x55de1bf07c90 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9b89ffb700 1 -- 172.21.15.31:0/4222459085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9809f190 con 0x55de1bf07c90 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.364+0000 7f9b89ffb700 1 -- 172.21.15.31:0/4222459085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9809a890 con 0x55de1bf07c90 2023-12-14T18:36:44.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/4222459085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 msgr2=0x55de1bf16a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 0x55de1bf16a20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9b98018fd0 tx=0x7f9b98009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/4222459085 shutdown_connections 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf16f60 0x55de1bf19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf07750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/4222459085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07c90 0x55de1bf16a20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/4222459085 >> 172.21.15.31:0/4222459085 conn(0x55de1befdca0 msgr2=0x55de1bf037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/4222459085 shutdown_connections 2023-12-14T18:36:44.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/4222459085 wait complete. 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 -- start start 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.365+0000 7f9bb5b4eb80 1 Processor -- start 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf17330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf17870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 0x55de1bf17dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bdf5f90 con 0x55de1bf16f60 2023-12-14T18:36:44.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bdf5c30 con 0x55de1bf07c90 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bcdd6a0 con 0x55de1bf07380 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9bb5b4eb80 10 client.0 _ll_register_callbacks cb 0x55de1be19910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 0x55de1bf17dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 0x55de1bf17dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36960/0 (socket says 172.21.15.31:36960) 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 -- 172.21.15.31:0/236124907 learned_addr learned my addr 172.21.15.31:0/236124907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf17870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf17330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 -- 172.21.15.31:0/236124907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 msgr2=0x55de1bf17330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf17330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 -- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 msgr2=0x55de1bf17870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf17870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 -- 172.21.15.31:0/236124907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba4110100 con 0x55de1bf16f60 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf17330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 0x55de1bf17dd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0060b0 tx=0x7f9b9c007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5b9a3ad2678744 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.366+0000 7f9b82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9b82ffd700 1 -- 172.21.15.31:0/236124907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c007e60 con 0x55de1bf16f60 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9b82ffd700 1 -- 172.21.15.31:0/236124907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9b9c010860 con 0x55de1bf16f60 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de1befb2b0 con 0x55de1bf16f60 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9b82ffd700 1 -- 172.21.15.31:0/236124907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b9c009820 con 0x55de1bf16f60 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 10 client.17505 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55de1be20170 con 0x55de1bf16f60 2023-12-14T18:36:44.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9b82ffd700 1 -- 172.21.15.31:0/236124907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de1befb2b0 con 0x55de1bf16f60 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9b82ffd700 1 -- 172.21.15.31:0/236124907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b9c016cd0 con 0x55de1bf16f60 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 10 client.17505 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 10 client.17505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 20 client.17505 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf26730 0x55de1bf28b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 4 client.17505 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.367+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de1be52ce0 con 0x55de1bf26730 2023-12-14T18:36:44.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.368+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf26730 0x55de1bf28b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.368+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf26730 0x55de1bf28b40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d84a22c6f53d50df server_cookie=981f9518226f0a2b in_seq=0 out_seq=0 2023-12-14T18:36:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.368+0000 7f9b82ffd700 10 client.17505 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9b82ffd700 1 -- 172.21.15.31:0/236124907 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de1be52ce0 con 0x55de1bf26730 2023-12-14T18:36:44.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9b82ffd700 10 client.17505 handle_command_reply: tid=0 2023-12-14T18:36:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9bb5b4eb80 1 client.17505 shutdown 2023-12-14T18:36:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 msgr2=0x55de1bf17dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 0x55de1bf17dd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0060b0 tx=0x7f9b9c007c20 comp rx=0 tx=0).stop 2023-12-14T18:36:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 shutdown_connections 2023-12-14T18:36:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf16f60 0x55de1bf17dd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.369+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf26730 0x55de1bf28b40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf07380 0x55de1bf17330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9b82ffd700 0 client.17505 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/236124907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf17870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 >> 172.21.15.31:0/236124907 conn(0x55de1befdca0 msgr2=0x55de1bf03370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 shutdown_connections 2023-12-14T18:36:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/236124907 wait complete. 2023-12-14T18:36:44.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.370+0000 7f9bb5b4eb80 20 client.17505 trim_cache size 0 max 16384 2023-12-14T18:36:44.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 Processor -- start 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 -- start start 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf15810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf21b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bf22450 con 0x55de1bf07380 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bf23d70 con 0x55de1bf07c90 2023-12-14T18:36:44.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bf04550 con 0x55de1bf16f60 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36974/0 (socket says 172.21.15.31:36974) 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/408004614 learned_addr learned my addr 172.21.15.31:0/408004614 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.384+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf21b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf15810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/408004614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 msgr2=0x55de1bf21b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf21b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/408004614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 msgr2=0x55de1bf15810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf15810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/408004614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1be1fec0 con 0x55de1bf07380 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c360 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55de1bf29450 tx=0x7f9b94015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb0b4218d43f37c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9b8a7fc700 1 -- 172.21.15.31:0/408004614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94025040 con 0x55de1bf07380 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9b8a7fc700 1 -- 172.21.15.31:0/408004614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b940204a0 con 0x55de1bf07380 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.385+0000 7f9b8a7fc700 1 -- 172.21.15.31:0/408004614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940116a0 con 0x55de1bf07380 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/408004614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 msgr2=0x55de1bf1c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c360 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x55de1bf29450 tx=0x7f9b94015000 comp rx=0 tx=0).stop 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/408004614 shutdown_connections 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf21b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c360 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/408004614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf15810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/408004614 >> 172.21.15.31:0/408004614 conn(0x55de1befdca0 msgr2=0x55de1bf17330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/408004614 shutdown_connections 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/408004614 wait complete. 2023-12-14T18:36:44.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 -- start start 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.386+0000 7f9bb5b4eb80 1 Processor -- start 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf1c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bf18c00 con 0x55de1bf07380 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bf18d70 con 0x55de1bf07c90 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1bf18f30 con 0x55de1bf16f60 2023-12-14T18:36:44.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9bb5b4eb80 10 client.0 _ll_register_callbacks cb 0x7f9b9c009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36980/0 (socket says 172.21.15.31:36980) 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/2282613727 learned_addr learned my addr 172.21.15.31:0/2282613727 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf1c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba37fe700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/2282613727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 msgr2=0x55de1bf184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 msgr2=0x55de1bf1c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf1c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 -- 172.21.15.31:0/2282613727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9400f980 con 0x55de1bf07380 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf1c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9ba2ffd700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c020 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9b94014040 tx=0x7f9b94027650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ed856f4df1fd6b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9b82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.387+0000 7f9b82ffd700 1 -- 172.21.15.31:0/2282613727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94025070 con 0x55de1bf07380 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9b82ffd700 1 -- 172.21.15.31:0/2282613727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9b940209e0 con 0x55de1bf07380 2023-12-14T18:36:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b9809f190 con 0x55de1bf07380 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9b82ffd700 1 -- 172.21.15.31:0/2282613727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b94020ba0 con 0x55de1bf07380 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 10 client.17511 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55de1be069a0 con 0x55de1bf07380 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9b82ffd700 1 -- 172.21.15.31:0/2282613727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9809f190 con 0x55de1bf07380 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9b82ffd700 1 -- 172.21.15.31:0/2282613727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b94004800 con 0x55de1bf07380 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 10 client.17511 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 10 client.17511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:36:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 20 client.17511 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf7b860 0x55de1bf7dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 4 client.17511 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de1be52a00 con 0x55de1bf7b860 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.388+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf7b860 0x55de1bf7dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9ba27fc700 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf7b860 0x55de1bf7dc70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=befc6182dadd09f6 server_cookie=74ea2f2221d1c5ff in_seq=0 out_seq=0 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9b82ffd700 10 client.17511 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:44.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9b82ffd700 1 -- 172.21.15.31:0/2282613727 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de1be52a00 con 0x55de1bf7b860 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9b82ffd700 10 client.17511 handle_command_reply: tid=0 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 client.17511 shutdown 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 msgr2=0x55de1bf1c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c020 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9b94014040 tx=0x7f9b94027650 comp rx=0 tx=0).stop 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 shutdown_connections 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55de1bf7b860 0x55de1bf7dc70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55de1bf16f60 0x55de1bf184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55de1bf07380 0x55de1bf1c020 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9b82ffd700 0 client.17511 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:36:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 --2- 172.21.15.31:0/2282613727 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55de1bf07c90 0x55de1bf1c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 >> 172.21.15.31:0/2282613727 conn(0x55de1befdca0 msgr2=0x55de1bf17bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 shutdown_connections 2023-12-14T18:36:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.389+0000 7f9bb5b4eb80 1 -- 172.21.15.31:0/2282613727 wait complete. 2023-12-14T18:36:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9bb5b4eb80 20 client.17511 trim_cache size 0 max 16384 2023-12-14T18:36:44.407 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:36:44.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9b8c0a12c0 msgr2=0x7f9b8c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9b8c0a12c0 0x7f9b8c0a3770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9b94005950 tx=0x7f9b940058e0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 msgr2=0x7f9ba4196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4196840 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009990 tx=0x7f9b98005e80 comp rx=0 tx=0).stop 2023-12-14T18:36:44.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 shutdown_connections 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9ba4105580 0x7f9ba419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9b8c0a12c0 0x7f9b8c0a3770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ba40692f0 0x7f9ba4196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 --2- 172.21.15.31:0/1012718498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ba4069c90 0x7f9ba4196840 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 >> 172.21.15.31:0/1012718498 conn(0x7f9ba4075fa0 msgr2=0x7f9ba4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 shutdown_connections 2023-12-14T18:36:44.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.390+0000 7f9ba99c0700 1 -- 172.21.15.31:0/1012718498 wait complete. 2023-12-14T18:36:44.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:36:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 -- 172.21.15.31:0/1495313224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 msgr2=0x7fca0c0ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 --2- 172.21.15.31:0/1495313224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c0ff470 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fca08009990 tx=0x7fca08009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 -- 172.21.15.31:0/1495313224 shutdown_connections 2023-12-14T18:36:44.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 --2- 172.21.15.31:0/1495313224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 0x7fca0c10cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 --2- 172.21.15.31:0/1495313224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c0ff470 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 --2- 172.21.15.31:0/1495313224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca0c0ff9b0 0x7fca0c0ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 -- 172.21.15.31:0/1495313224 >> 172.21.15.31:0/1495313224 conn(0x7fca0c074a60 msgr2=0x7fca0c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 -- 172.21.15.31:0/1495313224 shutdown_connections 2023-12-14T18:36:44.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.550+0000 7fca142a2700 1 -- 172.21.15.31:0/1495313224 wait complete. 2023-12-14T18:36:44.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 Processor -- start 2023-12-14T18:36:44.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 -- start start 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca0c0ff9b0 0x7fca0c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 0x7fca0c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c19b2e0 con 0x7fca0c0760f0 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c19b450 con 0x7fca0c10a880 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.551+0000 7fca142a2700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c19b610 con 0x7fca0c0ff9b0 2023-12-14T18:36:44.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36756/0 (socket says 172.21.15.31:36756) 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 -- 172.21.15.31:0/1915132896 learned_addr learned my addr 172.21.15.31:0/1915132896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1183d700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca0c0ff9b0 0x7fca0c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1283f700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 0x7fca0c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 -- 172.21.15.31:0/1915132896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca0c0ff9b0 msgr2=0x7fca0c196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca0c0ff9b0 0x7fca0c196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 -- 172.21.15.31:0/1915132896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 msgr2=0x7fca0c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 0x7fca0c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 -- 172.21.15.31:0/1915132896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080095f0 con 0x7fca0c0760f0 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1283f700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 0x7fca0c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca1203e700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c196300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fca080095c0 tx=0x7fca0800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca037fe700 1 -- 172.21.15.31:0/1915132896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0801d070 con 0x7fca0c0760f0 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c110270 con 0x7fca0c0760f0 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca037fe700 1 -- 172.21.15.31:0/1915132896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0800fb50 con 0x7fca0c0760f0 2023-12-14T18:36:44.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca037fe700 1 -- 172.21.15.31:0/1915132896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080173f0 con 0x7fca0c0760f0 2023-12-14T18:36:44.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.552+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1100e0 con 0x7fca0c0760f0 2023-12-14T18:36:44.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.555+0000 7fca037fe700 1 -- 172.21.15.31:0/1915132896 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fca0800fcc0 con 0x7fca0c0760f0 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.555+0000 7fca037fe700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc9f80a12c0 0x7fc9f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.555+0000 7fca037fe700 1 -- 172.21.15.31:0/1915132896 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fca080ce220 con 0x7fca0c0760f0 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.555+0000 7fca1183d700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc9f80a12c0 0x7fc9f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.556+0000 7fca1183d700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc9f80a12c0 0x7fc9f80a3770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca0c197910 tx=0x7fc9fc00b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.556+0000 7fca20430b80 1 Processor -- start 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 -- start start 2023-12-14T18:36:44.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d310750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d31b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 0x55a62d31df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d1fa5e0 con 0x55a62d31bb50 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d269870 con 0x55a62d310380 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca20430b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d262d00 con 0x55a62d310c90 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 0x55a62d31df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 0x55a62d31df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36760/0 (socket says 172.21.15.31:36760) 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 -- 172.21.15.31:0/311824724 learned_addr learned my addr 172.21.15.31:0/311824724 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1203e700 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d310750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1183d700 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d31b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 -- 172.21.15.31:0/311824724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 msgr2=0x55a62d31b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d31b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 -- 172.21.15.31:0/311824724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 msgr2=0x55a62d310750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d310750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.557+0000 7fca1283f700 1 -- 172.21.15.31:0/311824724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62d0e43b0 con 0x55a62d31bb50 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.558+0000 7fca1283f700 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 0x55a62d31df30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fca04009790 tx=0x7fca04007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3531e695ee9d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.558+0000 7fca00dac700 1 -- 172.21.15.31:0/311824724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04004bb0 con 0x55a62d31bb50 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.558+0000 7fca00dac700 1 -- 172.21.15.31:0/311824724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca04004d10 con 0x55a62d31bb50 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.558+0000 7fca00dac700 1 -- 172.21.15.31:0/311824724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca04014660 con 0x55a62d31bb50 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.558+0000 7fca20430b80 1 -- 172.21.15.31:0/311824724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 msgr2=0x55a62d31df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.558+0000 7fca20430b80 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 0x55a62d31df30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fca04009790 tx=0x7fca04007500 comp rx=0 tx=0).stop 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- 172.21.15.31:0/311824724 shutdown_connections 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d31bb50 0x55a62d31df30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d310750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 --2- 172.21.15.31:0/311824724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d31b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- 172.21.15.31:0/311824724 >> 172.21.15.31:0/311824724 conn(0x55a62d306ca0 msgr2=0x55a62d30c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- 172.21.15.31:0/311824724 shutdown_connections 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- 172.21.15.31:0/311824724 wait complete. 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- start start 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 Processor -- start 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 0x55a62d3187b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d31bb50 0x55a62d318d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d1fef90 con 0x55a62d310c90 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d1fec30 con 0x55a62d31bb50 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.559+0000 7fca20430b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d0e66a0 con 0x55a62d310380 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca20430b80 10 client.0 _ll_register_callbacks cb 0x55a62d222910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 0x55a62d3187b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 0x55a62d3187b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36774/0 (socket says 172.21.15.31:36774) 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 -- 172.21.15.31:0/1101155275 learned_addr learned my addr 172.21.15.31:0/1101155275 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1203e700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1283f700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d31bb50 0x55a62d318d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 -- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 msgr2=0x55a62d31b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 -- 172.21.15.31:0/1101155275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d31bb50 msgr2=0x55a62d318d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d31bb50 0x55a62d318d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 -- 172.21.15.31:0/1101155275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca04009980 con 0x55a62d310c90 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca1183d700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 0x55a62d3187b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0116f0 tx=0x7fc9fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f2bb116be66ffd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fc9f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/1101155275 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc004a30 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/1101155275 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fc9fc017070 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.560+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a62d3042b0 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/1101155275 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9fc010eb0 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 10 client.17523 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a62d229170 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/1101155275 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a62d3042b0 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/1101155275 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9fc01f6d0 con 0x55a62d310c90 2023-12-14T18:36:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 10 client.17523 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 20 client.17523 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d32b590 0x55a62d32d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 4 client.17523 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a62d25bce0 con 0x55a62d32b590 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.561+0000 7fca1203e700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d32b590 0x55a62d32d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.562+0000 7fc9f5ffb700 10 client.17523 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:44.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.562+0000 7fca1203e700 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d32b590 0x55a62d32d9a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d32e989ce393dd7e server_cookie=a181cf221c86f8af in_seq=0 out_seq=0 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.562+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/1101155275 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a62d25bce0 con 0x55a62d32b590 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.562+0000 7fc9f5ffb700 10 client.17523 handle_command_reply: tid=0 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.562+0000 7fca20430b80 1 client.17523 shutdown 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 msgr2=0x55a62d3187b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 0x55a62d3187b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0116f0 tx=0x7fc9fc0047c0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 shutdown_connections 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d32b590 0x55a62d32d9a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d31bb50 0x55a62d318d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fc9f5ffb700 0 client.17523 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 --2- 172.21.15.31:0/1101155275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d310c90 0x55a62d3187b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 >> 172.21.15.31:0/1101155275 conn(0x55a62d306ca0 msgr2=0x55a62d30c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 shutdown_connections 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 1 -- 172.21.15.31:0/1101155275 wait complete. 2023-12-14T18:36:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.563+0000 7fca20430b80 20 client.17523 trim_cache size 0 max 16384 2023-12-14T18:36:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 Processor -- start 2023-12-14T18:36:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 -- start start 2023-12-14T18:36:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d31afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d313ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d327540 con 0x55a62d306ca0 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d316d10 con 0x55a62d310380 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca20430b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d314600 con 0x55a62d310c90 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca1203e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d31afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca1203e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d31afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36776/0 (socket says 172.21.15.31:36776) 2023-12-14T18:36:44.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.577+0000 7fca1203e700 1 -- 172.21.15.31:0/1138738064 learned_addr learned my addr 172.21.15.31:0/1138738064 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1283f700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d313ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1183d700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1203e700 1 -- 172.21.15.31:0/1138738064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 msgr2=0x55a62d313ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1203e700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d313ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1203e700 1 -- 172.21.15.31:0/1138738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 msgr2=0x55a62d31b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1203e700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1203e700 1 -- 172.21.15.31:0/1138738064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a62d228ec0 con 0x55a62d306ca0 2023-12-14T18:36:44.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1283f700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d313ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1183d700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca1203e700 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d31afc0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fca0809af20 tx=0x7fca080cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe0532021632240 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca015ad700 1 -- 172.21.15.31:0/1138738064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0809efc0 con 0x55a62d306ca0 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca015ad700 1 -- 172.21.15.31:0/1138738064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca080a3350 con 0x55a62d306ca0 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.578+0000 7fca015ad700 1 -- 172.21.15.31:0/1138738064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0809a6c0 con 0x55a62d306ca0 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- 172.21.15.31:0/1138738064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 msgr2=0x55a62d31afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d31afc0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fca0809af20 tx=0x7fca080cd040 comp rx=0 tx=0).stop 2023-12-14T18:36:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- 172.21.15.31:0/1138738064 shutdown_connections 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310c90 0x55a62d313ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d31afc0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- 172.21.15.31:0/1138738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310380 0x55a62d31b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- 172.21.15.31:0/1138738064 >> 172.21.15.31:0/1138738064 conn(0x55a62d31e200 msgr2=0x55a62d326e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- 172.21.15.31:0/1138738064 shutdown_connections 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- 172.21.15.31:0/1138738064 wait complete. 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- start start 2023-12-14T18:36:44.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 Processor -- start 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d320540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d320a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310c90 0x55a62d378420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.579+0000 7fca20430b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d378960 con 0x55a62d306ca0 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca20430b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d378ad0 con 0x55a62d310c90 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca20430b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a62d378c90 con 0x55a62d310380 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca20430b80 10 client.0 _ll_register_callbacks cb 0x7fc9fc010d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d320540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d320540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36784/0 (socket says 172.21.15.31:36784) 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 -- 172.21.15.31:0/2552102105 learned_addr learned my addr 172.21.15.31:0/2552102105 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1183d700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d320a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 -- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 msgr2=0x55a62d320a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d320a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 -- 172.21.15.31:0/2552102105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310c90 msgr2=0x55a62d378420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1283f700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310c90 0x55a62d378420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310c90 0x55a62d378420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 -- 172.21.15.31:0/2552102105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1100e0 con 0x55a62d306ca0 2023-12-14T18:36:44.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca1203e700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d320540 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fca080cbb00 tx=0x7fca08009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244e1aa5f6374e86 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fc9f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/2552102105 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca080b90d0 con 0x55a62d306ca0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/2552102105 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fca080b9230 con 0x55a62d306ca0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca04004d10 con 0x55a62d306ca0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/2552102105 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca0809b1f0 con 0x55a62d306ca0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca20430b80 10 client.17529 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.580+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a62d20f9a0 con 0x55a62d306ca0 2023-12-14T18:36:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/2552102105 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca04004d10 con 0x55a62d306ca0 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/2552102105 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca08099ef0 con 0x55a62d306ca0 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca20430b80 10 client.17529 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca20430b80 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca20430b80 20 client.17529 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca20430b80 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d31bb50 0x55a62d381e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca20430b80 4 client.17529 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a62d25ba00 con 0x55a62d31bb50 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.581+0000 7fca1183d700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d31bb50 0x55a62d381e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:44.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fc9f5ffb700 10 client.17529 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca1183d700 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d31bb50 0x55a62d381e90 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0736ade1112543 server_cookie=6c693c3eb0411d03 in_seq=0 out_seq=0 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fc9f5ffb700 1 -- 172.21.15.31:0/2552102105 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a62d25ba00 con 0x55a62d31bb50 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fc9f5ffb700 10 client.17529 handle_command_reply: tid=0 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 client.17529 shutdown 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 msgr2=0x55a62d320540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d320540 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fca080cbb00 tx=0x7fca08009310 comp rx=0 tx=0).stop 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 shutdown_connections 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55a62d31bb50 0x55a62d381e90 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55a62d310c90 0x55a62d378420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55a62d306ca0 0x55a62d320540 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fc9f5ffb700 0 client.17529 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 --2- 172.21.15.31:0/2552102105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a62d310380 0x55a62d320a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 >> 172.21.15.31:0/2552102105 conn(0x55a62d31e200 msgr2=0x55a62d313ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 shutdown_connections 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 1 -- 172.21.15.31:0/2552102105 wait complete. 2023-12-14T18:36:44.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.582+0000 7fca20430b80 20 client.17529 trim_cache size 0 max 16384 2023-12-14T18:36:44.597 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:36:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc9f80a12c0 msgr2=0x7fc9f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc9f80a12c0 0x7fc9f80a3770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fca0c197910 tx=0x7fc9fc00b410 comp rx=0 tx=0).stop 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 msgr2=0x7fca0c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c196300 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fca080095c0 tx=0x7fca0800f710 comp rx=0 tx=0).stop 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 shutdown_connections 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca0c10a880 0x7fca0c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca0c0760f0 0x7fca0c196300 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fc9f80a12c0 0x7fc9f80a3770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 --2- 172.21.15.31:0/1915132896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca0c0ff9b0 0x7fca0c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 >> 172.21.15.31:0/1915132896 conn(0x7fca0c074a60 msgr2=0x7fca0c102790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 shutdown_connections 2023-12-14T18:36:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:44.583+0000 7fca142a2700 1 -- 172.21.15.31:0/1915132896 wait complete. 2023-12-14T18:36:44.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:44 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2485358284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:36:44.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:44 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2485358284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:36:44.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:44 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2485358284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:36:45.601 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:36:45.602 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:36:45.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 -- 172.21.15.31:0/3257622103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880105550 msgr2=0x7f488010d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 --2- 172.21.15.31:0/3257622103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880105550 0x7f488010d0d0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f487800b0d0 tx=0x7f487800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 -- 172.21.15.31:0/3257622103 shutdown_connections 2023-12-14T18:36:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 --2- 172.21.15.31:0/3257622103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880105550 0x7f488010d0d0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 --2- 172.21.15.31:0/3257622103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880100c50 0x7f4880101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 --2- 172.21.15.31:0/3257622103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 0x7f4880104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 -- 172.21.15.31:0/3257622103 >> 172.21.15.31:0/3257622103 conn(0x7f4880077570 msgr2=0x7f4880077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 -- 172.21.15.31:0/3257622103 shutdown_connections 2023-12-14T18:36:45.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.750+0000 7f4886f2d700 1 -- 172.21.15.31:0/3257622103 wait complete. 2023-12-14T18:36:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 Processor -- start 2023-12-14T18:36:45.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 -- start start 2023-12-14T18:36:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 0x7f4880196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 0x7f4880196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880105550 0x7f488019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488019b4f0 con 0x7f4880100c50 2023-12-14T18:36:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488019b660 con 0x7f48801015f0 2023-12-14T18:36:45.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4886f2d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488019b820 con 0x7f4880105550 2023-12-14T18:36:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4884cc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 0x7f4880196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4884cc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 0x7f4880196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36806/0 (socket says 172.21.15.31:36806) 2023-12-14T18:36:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f4884cc9700 1 -- 172.21.15.31:0/940038351 learned_addr learned my addr 172.21.15.31:0/940038351 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.751+0000 7f48854ca700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880105550 0x7f488019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f487ffff700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 0x7f4880196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4884cc9700 1 -- 172.21.15.31:0/940038351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880105550 msgr2=0x7f488019aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4884cc9700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880105550 0x7f488019aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4884cc9700 1 -- 172.21.15.31:0/940038351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 msgr2=0x7f4880196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4884cc9700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 0x7f4880196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4884cc9700 1 -- 172.21.15.31:0/940038351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4878009d00 con 0x7f4880100c50 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f48854ca700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880105550 0x7f488019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:45.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f487ffff700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 0x7f4880196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4884cc9700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 0x7f4880196510 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f487000e9a0 tx=0x7f487000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f487dffb700 1 -- 172.21.15.31:0/940038351 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870009790 con 0x7f4880100c50 2023-12-14T18:36:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f487dffb700 1 -- 172.21.15.31:0/940038351 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f487000cbd0 con 0x7f4880100c50 2023-12-14T18:36:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f487dffb700 1 -- 172.21.15.31:0/940038351 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870018680 con 0x7f4880100c50 2023-12-14T18:36:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4880110320 con 0x7f4880100c50 2023-12-14T18:36:45.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.752+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4880110190 con 0x7f4880100c50 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.755+0000 7f487dffb700 1 -- 172.21.15.31:0/940038351 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f487000cd40 con 0x7f4880100c50 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.756+0000 7f487dffb700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f48680a1340 0x7f48680a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.756+0000 7f487dffb700 1 -- 172.21.15.31:0/940038351 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f4870014070 con 0x7f4880100c50 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.756+0000 7f487ffff700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f48680a1340 0x7f48680a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f487ffff700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f48680a1340 0x7f48680a37f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4874005e10 tx=0x7f4874005d60 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 Processor -- start 2023-12-14T18:36:45.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 -- start start 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f09e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ef88660 con 0x55862f09e400 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862eff78f0 con 0x55862f0a9bd0 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48930bbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862eff0d80 con 0x55862f09ed10 2023-12-14T18:36:45.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f09e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f09e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36820/0 (socket says 172.21.15.31:36820) 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 -- 172.21.15.31:0/2450398562 learned_addr learned my addr 172.21.15.31:0/2450398562 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f487ffff700 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48854ca700 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 -- 172.21.15.31:0/2450398562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 msgr2=0x55862f0a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 -- 172.21.15.31:0/2450398562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 msgr2=0x55862f0abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f4884cc9700 1 -- 172.21.15.31:0/2450398562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862ee72430 con 0x55862f09e400 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.757+0000 7f48854ca700 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.758+0000 7f4884cc9700 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f09e7d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f487000f010 tx=0x7f487009a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2790e031129a8d0c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.758+0000 7f4866ffd700 1 -- 172.21.15.31:0/2450398562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700984a0 con 0x55862f09e400 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.758+0000 7f4866ffd700 1 -- 172.21.15.31:0/2450398562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48700cd170 con 0x55862f09e400 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.758+0000 7f4866ffd700 1 -- 172.21.15.31:0/2450398562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700a20c0 con 0x55862f09e400 2023-12-14T18:36:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 -- 172.21.15.31:0/2450398562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 msgr2=0x55862f09e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f09e7d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f487000f010 tx=0x7f487009a1d0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 -- 172.21.15.31:0/2450398562 shutdown_connections 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f09e7d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2450398562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 -- 172.21.15.31:0/2450398562 >> 172.21.15.31:0/2450398562 conn(0x55862f094d20 msgr2=0x55862f09a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 -- 172.21.15.31:0/2450398562 shutdown_connections 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 -- 172.21.15.31:0/2450398562 wait complete. 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 -- start start 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.759+0000 7f48930bbb80 1 Processor -- start 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09e400 0x55862f0a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 0x55862f0a6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ef8d010 con 0x55862f09ed10 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ef8ccb0 con 0x55862f0a9bd0 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862ee74720 con 0x55862f09e400 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48930bbb80 10 client.0 _ll_register_callbacks cb 0x55862efb0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 0x55862f0a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 0x55862f0a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36834/0 (socket says 172.21.15.31:36834) 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 -- 172.21.15.31:0/789804970 learned_addr learned my addr 172.21.15.31:0/789804970 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f48854ca700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f4884cc9700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09e400 0x55862f0a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 -- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09e400 msgr2=0x55862f0a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09e400 0x55862f0a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 -- 172.21.15.31:0/789804970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 msgr2=0x55862f0a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f487ffff700 1 -- 172.21.15.31:0/789804970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4880110190 con 0x55862f09ed10 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.760+0000 7f4884cc9700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09e400 0x55862f0a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f487ffff700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 0x55862f0a6890 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f48801019c0 tx=0x7f4874004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5101715b276daefe server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f485ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f485ffff700 1 -- 172.21.15.31:0/789804970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874017040 con 0x55862f09ed10 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55862f092330 con 0x55862f09ed10 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f485ffff700 1 -- 172.21.15.31:0/789804970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f4874003a90 con 0x55862f09ed10 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f485ffff700 1 -- 172.21.15.31:0/789804970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4874003c50 con 0x55862f09ed10 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 10 client.17541 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55862efb71f0 con 0x55862f09ed10 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f485ffff700 1 -- 172.21.15.31:0/789804970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55862f092330 con 0x55862f09ed10 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f485ffff700 1 -- 172.21.15.31:0/789804970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f487401d050 con 0x55862f09ed10 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 10 client.17541 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 20 client.17541 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0b9530 0x55862f0bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 4 client.17541 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.761+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55862efe9d60 con 0x55862f0b9530 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.762+0000 7f4884cc9700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0b9530 0x55862f0bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.762+0000 7f485ffff700 10 client.17541 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.762+0000 7f4884cc9700 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0b9530 0x55862f0bb940 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82131111db413486 server_cookie=9e113d1f0344f5a4 in_seq=0 out_seq=0 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f485ffff700 1 -- 172.21.15.31:0/789804970 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55862efe9d60 con 0x55862f0b9530 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f485ffff700 10 client.17541 handle_command_reply: tid=0 2023-12-14T18:36:45.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 client.17541 shutdown 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 msgr2=0x55862f0a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 0x55862f0a6890 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f48801019c0 tx=0x7f4874004750 comp rx=0 tx=0).stop 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 shutdown_connections 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0b9530 0x55862f0bb940 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f0a9bd0 0x55862f0a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09e400 0x55862f0a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f485ffff700 0 client.17541 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 --2- 172.21.15.31:0/789804970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09ed10 0x55862f0a6890 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 >> 172.21.15.31:0/789804970 conn(0x55862f094d20 msgr2=0x55862f09a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 shutdown_connections 2023-12-14T18:36:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 1 -- 172.21.15.31:0/789804970 wait complete. 2023-12-14T18:36:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.763+0000 7f48930bbb80 20 client.17541 trim_cache size 0 max 16384 2023-12-14T18:36:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.781+0000 7f48930bbb80 1 Processor -- start 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.781+0000 7f48930bbb80 1 -- start start 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.781+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f094d20 0x55862f0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.781+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48930bbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862f0a7ed0 con 0x55862f09e400 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48930bbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862efe9d60 con 0x55862f094d20 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48930bbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862f0a26e0 con 0x55862f09ed10 2023-12-14T18:36:45.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f0b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36850/0 (socket says 172.21.15.31:36850) 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 -- 172.21.15.31:0/2683525733 learned_addr learned my addr 172.21.15.31:0/2683525733 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f4884cc9700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f094d20 0x55862f0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 -- 172.21.15.31:0/2683525733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 msgr2=0x55862f0a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48854ca700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 -- 172.21.15.31:0/2683525733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f094d20 msgr2=0x55862f0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f094d20 0x55862f0a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 -- 172.21.15.31:0/2683525733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55862efb6f40 con 0x55862f09e400 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f4884cc9700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f094d20 0x55862f0a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48854ca700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f487ffff700 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f0b0710 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f487400b830 tx=0x7f4874005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fb0ef9673ff20 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48677fe700 1 -- 172.21.15.31:0/2683525733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874011940 con 0x55862f09e400 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48677fe700 1 -- 172.21.15.31:0/2683525733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4874037070 con 0x55862f09e400 2023-12-14T18:36:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.782+0000 7f48677fe700 1 -- 172.21.15.31:0/2683525733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740206a0 con 0x55862f09e400 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 -- 172.21.15.31:0/2683525733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 msgr2=0x55862f0b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f0b0710 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f487400b830 tx=0x7f4874005ee0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 -- 172.21.15.31:0/2683525733 shutdown_connections 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f0a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f094d20 0x55862f0a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 --2- 172.21.15.31:0/2683525733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f09e400 0x55862f0b0710 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 -- 172.21.15.31:0/2683525733 >> 172.21.15.31:0/2683525733 conn(0x55862f0ac280 msgr2=0x55862f0a1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 -- 172.21.15.31:0/2683525733 shutdown_connections 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 -- 172.21.15.31:0/2683525733 wait complete. 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.783+0000 7f48930bbb80 1 -- start start 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 Processor -- start 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 0x55862f0af950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f09e400 0x55862f0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f1065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862f106b70 con 0x55862f094d20 2023-12-14T18:36:45.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862f106ce0 con 0x55862f09e400 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55862f106ea0 con 0x55862f09ed10 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48930bbb80 10 client.0 _ll_register_callbacks cb 0x7f4874004140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 0x55862f0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 0x55862f0af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36864/0 (socket says 172.21.15.31:36864) 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 -- 172.21.15.31:0/1124536697 learned_addr learned my addr 172.21.15.31:0/1124536697 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f487ffff700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f09e400 0x55862f0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48854ca700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f1065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 -- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 msgr2=0x55862f1065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f1065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 -- 172.21.15.31:0/1124536697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f09e400 msgr2=0x55862f0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f09e400 0x55862f0afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f4884cc9700 1 -- 172.21.15.31:0/1124536697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55862efb71f0 con 0x55862f094d20 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.784+0000 7f48854ca700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f1065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f4884cc9700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 0x55862f0af950 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f48700a2020 tx=0x7f4870099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6a5d5a3085bf6a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f485ffff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f485ffff700 1 -- 172.21.15.31:0/1124536697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487009a570 con 0x55862f094d20 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48700cd170 con 0x55862f094d20 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f485ffff700 1 -- 172.21.15.31:0/1124536697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f48700995f0 con 0x55862f094d20 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f485ffff700 1 -- 172.21.15.31:0/1124536697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48700997b0 con 0x55862f094d20 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 10 client.17547 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55862ef9da20 con 0x55862f094d20 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f485ffff700 1 -- 172.21.15.31:0/1124536697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48700cd170 con 0x55862f094d20 2023-12-14T18:36:45.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f485ffff700 1 -- 172.21.15.31:0/1124536697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4870095ef0 con 0x55862f094d20 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 10 client.17547 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 20 client.17547 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0a9bd0 0x55862f111330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 4 client.17547 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.785+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55862efe9a80 con 0x55862f0a9bd0 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.786+0000 7f487ffff700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0a9bd0 0x55862f111330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.786+0000 7f487ffff700 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0a9bd0 0x55862f111330 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70cc82ab4234a82 server_cookie=6cc9402e6e0610c6 in_seq=0 out_seq=0 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.786+0000 7f485ffff700 10 client.17547 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f485ffff700 1 -- 172.21.15.31:0/1124536697 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55862efe9a80 con 0x55862f0a9bd0 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f485ffff700 10 client.17547 handle_command_reply: tid=0 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 client.17547 shutdown 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 msgr2=0x55862f0af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 0x55862f0af950 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f48700a2020 tx=0x7f4870099d70 comp rx=0 tx=0).stop 2023-12-14T18:36:45.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 shutdown_connections 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55862f0a9bd0 0x55862f111330 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55862f09ed10 0x55862f1065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55862f094d20 0x55862f0af950 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f485ffff700 0 client.17547 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 --2- 172.21.15.31:0/1124536697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55862f09e400 0x55862f0afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.787+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 >> 172.21.15.31:0/1124536697 conn(0x55862f0ac280 msgr2=0x55862f0a02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 shutdown_connections 2023-12-14T18:36:45.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f48930bbb80 1 -- 172.21.15.31:0/1124536697 wait complete. 2023-12-14T18:36:45.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f48930bbb80 20 client.17547 trim_cache size 0 max 16384 2023-12-14T18:36:45.803 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:36:45.803 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:36:45.803 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "75647bcc-6e51-47c8-a26e-806c308c0177", 2023-12-14T18:36:45.803 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:36:45.803 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:36:45.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f48680a1340 msgr2=0x7f48680a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f48680a1340 0x7f48680a37f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4874005e10 tx=0x7f4874005d60 comp rx=0 tx=0).stop 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 msgr2=0x7f4880196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 0x7f4880196510 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f487000e9a0 tx=0x7f487000ed60 comp rx=0 tx=0).stop 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 shutdown_connections 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f48680a1340 0x7f48680a37f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4880105550 0x7f488019aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4880100c50 0x7f4880196510 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 --2- 172.21.15.31:0/940038351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f48801015f0 0x7f4880196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.788+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 >> 172.21.15.31:0/940038351 conn(0x7f4880077570 msgr2=0x7f48800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.789+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 shutdown_connections 2023-12-14T18:36:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.789+0000 7f4886f2d700 1 -- 172.21.15.31:0/940038351 wait complete. 2023-12-14T18:36:45.806 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:45 smithi031 ceph-mon[42085]: pgmap v277: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:45.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:36:45.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:45 smithi204 ceph-mon[43866]: pgmap v277: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 -- 172.21.15.31:0/4062682739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f4104400 msgr2=0x7f60f410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 --2- 172.21.15.31:0/4062682739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f4104400 0x7f60f410bfd0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b0d0 tx=0x7f60ec00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 -- 172.21.15.31:0/4062682739 shutdown_connections 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 --2- 172.21.15.31:0/4062682739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f4104400 0x7f60f410bfd0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 --2- 172.21.15.31:0/4062682739 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60f40ff3d0 0x7f60f40ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 --2- 172.21.15.31:0/4062682739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f40ffd70 0x7f60f4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 -- 172.21.15.31:0/4062682739 >> 172.21.15.31:0/4062682739 conn(0x7f60f4075ff0 msgr2=0x7f60f40763f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 -- 172.21.15.31:0/4062682739 shutdown_connections 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.947+0000 7f60fc41d700 1 -- 172.21.15.31:0/4062682739 wait complete. 2023-12-14T18:36:45.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 Processor -- start 2023-12-14T18:36:45.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 -- start start 2023-12-14T18:36:45.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60f40ff3d0 0x7f60f4195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 0x7f60f4195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f4104400 0x7f60f4195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f41a4960 con 0x7f60f40ffd70 2023-12-14T18:36:45.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f41a4ad0 con 0x7f60f40ff3d0 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fc41d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f41a4c90 con 0x7f60f4104400 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 0x7f60f4195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 0x7f60f4195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36876/0 (socket says 172.21.15.31:36876) 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 -- 172.21.15.31:0/3150539844 learned_addr learned my addr 172.21.15.31:0/3150539844 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60f40ff3d0 0x7f60f4195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fa9ba700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f4104400 0x7f60f4195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 -- 172.21.15.31:0/3150539844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f4104400 msgr2=0x7f60f4195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f4104400 0x7f60f4195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 -- 172.21.15.31:0/3150539844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60f40ff3d0 msgr2=0x7f60f4195210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60f40ff3d0 0x7f60f4195210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60f99b8700 1 -- 172.21.15.31:0/3150539844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009d00 con 0x7f60f40ffd70 2023-12-14T18:36:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.948+0000 7f60fa9ba700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f4104400 0x7f60f4195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:45.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.949+0000 7f60f99b8700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 0x7f60f4195750 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d950 tx=0x7f60e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.949+0000 7f60eb7fe700 1 -- 172.21.15.31:0/3150539844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4004220 con 0x7f60f40ffd70 2023-12-14T18:36:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.949+0000 7f60eb7fe700 1 -- 172.21.15.31:0/3150539844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60e4004cf0 con 0x7f60f40ffd70 2023-12-14T18:36:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.949+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f410f1d0 con 0x7f60f40ffd70 2023-12-14T18:36:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.949+0000 7f60eb7fe700 1 -- 172.21.15.31:0/3150539844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40056c0 con 0x7f60f40ffd70 2023-12-14T18:36:45.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.949+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f410f040 con 0x7f60f40ffd70 2023-12-14T18:36:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.951+0000 7f60eb7fe700 1 -- 172.21.15.31:0/3150539844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f60e4004380 con 0x7f60f40ffd70 2023-12-14T18:36:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.952+0000 7f60eb7fe700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f60e00a12c0 0x7f60e00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.952+0000 7f60eb7fe700 1 -- 172.21.15.31:0/3150539844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f60e40cceb0 con 0x7f60f40ffd70 2023-12-14T18:36:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.952+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f60e00a12c0 0x7f60e00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.952+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f60e00a12c0 0x7f60e00a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009c60 tx=0x7f60f0009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 Processor -- start 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 -- start start 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 0x56060b828750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b833610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b835f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b7125e0 con 0x56060b828380 2023-12-14T18:36:45.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b781870 con 0x56060b828c90 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f61085abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b77ad00 con 0x56060b833b50 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f60fa1b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 0x56060b828750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f60fa1b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 0x56060b828750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36890/0 (socket says 172.21.15.31:36890) 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.953+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3124483945 learned_addr learned my addr 172.21.15.31:0/3124483945 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa9ba700 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b835f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60f99b8700 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b833610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3124483945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 msgr2=0x56060b835f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b835f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3124483945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 msgr2=0x56060b833610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b833610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3124483945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060b5fc3b0 con 0x56060b828380 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60f99b8700 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b833610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 0x56060b828750 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f60f000e4e0 tx=0x7f60f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3d3aff7ce68d4c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60e8dac700 1 -- 172.21.15.31:0/3124483945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f000ee00 con 0x56060b828380 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60e8dac700 1 -- 172.21.15.31:0/3124483945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60f0021070 con 0x56060b828380 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.954+0000 7f60e8dac700 1 -- 172.21.15.31:0/3124483945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f001d7b0 con 0x56060b828380 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 -- 172.21.15.31:0/3124483945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 msgr2=0x56060b828750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 0x56060b828750 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f60f000e4e0 tx=0x7f60f0014000 comp rx=0 tx=0).stop 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 -- 172.21.15.31:0/3124483945 shutdown_connections 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b835f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828380 0x56060b828750 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 --2- 172.21.15.31:0/3124483945 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b833610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 -- 172.21.15.31:0/3124483945 >> 172.21.15.31:0/3124483945 conn(0x56060b81eca0 msgr2=0x56060b8247e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 -- 172.21.15.31:0/3124483945 shutdown_connections 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.955+0000 7f61085abb80 1 -- 172.21.15.31:0/3124483945 wait complete. 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 -- start start 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 Processor -- start 2023-12-14T18:36:45.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828380 0x56060b833500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b8307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b830d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b716f90 con 0x56060b828c90 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b716c30 con 0x56060b828380 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b5fe6a0 con 0x56060b833b50 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f61085abb80 10 client.0 _ll_register_callbacks cb 0x56060b73a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b8307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b8307b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36898/0 (socket says 172.21.15.31:36898) 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 -- 172.21.15.31:0/1333273062 learned_addr learned my addr 172.21.15.31:0/1333273062 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828380 0x56060b833500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60fa9ba700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b830d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 -- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 msgr2=0x56060b830d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b830d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 -- 172.21.15.31:0/1333273062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828380 msgr2=0x56060b833500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828380 0x56060b833500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60f99b8700 1 -- 172.21.15.31:0/1333273062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f0017070 con 0x56060b828c90 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.956+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828380 0x56060b833500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60f99b8700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b8307b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f60e4099890 tx=0x7f60e4099b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404016d55e36e112 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60ddffb700 1 -- 172.21.15.31:0/1333273062 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e4099ff0 con 0x56060b828c90 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60ddffb700 1 -- 172.21.15.31:0/1333273062 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f60e40a1fe0 con 0x56060b828c90 2023-12-14T18:36:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56060b81c2b0 con 0x56060b828c90 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60ddffb700 1 -- 172.21.15.31:0/1333273062 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60e40b2200 con 0x56060b828c90 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 10 client.17559 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56060b741170 con 0x56060b828c90 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60ddffb700 1 -- 172.21.15.31:0/1333273062 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56060b81c2b0 con 0x56060b828c90 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f60ddffb700 1 -- 172.21.15.31:0/1333273062 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60e4098af0 con 0x56060b828c90 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 10 client.17559 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 20 client.17559 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b843370 0x56060b845750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 4 client.17559 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.957+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56060b773ce0 con 0x56060b843370 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.958+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b843370 0x56060b845750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.958+0000 7f60ddffb700 10 client.17559 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.958+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b843370 0x56060b845750 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=988625a63c0d7da0 server_cookie=2f2604ee15846dd in_seq=0 out_seq=0 2023-12-14T18:36:45.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f60ddffb700 1 -- 172.21.15.31:0/1333273062 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56060b773ce0 con 0x56060b843370 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f60ddffb700 10 client.17559 handle_command_reply: tid=0 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 client.17559 shutdown 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 msgr2=0x56060b8307b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b8307b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f60e4099890 tx=0x7f60e4099b90 comp rx=0 tx=0).stop 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 shutdown_connections 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b843370 0x56060b845750 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b833b50 0x56060b830d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828380 0x56060b833500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f60ddffb700 0 client.17559 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 --2- 172.21.15.31:0/1333273062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b8307b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 >> 172.21.15.31:0/1333273062 conn(0x56060b81eca0 msgr2=0x56060b8243d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 shutdown_connections 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 1 -- 172.21.15.31:0/1333273062 wait complete. 2023-12-14T18:36:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.959+0000 7f61085abb80 20 client.17559 trim_cache size 0 max 16384 2023-12-14T18:36:45.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 Processor -- start 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 -- start start 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 0x56060b83a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b83ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b832fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b8295e0 con 0x56060b81eca0 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b843150 con 0x56060b828c90 2023-12-14T18:36:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f61085abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b82ed10 con 0x56060b828380 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 0x56060b83a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 0x56060b83a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36902/0 (socket says 172.21.15.31:36902) 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3219212187 learned_addr learned my addr 172.21.15.31:0/3219212187 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa9ba700 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b832fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60f99b8700 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b83ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3219212187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 msgr2=0x56060b83ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b83ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3219212187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 msgr2=0x56060b832fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b832fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.977+0000 7f60fa1b9700 1 -- 172.21.15.31:0/3219212187 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060b740ec0 con 0x56060b81eca0 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 0x56060b83a690 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f60f000e4e0 tx=0x7f60f000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b409b21f2929 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f60e95ad700 1 -- 172.21.15.31:0/3219212187 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f000ed10 con 0x56060b81eca0 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f60e95ad700 1 -- 172.21.15.31:0/3219212187 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60f0032110 con 0x56060b81eca0 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f60e95ad700 1 -- 172.21.15.31:0/3219212187 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0005640 con 0x56060b81eca0 2023-12-14T18:36:45.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 -- 172.21.15.31:0/3219212187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 msgr2=0x56060b83a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 0x56060b83a690 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f60f000e4e0 tx=0x7f60f000ec20 comp rx=0 tx=0).stop 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 -- 172.21.15.31:0/3219212187 shutdown_connections 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b828c90 0x56060b832fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b81eca0 0x56060b83a690 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 --2- 172.21.15.31:0/3219212187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b83ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 -- 172.21.15.31:0/3219212187 >> 172.21.15.31:0/3219212187 conn(0x56060b836200 msgr2=0x56060b83f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 -- 172.21.15.31:0/3219212187 shutdown_connections 2023-12-14T18:36:45.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.978+0000 7f61085abb80 1 -- 172.21.15.31:0/3219212187 wait complete. 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 -- start start 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 Processor -- start 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b81eca0 0x56060b83a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b838a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b838f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b839660 con 0x56060b828c90 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b8397d0 con 0x56060b81eca0 2023-12-14T18:36:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060b897210 con 0x56060b828380 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f61085abb80 10 client.0 _ll_register_callbacks cb 0x7f60e40999a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f60fa9ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b838f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f60fa1b9700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b81eca0 0x56060b83a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f60fa9ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b838f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36904/0 (socket says 172.21.15.31:36904) 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f60fa9ba700 1 -- 172.21.15.31:0/2660046385 learned_addr learned my addr 172.21.15.31:0/2660046385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.979+0000 7f60f99b8700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b838a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa1b9700 1 -- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 msgr2=0x56060b838a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b838a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa1b9700 1 -- 172.21.15.31:0/2660046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 msgr2=0x56060b838f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b838f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa1b9700 1 -- 172.21.15.31:0/2660046385 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f002a040 con 0x56060b81eca0 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa9ba700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b838f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60fa1b9700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b81eca0 0x56060b83a6e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60f0035040 tx=0x7f60f0009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a3966c5f46d311 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:45.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60ddffb700 1 -- 172.21.15.31:0/2660046385 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0032480 con 0x56060b81eca0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60ddffb700 1 -- 172.21.15.31:0/2660046385 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f60f0032a70 con 0x56060b81eca0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60f0021070 con 0x56060b81eca0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f60ddffb700 1 -- 172.21.15.31:0/2660046385 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60f00325e0 con 0x56060b81eca0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f61085abb80 10 client.27272 fetch_fsmap learned FSMap version 34 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.980+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56060b7279a0 con 0x56060b81eca0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f60ddffb700 1 -- 172.21.15.31:0/2660046385 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60f0021070 con 0x56060b81eca0 2023-12-14T18:36:45.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f60ddffb700 1 -- 172.21.15.31:0/2660046385 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60f0003a60 con 0x56060b81eca0 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f61085abb80 10 client.27272 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f61085abb80 10 client.27272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f61085abb80 20 client.27272 populate_metadata read hostname 'smithi031' 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f61085abb80 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b833b50 0x56060b89a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f61085abb80 4 client.27272 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56060b773a00 con 0x56060b833b50 2023-12-14T18:36:45.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.981+0000 7f60f99b8700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b833b50 0x56060b89a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:45.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:45 smithi176 ceph-mon[43506]: pgmap v277: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f60f99b8700 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b833b50 0x56060b89a580 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a063c6816d163d server_cookie=92dbb9ade38a187b in_seq=0 out_seq=0 2023-12-14T18:36:45.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f60ddffb700 10 client.27272 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f60ddffb700 1 -- 172.21.15.31:0/2660046385 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56060b773a00 con 0x56060b833b50 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f60ddffb700 10 client.27272 handle_command_reply: tid=0 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 client.27272 shutdown 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b81eca0 msgr2=0x56060b83a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b81eca0 0x56060b83a6e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60f0035040 tx=0x7f60f0009f80 comp rx=0 tx=0).stop 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 shutdown_connections 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56060b833b50 0x56060b89a580 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56060b828c90 0x56060b838f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f60ddffb700 0 client.27272 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56060b81eca0 0x56060b83a6e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 --2- 172.21.15.31:0/2660046385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56060b828380 0x56060b838a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 >> 172.21.15.31:0/2660046385 conn(0x56060b836200 msgr2=0x56060b83ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 shutdown_connections 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 1 -- 172.21.15.31:0/2660046385 wait complete. 2023-12-14T18:36:45.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.982+0000 7f61085abb80 20 client.27272 trim_cache size 0 max 16384 2023-12-14T18:36:45.997 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:36:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:36:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:36:45.997 INFO:teuthology.orchestra.run.smithi031.stdout: "75647bcc-6e51-47c8-a26e-806c308c0177": { 2023-12-14T18:36:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:36:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "75647bcc-6e51-47c8-a26e-806c308c0177", 2023-12-14T18:36:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:36:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:36:45.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:36:45.998 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:36:45.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f60e00a12c0 msgr2=0x7f60e00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f60e00a12c0 0x7f60e00a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009c60 tx=0x7f60f0009260 comp rx=0 tx=0).stop 2023-12-14T18:36:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 msgr2=0x7f60f4195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 0x7f60f4195750 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60e400d950 tx=0x7f60e400dc60 comp rx=0 tx=0).stop 2023-12-14T18:36:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 shutdown_connections 2023-12-14T18:36:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f60e00a12c0 0x7f60e00a3770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:46.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60f4104400 0x7f60f4195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60f40ff3d0 0x7f60f4195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 --2- 172.21.15.31:0/3150539844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40ffd70 0x7f60f4195750 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 >> 172.21.15.31:0/3150539844 conn(0x7f60f4075ff0 msgr2=0x7f60f40fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 shutdown_connections 2023-12-14T18:36:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:36:45.983+0000 7f60fc41d700 1 -- 172.21.15.31:0/3150539844 wait complete. 2023-12-14T18:36:46.001 INFO:tasks.cephfs.filesystem:scrub status for tag:75647bcc-6e51-47c8-a26e-806c308c0177 - {'path': '/', 'tag': '75647bcc-6e51-47c8-a26e-806c308c0177', 'options': 'recursive,force'} 2023-12-14T18:36:46.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:46 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:36:46.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:46 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:46 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:36:46.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:46 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:36:46.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:46 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:46 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:46 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:36:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:46 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:46 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:36:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:46 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:36:46.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:46 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:46 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:46 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:36:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:46 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:46 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:36:46.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:46 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:36:46.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:46 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:46.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:46 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:36:47.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:47 smithi204 ceph-mon[43866]: pgmap v278: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:47.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:47 smithi031 ceph-mon[42085]: pgmap v278: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:47.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:47 smithi176 ceph-mon[43506]: pgmap v278: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:49 smithi204 ceph-mon[43866]: pgmap v279: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:49 smithi031 ceph-mon[42085]: pgmap v279: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:49.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:49 smithi176 ceph-mon[43506]: pgmap v279: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:51 smithi204 ceph-mon[43866]: pgmap v280: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:51.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:51 smithi031 ceph-mon[42085]: pgmap v280: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:51.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:51 smithi176 ceph-mon[43506]: pgmap v280: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:53 smithi204 ceph-mon[43866]: pgmap v281: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:53.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:53 smithi031 ceph-mon[42085]: pgmap v281: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:53 smithi176 ceph-mon[43506]: pgmap v281: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:54.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:54 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:36:54.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:54 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:36:54.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:54 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:36:54.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:54 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:36:54.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:54 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:36:54.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:54 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:36:55.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:55 smithi204 ceph-mon[43866]: pgmap v282: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:55.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:55 smithi031 ceph-mon[42085]: pgmap v282: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:55.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:55 smithi176 ceph-mon[43506]: pgmap v282: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:57 smithi204 ceph-mon[43866]: pgmap v283: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:57.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:57 smithi031 ceph-mon[42085]: pgmap v283: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:57 smithi176 ceph-mon[43506]: pgmap v283: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:36:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:36:59 smithi204 ceph-mon[43866]: pgmap v284: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:36:59 smithi031 ceph-mon[42085]: pgmap v284: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:36:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:36:59 smithi176 ceph-mon[43506]: pgmap v284: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:37:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:01 smithi204 ceph-mon[43866]: pgmap v285: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:01.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:01 smithi031 ceph-mon[42085]: pgmap v285: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:01 smithi176 ceph-mon[43506]: pgmap v285: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:03 smithi204 ceph-mon[43866]: pgmap v286: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:03 smithi031 ceph-mon[42085]: pgmap v286: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:03 smithi176 ceph-mon[43506]: pgmap v286: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:05.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:05 smithi031 ceph-mon[42085]: pgmap v287: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:37:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:05 smithi176 ceph-mon[43506]: pgmap v287: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:37:06.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:05 smithi204 ceph-mon[43866]: pgmap v287: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:37:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:07 smithi031 ceph-mon[42085]: pgmap v288: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:37:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:07 smithi176 ceph-mon[43506]: pgmap v288: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:37:08.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:07 smithi204 ceph-mon[43866]: pgmap v288: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2023-12-14T18:37:09.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:09 smithi031 ceph-mon[42085]: pgmap v289: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:37:09.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:09 smithi176 ceph-mon[43506]: pgmap v289: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:37:10.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:09 smithi204 ceph-mon[43866]: pgmap v289: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:37:10.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:10 smithi031 ceph-mon[42085]: osdmap e97: 12 total, 12 up, 12 in 2023-12-14T18:37:10.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:10 smithi176 ceph-mon[43506]: osdmap e97: 12 total, 12 up, 12 in 2023-12-14T18:37:11.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:10 smithi204 ceph-mon[43866]: osdmap e97: 12 total, 12 up, 12 in 2023-12-14T18:37:11.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:11 smithi031 ceph-mon[42085]: pgmap v291: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:37:11.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:11 smithi176 ceph-mon[43506]: pgmap v291: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:37:12.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:11 smithi204 ceph-mon[43866]: pgmap v291: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:37:13.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:13 smithi031 ceph-mon[42085]: pgmap v292: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:37:13.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:13 smithi176 ceph-mon[43506]: pgmap v292: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:37:14.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:13 smithi204 ceph-mon[43866]: pgmap v292: 129 pgs: 129 active+clean; 544 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:37:15.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:15 smithi031 ceph-mon[42085]: pgmap v293: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 544 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:37:15.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:15 smithi176 ceph-mon[43506]: pgmap v293: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 544 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:37:16.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:37:16.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:15 smithi204 ceph-mon[43866]: pgmap v293: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 544 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:37:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 -- 172.21.15.31:0/1751880691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c107d60 msgr2=0x7f2d2c108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 --2- 172.21.15.31:0/1751880691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c107d60 0x7f2d2c108130 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d960 tx=0x7f2d2000dc70 comp rx=0 tx=0).stop 2023-12-14T18:37:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 -- 172.21.15.31:0/1751880691 shutdown_connections 2023-12-14T18:37:16.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 --2- 172.21.15.31:0/1751880691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2c1111d0 0x7f2d2c113630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 --2- 172.21.15.31:0/1751880691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c107d60 0x7f2d2c108130 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 --2- 172.21.15.31:0/1751880691 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2c108670 0x7f2d2c110b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.269+0000 7f2d32df7700 1 -- 172.21.15.31:0/1751880691 >> 172.21.15.31:0/1751880691 conn(0x7f2d2c06b7e0 msgr2=0x7f2d2c071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.272+0000 7f2d32df7700 1 -- 172.21.15.31:0/1751880691 shutdown_connections 2023-12-14T18:37:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.276+0000 7f2d32df7700 1 -- 172.21.15.31:0/1751880691 wait complete. 2023-12-14T18:37:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.276+0000 7f2d32df7700 1 Processor -- start 2023-12-14T18:37:16.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 -- start start 2023-12-14T18:37:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2c107d60 0x7f2d2c1a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 0x7f2d2c1a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2c1111d0 0x7f2d2c1a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1aa1b0 con 0x7f2d2c108670 2023-12-14T18:37:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1aa320 con 0x7f2d2c107d60 2023-12-14T18:37:16.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.277+0000 7f2d32df7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c1aa4e0 con 0x7f2d2c1111d0 2023-12-14T18:37:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.279+0000 7f2d2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 0x7f2d2c1a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.279+0000 7f2d2bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 0x7f2d2c1a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52950/0 (socket says 172.21.15.31:52950) 2023-12-14T18:37:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.279+0000 7f2d2bfff700 1 -- 172.21.15.31:0/937764272 learned_addr learned my addr 172.21.15.31:0/937764272 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:16.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.279+0000 7f2d31394700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2c1111d0 0x7f2d2c1a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.282+0000 7f2d2bfff700 1 -- 172.21.15.31:0/937764272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2c1111d0 msgr2=0x7f2d2c1a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.282+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2c1111d0 0x7f2d2c1a9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.282+0000 7f2d2bfff700 1 -- 172.21.15.31:0/937764272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2c107d60 msgr2=0x7f2d2c1a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.282+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2c107d60 0x7f2d2c1a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.282+0000 7f2d2bfff700 1 -- 172.21.15.31:0/937764272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2000d590 con 0x7f2d2c108670 2023-12-14T18:37:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.282+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 0x7f2d2c1a5710 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800d950 tx=0x7f2d1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.285+0000 7f2d29ffb700 1 -- 172.21.15.31:0/937764272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18009940 con 0x7f2d2c108670 2023-12-14T18:37:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.286+0000 7f2d29ffb700 1 -- 172.21.15.31:0/937764272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d1800b6e0 con 0x7f2d2c108670 2023-12-14T18:37:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.286+0000 7f2d29ffb700 1 -- 172.21.15.31:0/937764272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800f3c0 con 0x7f2d2c108670 2023-12-14T18:37:16.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.287+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1168b0 con 0x7f2d2c108670 2023-12-14T18:37:16.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.287+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c116720 con 0x7f2d2c108670 2023-12-14T18:37:16.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.300+0000 7f2d3ef85b80 1 Processor -- start 2023-12-14T18:37:16.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.300+0000 7f2d3ef85b80 1 -- start start 2023-12-14T18:37:16.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1bdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a09a5e0 con 0x55bd6a1b0c90 2023-12-14T18:37:16.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a109870 con 0x55bd6a1b0380 2023-12-14T18:37:16.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a102d00 con 0x55bd6a1bbb50 2023-12-14T18:37:16.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52954/0 (socket says 172.21.15.31:52954) 2023-12-14T18:37:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 -- 172.21.15.31:0/1375670592 learned_addr learned my addr 172.21.15.31:0/1375670592 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 -- 172.21.15.31:0/1375670592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 msgr2=0x55bd6a1bdf30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1bdf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 -- 172.21.15.31:0/1375670592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 msgr2=0x55bd6a1b0750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1b0750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.301+0000 7f2d30b93700 1 -- 172.21.15.31:0/1375670592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd69f843b0 con 0x55bd6a1b0c90 2023-12-14T18:37:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.302+0000 7f2d30b93700 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb610 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d210 tx=0x7f2d20005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacd1444d0a9a5b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.302+0000 7f2d29ffb700 1 -- 172.21.15.31:0/937764272 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2d18009aa0 con 0x7f2d2c108670 2023-12-14T18:37:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.302+0000 7f2d16ffd700 1 -- 172.21.15.31:0/1375670592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000c330 con 0x55bd6a1b0c90 2023-12-14T18:37:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.302+0000 7f2d16ffd700 1 -- 172.21.15.31:0/1375670592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d2000c490 con 0x55bd6a1b0c90 2023-12-14T18:37:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.302+0000 7f2d16ffd700 1 -- 172.21.15.31:0/1375670592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000cd90 con 0x55bd6a1b0c90 2023-12-14T18:37:16.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.303+0000 7f2d29ffb700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d1c0a1450 0x7f2d1c0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.303+0000 7f2d29ffb700 1 -- 172.21.15.31:0/937764272 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2d180ccdd0 con 0x7f2d2c108670 2023-12-14T18:37:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.303+0000 7f2d31394700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d1c0a1450 0x7f2d1c0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.303+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/1375670592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 msgr2=0x55bd6a1bb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.303+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb610 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d210 tx=0x7f2d20005b20 comp rx=0 tx=0).stop 2023-12-14T18:37:16.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.304+0000 7f2d31394700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d1c0a1450 0x7f2d1c0a3900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d24009040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.305+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/1375670592 shutdown_connections 2023-12-14T18:37:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.305+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1bdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.305+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.305+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/1375670592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb610 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.305+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/1375670592 >> 172.21.15.31:0/1375670592 conn(0x55bd6a1a6ca0 msgr2=0x55bd6a1ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.312+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/1375670592 shutdown_connections 2023-12-14T18:37:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.318+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/1375670592 wait complete. 2023-12-14T18:37:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.319+0000 7f2d3ef85b80 1 -- start start 2023-12-14T18:37:16.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.319+0000 7f2d3ef85b80 1 Processor -- start 2023-12-14T18:37:16.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a09ef90 con 0x55bd6a1b0380 2023-12-14T18:37:16.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a09ec30 con 0x55bd6a1b0c90 2023-12-14T18:37:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd69f866a0 con 0x55bd6a1bbb50 2023-12-14T18:37:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d3ef85b80 10 client.0 _ll_register_callbacks cb 0x55bd6a0c2910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d30b93700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d30b93700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52970/0 (socket says 172.21.15.31:52970) 2023-12-14T18:37:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d30b93700 1 -- 172.21.15.31:0/3367924957 learned_addr learned my addr 172.21.15.31:0/3367924957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:16.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.321+0000 7f2d31394700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.322+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.322+0000 7f2d31394700 1 -- 172.21.15.31:0/3367924957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0c90 msgr2=0x55bd6a1b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:16.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.322+0000 7f2d31394700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.322+0000 7f2d31394700 1 -- 172.21.15.31:0/3367924957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0380 msgr2=0x55bd6a1bb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:16.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.322+0000 7f2d31394700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.322+0000 7f2d31394700 1 -- 172.21.15.31:0/3367924957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c1142f0 con 0x55bd6a1bbb50 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d31394700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b8e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d24018d50 tx=0x7f2d24004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8093e51cdca590 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3367924957 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24016b30 con 0x55bd6a1bbb50 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3367924957 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2d24016c90 con 0x55bd6a1bbb50 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd6a1a42b0 con 0x55bd6a1bbb50 2023-12-14T18:37:16.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3367924957 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d24016e50 con 0x55bd6a1bbb50 2023-12-14T18:37:16.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d3ef85b80 10 client.27352 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:16.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.323+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bd6a0c9170 con 0x55bd6a1bbb50 2023-12-14T18:37:16.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.327+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3367924957 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd6a1a42b0 con 0x55bd6a1bbb50 2023-12-14T18:37:16.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.327+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3367924957 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d2402b070 con 0x55bd6a1bbb50 2023-12-14T18:37:16.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.328+0000 7f2d3ef85b80 10 client.27352 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:16.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.328+0000 7f2d3ef85b80 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:16.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.328+0000 7f2d3ef85b80 20 client.27352 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:16.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.328+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a1cb400 0x55bd6a1cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.328+0000 7f2d3ef85b80 4 client.27352 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:16.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.328+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd6a0fbce0 con 0x55bd6a1cb400 2023-12-14T18:37:16.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.329+0000 7f2d30b93700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a1cb400 0x55bd6a1cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.329+0000 7f2d30b93700 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a1cb400 0x55bd6a1cd810 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6096979e7b1f58fd server_cookie=d9ddb1286bc05f2c in_seq=0 out_seq=0 2023-12-14T18:37:16.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.329+0000 7f2d0bfff700 10 client.27352 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.331+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3367924957 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd6a0fbce0 con 0x55bd6a1cb400 2023-12-14T18:37:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.331+0000 7f2d0bfff700 10 client.27352 handle_command_reply: tid=0 2023-12-14T18:37:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.331+0000 7f2d3ef85b80 1 client.27352 shutdown 2023-12-14T18:37:16.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.334+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 msgr2=0x55bd6a1b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.334+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b8e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d24018d50 tx=0x7f2d24004ad0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 shutdown_connections 2023-12-14T18:37:16.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a1cb400 0x55bd6a1cd810 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b8e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d0bfff700 0 client.27352 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:16.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3367924957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.335+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 >> 172.21.15.31:0/3367924957 conn(0x55bd6a1a6ca0 msgr2=0x55bd6a1ac3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.342+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 shutdown_connections 2023-12-14T18:37:16.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.349+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3367924957 wait complete. 2023-12-14T18:37:16.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.357+0000 7f2d3ef85b80 20 client.27352 trim_cache size 0 max 16384 2023-12-14T18:37:16.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.391+0000 7f2d3ef85b80 1 Processor -- start 2023-12-14T18:37:16.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.391+0000 7f2d3ef85b80 1 -- start start 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a1ae0e0 con 0x55bd6a1b0c90 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a1b6d80 con 0x55bd6a1b0380 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a1b4690 con 0x55bd6a1bbb50 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d31394700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d31394700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45524/0 (socket says 172.21.15.31:45524) 2023-12-14T18:37:16.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.392+0000 7f2d31394700 1 -- 172.21.15.31:0/419649575 learned_addr learned my addr 172.21.15.31:0/419649575 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:16.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.393+0000 7f2d31394700 1 -- 172.21.15.31:0/419649575 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 msgr2=0x55bd6a1bb010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:16.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.393+0000 7f2d31394700 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.393+0000 7f2d31394700 1 -- 172.21.15.31:0/419649575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 msgr2=0x55bd6a1bb550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:16.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.393+0000 7f2d31394700 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.393+0000 7f2d31394700 1 -- 172.21.15.31:0/419649575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd6a1a84e0 con 0x55bd6a1bbb50 2023-12-14T18:37:16.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.396+0000 7f2d31394700 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b3b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009010 tx=0x7f2d24009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82aecd413991e961 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.396+0000 7f2d177fe700 1 -- 172.21.15.31:0/419649575 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24006000 con 0x55bd6a1bbb50 2023-12-14T18:37:16.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.396+0000 7f2d177fe700 1 -- 172.21.15.31:0/419649575 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d24012040 con 0x55bd6a1bbb50 2023-12-14T18:37:16.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.396+0000 7f2d177fe700 1 -- 172.21.15.31:0/419649575 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24003a00 con 0x55bd6a1bbb50 2023-12-14T18:37:16.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/419649575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 msgr2=0x55bd6a1b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b3b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009010 tx=0x7f2d24009ec0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/419649575 shutdown_connections 2023-12-14T18:37:16.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1b3b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/419649575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/419649575 >> 172.21.15.31:0/419649575 conn(0x55bd6a1a6ca0 msgr2=0x55bd6a1c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/419649575 shutdown_connections 2023-12-14T18:37:16.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.397+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/419649575 wait complete. 2023-12-14T18:37:16.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.398+0000 7f2d3ef85b80 1 -- start start 2023-12-14T18:37:16.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.398+0000 7f2d3ef85b80 1 Processor -- start 2023-12-14T18:37:16.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1c2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a1c1630 con 0x55bd6a1bbb50 2023-12-14T18:37:16.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a1c17a0 con 0x55bd6a1b0380 2023-12-14T18:37:16.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd6a1c1960 con 0x55bd6a1b0c90 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d3ef85b80 10 client.0 _ll_register_callbacks cb 0x7f2d24018720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1c0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45536/0 (socket says 172.21.15.31:45536) 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 -- 172.21.15.31:0/3374294759 learned_addr learned my addr 172.21.15.31:0/3374294759 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d30b93700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 -- 172.21.15.31:0/3374294759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 msgr2=0x55bd6a1c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 -- 172.21.15.31:0/3374294759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1bbb50 msgr2=0x55bd6a1c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:16.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d2bfff700 1 -- 172.21.15.31:0/3374294759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd6a0c9170 con 0x55bd6a1b0c90 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.399+0000 7f2d30b93700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d2bfff700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1c0b10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d180964d0 tx=0x7f2d1809db00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3139290c7be4fd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d0bfff700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3374294759 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18014030 con 0x55bd6a1b0c90 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3374294759 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2d18099ec0 con 0x55bd6a1b0c90 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d2000c490 con 0x55bd6a1b0c90 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3374294759 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d180b2240 con 0x55bd6a1b0c90 2023-12-14T18:37:16.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d3ef85b80 10 client.27358 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.400+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bd69f30b60 con 0x55bd6a1b0c90 2023-12-14T18:37:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.401+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3374294759 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d2000c490 con 0x55bd6a1b0c90 2023-12-14T18:37:16.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3374294759 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d180989c0 con 0x55bd6a1b0c90 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d3ef85b80 10 client.27358 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d3ef85b80 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d3ef85b80 20 client.27358 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a220780 0x55bd6a222b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d3ef85b80 4 client.27358 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.402+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bd6a0fba00 con 0x55bd6a220780 2023-12-14T18:37:16.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.404+0000 7f2d30b93700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a220780 0x55bd6a222b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.406+0000 7f2d30b93700 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a220780 0x55bd6a222b90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ae3fb27af86f97 server_cookie=bc1b2795c51e0180 in_seq=0 out_seq=0 2023-12-14T18:37:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.406+0000 7f2d0bfff700 10 client.27358 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.407+0000 7f2d0bfff700 1 -- 172.21.15.31:0/3374294759 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bd6a0fba00 con 0x55bd6a220780 2023-12-14T18:37:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.407+0000 7f2d0bfff700 10 client.27358 handle_command_reply: tid=0 2023-12-14T18:37:16.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 client.27358 shutdown 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 msgr2=0x55bd6a1c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1c0b10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d180964d0 tx=0x7f2d1809db00 comp rx=0 tx=0).stop 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 shutdown_connections 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55bd6a220780 0x55bd6a222b90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bd6a1bbb50 0x55bd6a1c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bd6a1b0380 0x55bd6a1c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 --2- 172.21.15.31:0/3374294759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bd6a1b0c90 0x55bd6a1c0b10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 >> 172.21.15.31:0/3374294759 conn(0x55bd6a1a6ca0 msgr2=0x55bd6a1b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d0bfff700 0 client.27358 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.409+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 shutdown_connections 2023-12-14T18:37:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.410+0000 7f2d3ef85b80 1 -- 172.21.15.31:0/3374294759 wait complete. 2023-12-14T18:37:16.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.411+0000 7f2d3ef85b80 20 client.27358 trim_cache size 0 max 16384 2023-12-14T18:37:16.414 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:37:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:37:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:37:16.415 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:37:16.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.419+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d1c0a1450 msgr2=0x7f2d1c0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.419+0000 7f2d32df7700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d1c0a1450 0x7f2d1c0a3900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009fd0 tx=0x7f2d24009040 comp rx=0 tx=0).stop 2023-12-14T18:37:16.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.419+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 msgr2=0x7f2d2c1a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.419+0000 7f2d32df7700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 0x7f2d2c1a5710 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800d950 tx=0x7f2d1800dc60 comp rx=0 tx=0).stop 2023-12-14T18:37:16.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.421+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 shutdown_connections 2023-12-14T18:37:16.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.421+0000 7f2d32df7700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d1c0a1450 0x7f2d1c0a3900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.421+0000 7f2d32df7700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d2c1111d0 0x7f2d2c1a9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.421+0000 7f2d32df7700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d2c107d60 0x7f2d2c1a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.421+0000 7f2d32df7700 1 --2- 172.21.15.31:0/937764272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d2c108670 0x7f2d2c1a5710 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.421+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 >> 172.21.15.31:0/937764272 conn(0x7f2d2c06b7e0 msgr2=0x7f2d2c113380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.430+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 shutdown_connections 2023-12-14T18:37:16.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.430+0000 7f2d32df7700 1 -- 172.21.15.31:0/937764272 wait complete. 2023-12-14T18:37:16.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:37:16.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:37:16.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 -- 172.21.15.31:0/2163620749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d74101390 msgr2=0x7f2d7410d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/2163620749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d74101390 0x7f2d7410d060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c01c0d0 tx=0x7f2d6c01c3e0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 -- 172.21.15.31:0/2163620749 shutdown_connections 2023-12-14T18:37:16.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/2163620749 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d74101390 0x7f2d7410d060 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/2163620749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d74105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/2163620749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d741009e0 0x7f2d74100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.700+0000 7f2d7b47a700 1 -- 172.21.15.31:0/2163620749 >> 172.21.15.31:0/2163620749 conn(0x7f2d74077520 msgr2=0x7f2d74077920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.701+0000 7f2d7b47a700 1 -- 172.21.15.31:0/2163620749 shutdown_connections 2023-12-14T18:37:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.701+0000 7f2d7b47a700 1 -- 172.21.15.31:0/2163620749 wait complete. 2023-12-14T18:37:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.702+0000 7f2d7b47a700 1 Processor -- start 2023-12-14T18:37:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.702+0000 7f2d7b47a700 1 -- start start 2023-12-14T18:37:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.702+0000 7f2d7b47a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d741009e0 0x7f2d74107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.702+0000 7f2d7b47a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d74101390 0x7f2d741076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d7b47a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d7410b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d7b47a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74107e60 con 0x7f2d74101390 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d7b47a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7410b660 con 0x7f2d741009e0 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d7b47a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7410b820 con 0x7f2d741051b0 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d79a17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d7410b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d78a15700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d74101390 0x7f2d741076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d79a17700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d7410b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45540/0 (socket says 172.21.15.31:45540) 2023-12-14T18:37:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.703+0000 7f2d79a17700 1 -- 172.21.15.31:0/4057000671 learned_addr learned my addr 172.21.15.31:0/4057000671 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.704+0000 7f2d78a15700 1 -- 172.21.15.31:0/4057000671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 msgr2=0x7f2d7410b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.704+0000 7f2d78a15700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d7410b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.704+0000 7f2d78a15700 1 -- 172.21.15.31:0/4057000671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d741009e0 msgr2=0x7f2d74107180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:37:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.704+0000 7f2d78a15700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d741009e0 0x7f2d74107180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.704+0000 7f2d78a15700 1 -- 172.21.15.31:0/4057000671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c01ad10 con 0x7f2d74101390 2023-12-14T18:37:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.704+0000 7f2d78a15700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d74101390 0x7f2d741076c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b450 tx=0x7f2d6400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.705+0000 7f2d79a17700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d7410b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:16.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.706+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64011830 con 0x7f2d74101390 2023-12-14T18:37:16.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.706+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d64011e50 con 0x7f2d74101390 2023-12-14T18:37:16.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.706+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400f2e0 con 0x7f2d74101390 2023-12-14T18:37:16.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.706+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7406bf00 con 0x7f2d74101390 2023-12-14T18:37:16.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.706+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7406bd70 con 0x7f2d74101390 2023-12-14T18:37:16.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.713+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7406c090 con 0x7f2d74101390 2023-12-14T18:37:16.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.717+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2d64011990 con 0x7f2d74101390 2023-12-14T18:37:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.718+0000 7f2d6a7fc700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d600a14d0 0x7f2d600a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:16.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.718+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2d640ccfa0 con 0x7f2d74101390 2023-12-14T18:37:16.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.719+0000 7f2d79216700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d600a14d0 0x7f2d600a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:16.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.723+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d640960a0 con 0x7f2d74101390 2023-12-14T18:37:16.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:16.725+0000 7f2d79216700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d600a14d0 0x7f2d600a3980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009710 tx=0x7f2d70006c60 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:16.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:16 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:37:16.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:16 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:37:17.094 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:16 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:37:17.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.183+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d74074a30 con 0x7f2d74101390 2023-12-14T18:37:17.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.186+0000 7f2d6a7fc700 1 -- 172.21.15.31:0/4057000671 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f2d640996c0 con 0x7f2d74101390 2023-12-14T18:37:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:37:17.190 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:29.509045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:37:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d600a14d0 msgr2=0x7f2d600a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d600a14d0 0x7f2d600a3980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009710 tx=0x7f2d70006c60 comp rx=0 tx=0).stop 2023-12-14T18:37:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d74101390 msgr2=0x7f2d741076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d74101390 0x7f2d741076c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b450 tx=0x7f2d6400b810 comp rx=0 tx=0).stop 2023-12-14T18:37:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 shutdown_connections 2023-12-14T18:37:17.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f2d600a14d0 0x7f2d600a3980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d741051b0 0x7f2d7410b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d741009e0 0x7f2d74107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 --2- 172.21.15.31:0/4057000671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d74101390 0x7f2d741076c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.191+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 >> 172.21.15.31:0/4057000671 conn(0x7f2d74077520 msgr2=0x7f2d7406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.192+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 shutdown_connections 2023-12-14T18:37:17.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.192+0000 7f2d7b47a700 1 -- 172.21.15.31:0/4057000671 wait complete. 2023-12-14T18:37:17.211 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2023-12-14T18:37:17.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:37:17.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.522+0000 7f580c206700 1 -- 172.21.15.31:0/2904591449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58041133b0 msgr2=0x7f5804115790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.522+0000 7f580c206700 1 --2- 172.21.15.31:0/2904591449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58041133b0 0x7f5804115790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5800009990 tx=0x7f5800009ca0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.529+0000 7f580c206700 1 -- 172.21.15.31:0/2904591449 shutdown_connections 2023-12-14T18:37:17.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.529+0000 7f580c206700 1 --2- 172.21.15.31:0/2904591449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f58041133b0 0x7f5804115790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.529+0000 7f580c206700 1 --2- 172.21.15.31:0/2904591449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5804107af0 0x7f5804107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.529+0000 7f580c206700 1 --2- 172.21.15.31:0/2904591449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5804108400 0x7f5804112e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.529+0000 7f580c206700 1 -- 172.21.15.31:0/2904591449 >> 172.21.15.31:0/2904591449 conn(0x7f580406b7e0 msgr2=0x7f5804071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.536+0000 7f580c206700 1 -- 172.21.15.31:0/2904591449 shutdown_connections 2023-12-14T18:37:17.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.537+0000 7f580c206700 1 -- 172.21.15.31:0/2904591449 wait complete. 2023-12-14T18:37:17.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 Processor -- start 2023-12-14T18:37:17.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 -- start start 2023-12-14T18:37:17.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5804107af0 0x7f5804112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5804108400 0x7f580410fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58041133b0 0x7f580410ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041105a0 con 0x7f5804108400 2023-12-14T18:37:17.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041106e0 con 0x7f5804107af0 2023-12-14T18:37:17.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f580c206700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58041108a0 con 0x7f58041133b0 2023-12-14T18:37:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f5809fa2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5804107af0 0x7f5804112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f5809fa2700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5804107af0 0x7f5804112ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44522/0 (socket says 172.21.15.31:44522) 2023-12-14T18:37:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.538+0000 7f5809fa2700 1 -- 172.21.15.31:0/714046022 learned_addr learned my addr 172.21.15.31:0/714046022 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58041133b0 0x7f580410ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 -- 172.21.15.31:0/714046022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5804107af0 msgr2=0x7f5804112ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5804107af0 0x7f5804112ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 -- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5804108400 msgr2=0x7f580410fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5804108400 0x7f580410fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 -- 172.21.15.31:0/714046022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000095f0 con 0x7f58041133b0 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.539+0000 7f580a7a3700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58041133b0 0x7f580410ffd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5800003730 tx=0x7f580000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.540+0000 7f57faffd700 1 -- 172.21.15.31:0/714046022 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580001d070 con 0x7f58041133b0 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.541+0000 7f57faffd700 1 -- 172.21.15.31:0/714046022 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5800005030 con 0x7f58041133b0 2023-12-14T18:37:17.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.541+0000 7f57faffd700 1 -- 172.21.15.31:0/714046022 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800004bb0 con 0x7f58041133b0 2023-12-14T18:37:17.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.544+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580410a6e0 con 0x7f58041133b0 2023-12-14T18:37:17.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.544+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580410a550 con 0x7f58041133b0 2023-12-14T18:37:17.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.547+0000 7f57faffd700 1 -- 172.21.15.31:0/714046022 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f58000051a0 con 0x7f58041133b0 2023-12-14T18:37:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.549+0000 7f57faffd700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f57f00a14d0 0x7f57f00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.549+0000 7f57faffd700 1 -- 172.21.15.31:0/714046022 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f58000ce190 con 0x7f58041133b0 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.551+0000 7f5818394b80 1 Processor -- start 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 -- start start 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5809fa2700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f57f00a14d0 0x7f57f00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc788a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788ac90 0x55cbc7895610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 0x55cbc7897f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc77745e0 con 0x55cbc788a380 2023-12-14T18:37:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc77e3870 con 0x55cbc788ac90 2023-12-14T18:37:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.552+0000 7f5818394b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc77dcd00 con 0x55cbc7895b50 2023-12-14T18:37:17.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 0x55cbc7897f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 0x55cbc7897f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45558/0 (socket says 172.21.15.31:45558) 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 -- 172.21.15.31:0/312692113 learned_addr learned my addr 172.21.15.31:0/312692113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 -- 172.21.15.31:0/312692113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788ac90 msgr2=0x55cbc7895610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788ac90 0x55cbc7895610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 -- 172.21.15.31:0/312692113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 msgr2=0x55cbc788a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc788a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f580a7a3700 1 -- 172.21.15.31:0/312692113 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc765e3b0 con 0x55cbc7895b50 2023-12-14T18:37:17.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.553+0000 7f5809fa2700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f57f00a14d0 0x7f57f00a3980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009550 tx=0x7f57fc009cd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.554+0000 7f580a7a3700 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 0x55cbc7897f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58000a2ce0 tx=0x7f580009a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa703b97d32659c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.554+0000 7f57effff700 1 -- 172.21.15.31:0/312692113 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800005480 con 0x55cbc7895b50 2023-12-14T18:37:17.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.554+0000 7f57effff700 1 -- 172.21.15.31:0/312692113 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58000055e0 con 0x55cbc7895b50 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.554+0000 7f57effff700 1 -- 172.21.15.31:0/312692113 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580009a5a0 con 0x55cbc7895b50 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 -- 172.21.15.31:0/312692113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 msgr2=0x55cbc7897f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 0x55cbc7897f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58000a2ce0 tx=0x7f580009a8e0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 -- 172.21.15.31:0/312692113 shutdown_connections 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc7895b50 0x55cbc7897f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc788a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 --2- 172.21.15.31:0/312692113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788ac90 0x55cbc7895610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.555+0000 7f5818394b80 1 -- 172.21.15.31:0/312692113 >> 172.21.15.31:0/312692113 conn(0x55cbc7880ca0 msgr2=0x55cbc78867e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.556+0000 7f5818394b80 1 -- 172.21.15.31:0/312692113 shutdown_connections 2023-12-14T18:37:17.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.556+0000 7f5818394b80 1 -- 172.21.15.31:0/312692113 wait complete. 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 -- start start 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 Processor -- start 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc7895f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc7896460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc78969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc7778f90 con 0x55cbc7895b50 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc7778c30 con 0x55cbc788a380 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc76606a0 con 0x55cbc788ac90 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.557+0000 7f5818394b80 10 client.0 _ll_register_callbacks cb 0x55cbc779c910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:17.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.558+0000 7f580a7a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc78969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.558+0000 7f580a7a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc78969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53042/0 (socket says 172.21.15.31:53042) 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.558+0000 7f580a7a3700 1 -- 172.21.15.31:0/2652480578 learned_addr learned my addr 172.21.15.31:0/2652480578 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.559+0000 7f580a7a3700 1 -- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 msgr2=0x55cbc7896460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.559+0000 7f580a7a3700 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc7896460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.559+0000 7f580a7a3700 1 -- 172.21.15.31:0/2652480578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 msgr2=0x55cbc7895f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.559+0000 7f580a7a3700 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc7895f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.559+0000 7f580a7a3700 1 -- 172.21.15.31:0/2652480578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580410a550 con 0x55cbc7895b50 2023-12-14T18:37:17.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.559+0000 7f580a7a3700 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc78969a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f58000978c0 tx=0x7f5800009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc37b10407b1991f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:17.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.560+0000 7f57ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:17.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.560+0000 7f57ed7fa700 1 -- 172.21.15.31:0/2652480578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580009a340 con 0x55cbc7895b50 2023-12-14T18:37:17.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.560+0000 7f57ed7fa700 1 -- 172.21.15.31:0/2652480578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f580009a4a0 con 0x55cbc7895b50 2023-12-14T18:37:17.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.560+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbc787e2b0 con 0x55cbc7895b50 2023-12-14T18:37:17.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.563+0000 7f57ed7fa700 1 -- 172.21.15.31:0/2652480578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58000cb250 con 0x55cbc7895b50 2023-12-14T18:37:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.569+0000 7f5818394b80 10 client.17586 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:17.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.569+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cbc77a3170 con 0x55cbc7895b50 2023-12-14T18:37:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.570+0000 7f57ed7fa700 1 -- 172.21.15.31:0/2652480578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58000be070 con 0x55cbc7895b50 2023-12-14T18:37:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.570+0000 7f57ed7fa700 1 -- 172.21.15.31:0/2652480578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5800096c40 con 0x55cbc7895b50 2023-12-14T18:37:17.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.571+0000 7f5818394b80 10 client.17586 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.571+0000 7f5818394b80 10 client.17586 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:37:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.571+0000 7f5818394b80 20 client.17586 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.571+0000 7f5818394b80 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78a4ca0 0x55cbc78a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.571+0000 7f5818394b80 4 client.17586 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:17.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.571+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbc77d5ce0 con 0x55cbc78a4ca0 2023-12-14T18:37:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.573+0000 7f5809fa2700 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78a4ca0 0x55cbc78a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.573+0000 7f5809fa2700 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78a4ca0 0x55cbc78a70b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b9b25ee3f43150 server_cookie=e5fdcf6fdc87c2c7 in_seq=0 out_seq=0 2023-12-14T18:37:17.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.574+0000 7f57ed7fa700 10 client.17586 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:17.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.576+0000 7f57ed7fa700 1 -- 172.21.15.31:0/2652480578 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cbc77d5ce0 con 0x55cbc78a4ca0 2023-12-14T18:37:17.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.576+0000 7f57ed7fa700 10 client.17586 handle_command_reply: tid=0 2023-12-14T18:37:17.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.576+0000 7f5818394b80 1 client.17586 shutdown 2023-12-14T18:37:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 msgr2=0x55cbc78969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc78969a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f58000978c0 tx=0x7f5800009310 comp rx=0 tx=0).stop 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 shutdown_connections 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78a4ca0 0x55cbc78a70b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc78969a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc7895f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.582+0000 7f5818394b80 1 --2- 172.21.15.31:0/2652480578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc7896460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.583+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 >> 172.21.15.31:0/2652480578 conn(0x55cbc7880ca0 msgr2=0x55cbc78863d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.583+0000 7f57ed7fa700 0 client.17586 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:17.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.585+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 shutdown_connections 2023-12-14T18:37:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.586+0000 7f5818394b80 1 -- 172.21.15.31:0/2652480578 wait complete. 2023-12-14T18:37:17.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.586+0000 7f5818394b80 20 client.17586 trim_cache size 0 max 16384 2023-12-14T18:37:17.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.607+0000 7f5818394b80 1 Processor -- start 2023-12-14T18:37:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 -- start start 2023-12-14T18:37:17.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc78a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc788f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc7891f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc7890bb0 con 0x55cbc7895b50 2023-12-14T18:37:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc78908c0 con 0x55cbc788a380 2023-12-14T18:37:17.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.608+0000 7f5818394b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc7894210 con 0x55cbc788ac90 2023-12-14T18:37:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.611+0000 7f580a7a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc7891f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.611+0000 7f580a7a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc7891f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53044/0 (socket says 172.21.15.31:53044) 2023-12-14T18:37:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f580a7a3700 1 -- 172.21.15.31:0/953696246 learned_addr learned my addr 172.21.15.31:0/953696246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:17.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f5809fa2700 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc78a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f580a7a3700 1 -- 172.21.15.31:0/953696246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 msgr2=0x55cbc788f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f580a7a3700 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc788f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f580a7a3700 1 -- 172.21.15.31:0/953696246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 msgr2=0x55cbc78a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f580a7a3700 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc78a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.612+0000 7f580a7a3700 1 -- 172.21.15.31:0/953696246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc78957d0 con 0x55cbc7895b50 2023-12-14T18:37:17.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.613+0000 7f580a7a3700 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc7891f10 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f580009ade0 tx=0x7f5800005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22ace6e88058f19 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:17.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.630+0000 7f57f8ff9700 1 -- 172.21.15.31:0/953696246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000cbdf0 con 0x55cbc7895b50 2023-12-14T18:37:17.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.630+0000 7f57f8ff9700 1 -- 172.21.15.31:0/953696246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58000be040 con 0x55cbc7895b50 2023-12-14T18:37:17.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.641+0000 7f57f8ff9700 1 -- 172.21.15.31:0/953696246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58000a2d80 con 0x55cbc7895b50 2023-12-14T18:37:17.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.645+0000 7f5818394b80 1 -- 172.21.15.31:0/953696246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 msgr2=0x55cbc7891f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.645+0000 7f5818394b80 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc7891f10 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f580009ade0 tx=0x7f5800005540 comp rx=0 tx=0).stop 2023-12-14T18:37:17.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.646+0000 7f5818394b80 1 -- 172.21.15.31:0/953696246 shutdown_connections 2023-12-14T18:37:17.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.646+0000 7f5818394b80 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc7895b50 0x55cbc7891f10 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.646+0000 7f5818394b80 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc788a380 0x55cbc78a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.646+0000 7f5818394b80 1 --2- 172.21.15.31:0/953696246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc788f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.646+0000 7f5818394b80 1 -- 172.21.15.31:0/953696246 >> 172.21.15.31:0/953696246 conn(0x55cbc7880ca0 msgr2=0x55cbc78a14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.648+0000 7f5818394b80 1 -- 172.21.15.31:0/953696246 shutdown_connections 2023-12-14T18:37:17.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.654+0000 7f5818394b80 1 -- 172.21.15.31:0/953696246 wait complete. 2023-12-14T18:37:17.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.654+0000 7f5818394b80 1 -- start start 2023-12-14T18:37:17.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.654+0000 7f5818394b80 1 Processor -- start 2023-12-14T18:37:17.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc789a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc789aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc7895b50 0x55cbc7898ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc78995b0 con 0x55cbc788a380 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc7899720 con 0x55cbc7895b50 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc78998e0 con 0x55cbc788ac90 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5818394b80 10 client.0 _ll_register_callbacks cb 0x7f580009a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5809fa2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc789a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5809fa2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc789a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53052/0 (socket says 172.21.15.31:53052) 2023-12-14T18:37:17.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.656+0000 7f5809fa2700 1 -- 172.21.15.31:0/3653710944 learned_addr learned my addr 172.21.15.31:0/3653710944 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:17.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f5809fa2700 1 -- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 msgr2=0x55cbc789aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f5809fa2700 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc789aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f5809fa2700 1 -- 172.21.15.31:0/3653710944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc7895b50 msgr2=0x55cbc7898ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f5809fa2700 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc7895b50 0x55cbc7898ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f5809fa2700 1 -- 172.21.15.31:0/3653710944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbc77a3170 con 0x55cbc788a380 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f5809fa2700 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc789a990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55cbc78a7980 tx=0x7f57fc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263b2003ce79b606 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f57ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.657+0000 7f57ed7fa700 1 -- 172.21.15.31:0/3653710944 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc01e070 con 0x55cbc788a380 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f57ed7fa700 1 -- 172.21.15.31:0/3653710944 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f57fc0228e0 con 0x55cbc788a380 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58000055e0 con 0x55cbc788a380 2023-12-14T18:37:17.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f57ed7fa700 1 -- 172.21.15.31:0/3653710944 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57fc022780 con 0x55cbc788a380 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f5818394b80 10 client.17592 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cbc760ab60 con 0x55cbc788a380 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f57ed7fa700 1 -- 172.21.15.31:0/3653710944 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58000055e0 con 0x55cbc788a380 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f57ed7fa700 1 -- 172.21.15.31:0/3653710944 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57fc0185c0 con 0x55cbc788a380 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f5818394b80 10 client.17592 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f5818394b80 10 client.17592 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.658+0000 7f5818394b80 20 client.17592 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.659+0000 7f5818394b80 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78f9e20 0x55cbc78fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.659+0000 7f5818394b80 4 client.17592 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:37:17.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.659+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cbc77d5a00 con 0x55cbc78f9e20 2023-12-14T18:37:17.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.659+0000 7f58097a1700 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78f9e20 0x55cbc78fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:17.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.659+0000 7f57ed7fa700 10 client.17592 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:17.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.659+0000 7f58097a1700 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78f9e20 0x55cbc78fc230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b502b2e434556a6a server_cookie=95f2d9331d5a4718 in_seq=0 out_seq=0 2023-12-14T18:37:17.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.660+0000 7f57ed7fa700 1 -- 172.21.15.31:0/3653710944 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cbc77d5a00 con 0x55cbc78f9e20 2023-12-14T18:37:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.660+0000 7f57ed7fa700 10 client.17592 handle_command_reply: tid=0 2023-12-14T18:37:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.660+0000 7f5818394b80 1 client.17592 shutdown 2023-12-14T18:37:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 msgr2=0x55cbc789a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc789a990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55cbc78a7980 tx=0x7f57fc004b40 comp rx=0 tx=0).stop 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 shutdown_connections 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55cbc78f9e20 0x55cbc78fc230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55cbc7895b50 0x55cbc7898ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55cbc788a380 0x55cbc789a990 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 --2- 172.21.15.31:0/3653710944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cbc788ac90 0x55cbc789aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 >> 172.21.15.31:0/3653710944 conn(0x55cbc7880ca0 msgr2=0x55cbc7895f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f57ed7fa700 0 client.17592 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 shutdown_connections 2023-12-14T18:37:17.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 1 -- 172.21.15.31:0/3653710944 wait complete. 2023-12-14T18:37:17.669 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:37:17.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.662+0000 7f5818394b80 20 client.17592 trim_cache size 0 max 16384 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f57f00a14d0 msgr2=0x7f57f00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f57f00a14d0 0x7f57f00a3980 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009550 tx=0x7f57fc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58041133b0 msgr2=0x7f580410ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58041133b0 0x7f580410ffd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5800003730 tx=0x7f580000f690 comp rx=0 tx=0).stop 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 shutdown_connections 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f57f00a14d0 0x7f57f00a3980 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009550 tx=0x7f57fc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58041133b0 0x7f580410ffd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5804107af0 0x7f5804112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 --2- 172.21.15.31:0/714046022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5804108400 0x7f580410fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:17.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.665+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 >> 172.21.15.31:0/714046022 conn(0x7f580406b7e0 msgr2=0x7f580406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.669+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 shutdown_connections 2023-12-14T18:37:17.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.670+0000 7f580c206700 1 -- 172.21.15.31:0/714046022 wait complete. 2023-12-14T18:37:17.703 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:17 smithi031 ceph-mon[42085]: pgmap v294: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 544 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:37:17.703 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:17 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4057000671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:17.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:37:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:17 smithi176 ceph-mon[43506]: pgmap v294: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 544 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:37:17.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:17 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4057000671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:17.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.993+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1041869594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 msgr2=0x7fd5ec115890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:17.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:17.993+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1041869594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 0x7fd5ec115890 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80098f0 tx=0x7fd5e8009c00 comp rx=0 tx=0).stop 2023-12-14T18:37:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.001+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1041869594 shutdown_connections 2023-12-14T18:37:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.001+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1041869594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 0x7fd5ec115890 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.001+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1041869594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.001+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1041869594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec108b00 0x7fd5ec108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.001+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1041869594 >> 172.21.15.31:0/1041869594 conn(0x7fd5ec06d610 msgr2=0x7fd5ec06fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.003+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1041869594 shutdown_connections 2023-12-14T18:37:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.003+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1041869594 wait complete. 2023-12-14T18:37:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.004+0000 7fd5f43d3700 1 Processor -- start 2023-12-14T18:37:18.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.004+0000 7fd5f43d3700 1 -- start start 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f43d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ec108b00 0x7fd5ec079ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f43d3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 0x7fd5ec07a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f43d3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f43d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0755c0 con 0x7fd5ec108b00 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f43d3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec075730 con 0x7fd5ec1094b0 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f43d3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec0758f0 con 0x7fd5ec10eab0 2023-12-14T18:37:18.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f2970700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f2970700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45602/0 (socket says 172.21.15.31:45602) 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f2970700 1 -- 172.21.15.31:0/1082274077 learned_addr learned my addr 172.21.15.31:0/1082274077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.005+0000 7fd5f196e700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 0x7fd5ec07a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f2970700 1 -- 172.21.15.31:0/1082274077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 msgr2=0x7fd5ec07a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f2970700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 0x7fd5ec07a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f2970700 1 -- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ec108b00 msgr2=0x7fd5ec079ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f2970700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ec108b00 0x7fd5ec079ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f2970700 1 -- 172.21.15.31:0/1082274077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8009520 con 0x7fd5ec10eab0 2023-12-14T18:37:18.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f2970700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec074df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005950 tx=0x7fd5e800fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5e37fe700 1 -- 172.21.15.31:0/1082274077 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e801c070 con 0x7fd5ec10eab0 2023-12-14T18:37:18.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5e37fe700 1 -- 172.21.15.31:0/1082274077 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e800bb50 con 0x7fd5ec10eab0 2023-12-14T18:37:18.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5e37fe700 1 -- 172.21.15.31:0/1082274077 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8003970 con 0x7fd5ec10eab0 2023-12-14T18:37:18.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec073c30 con 0x7fd5ec10eab0 2023-12-14T18:37:18.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.006+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec073aa0 con 0x7fd5ec10eab0 2023-12-14T18:37:18.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.012+0000 7fd600561b80 1 Processor -- start 2023-12-14T18:37:18.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.012+0000 7fd600561b80 1 -- start start 2023-12-14T18:37:18.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd600561b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20380 0x5593c0b20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd600561b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b2b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd600561b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd600561b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0a0a5e0 con 0x5593c0b20380 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd600561b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0a79870 con 0x5593c0b2bb50 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd600561b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0a72d00 con 0x5593c0b20c90 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd5f216f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b2b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd5f216f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b2b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45614/0 (socket says 172.21.15.31:45614) 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd5f216f700 1 -- 172.21.15.31:0/2194135271 learned_addr learned my addr 172.21.15.31:0/2194135271 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd5f196e700 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.013+0000 7fd5f196e700 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20380 0x5593c0b20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5f216f700 1 -- 172.21.15.31:0/2194135271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 msgr2=0x5593c0b2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5f216f700 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5f216f700 1 -- 172.21.15.31:0/2194135271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20380 msgr2=0x5593c0b20750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5f216f700 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20380 0x5593c0b20750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5f216f700 1 -- 172.21.15.31:0/2194135271 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593c08f43b0 con 0x5593c0b20c90 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5f216f700 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b2b610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009b50 tx=0x7fd5dc009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f287d5709d800eee server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5e0dac700 1 -- 172.21.15.31:0/2194135271 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc004a30 con 0x5593c0b20c90 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5e0dac700 1 -- 172.21.15.31:0/2194135271 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc00ed00 con 0x5593c0b20c90 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.014+0000 7fd5e0dac700 1 -- 172.21.15.31:0/2194135271 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc005670 con 0x5593c0b20c90 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.015+0000 7fd5e37fe700 1 -- 172.21.15.31:0/1082274077 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd5e8020030 con 0x7fd5ec10eab0 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.015+0000 7fd5e37fe700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd5d80a13c0 0x7fd5d80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.015+0000 7fd600561b80 1 -- 172.21.15.31:0/2194135271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 msgr2=0x5593c0b2b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.015+0000 7fd600561b80 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b2b610 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc009b50 tx=0x7fd5dc009e60 comp rx=0 tx=0).stop 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd5e37fe700 1 -- 172.21.15.31:0/1082274077 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fd5e80ce0f0 con 0x7fd5ec10eab0 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd5f196e700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd5d80a13c0 0x7fd5d80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd600561b80 1 -- 172.21.15.31:0/2194135271 shutdown_connections 2023-12-14T18:37:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd600561b80 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd600561b80 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20380 0x5593c0b20750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd600561b80 1 --2- 172.21.15.31:0/2194135271 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b2b610 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd600561b80 1 -- 172.21.15.31:0/2194135271 >> 172.21.15.31:0/2194135271 conn(0x5593c0b16ca0 msgr2=0x5593c0b1c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.016+0000 7fd600561b80 1 -- 172.21.15.31:0/2194135271 shutdown_connections 2023-12-14T18:37:18.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.017+0000 7fd5f196e700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd5d80a13c0 0x7fd5d80a3870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009550 tx=0x7fd5e400b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.017+0000 7fd600561b80 1 -- 172.21.15.31:0/2194135271 wait complete. 2023-12-14T18:37:18.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.018+0000 7fd600561b80 1 -- start start 2023-12-14T18:37:18.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.018+0000 7fd600561b80 1 Processor -- start 2023-12-14T18:37:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 0x5593c0b2b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b28710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b2bb50 0x5593c0b28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0a0ef90 con 0x5593c0b2bb50 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0a0ec30 con 0x5593c0b20380 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c08f66a0 con 0x5593c0b20c90 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd600561b80 10 client.0 _ll_register_callbacks cb 0x5593c0a32910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 0x5593c0b2b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 0x5593c0b2b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44574/0 (socket says 172.21.15.31:44574) 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 -- 172.21.15.31:0/202302185 learned_addr learned my addr 172.21.15.31:0/202302185 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f196e700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b28710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f2970700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b2bb50 0x5593c0b28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 -- 172.21.15.31:0/202302185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 msgr2=0x5593c0b28710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b28710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 -- 172.21.15.31:0/202302185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b2bb50 msgr2=0x5593c0b28c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b2bb50 0x5593c0b28c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f216f700 1 -- 172.21.15.31:0/202302185 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5dc009800 con 0x5593c0b20380 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.019+0000 7fd5f2970700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b2bb50 0x5593c0b28c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.020+0000 7fd5f216f700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 0x5593c0b2b460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006010 tx=0x7fd5dc004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7a1b20ab2ecffd server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.020+0000 7fd5d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.020+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/202302185 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc005470 con 0x5593c0b20380 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.020+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/202302185 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fd5dc014070 con 0x5593c0b20380 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593c0b142b0 con 0x5593c0b20380 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/202302185 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc005da0 con 0x5593c0b20380 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 10 client.27305 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5593c0a39170 con 0x5593c0b20380 2023-12-14T18:37:18.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/202302185 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5593c0b142b0 con 0x5593c0b20380 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/202302185 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd5dc010430 con 0x5593c0b20380 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 10 client.27305 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 10 client.27305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 20 client.27305 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0b39340 0x5593c0b3b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 4 client.27305 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.021+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593c0a6bce0 con 0x5593c0b39340 2023-12-14T18:37:18.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.022+0000 7fd5f196e700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0b39340 0x5593c0b3b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.022+0000 7fd5f196e700 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0b39340 0x5593c0b3b7b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e7af8793cb36722 server_cookie=ad9d63f7b9a94b51 in_seq=0 out_seq=0 2023-12-14T18:37:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.022+0000 7fd5d5ffb700 10 client.27305 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.023+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/202302185 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593c0a6bce0 con 0x5593c0b39340 2023-12-14T18:37:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.023+0000 7fd5d5ffb700 10 client.27305 handle_command_reply: tid=0 2023-12-14T18:37:18.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.024+0000 7fd600561b80 1 client.27305 shutdown 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.024+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 msgr2=0x5593c0b2b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.024+0000 7fd600561b80 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 0x5593c0b2b460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc006010 tx=0x7fd5dc004b10 comp rx=0 tx=0).stop 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 shutdown_connections 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd600561b80 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b2bb50 0x5593c0b28c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd600561b80 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b20380 0x5593c0b2b460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd600561b80 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0b39340 0x5593c0b3b7b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd600561b80 1 --2- 172.21.15.31:0/202302185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20c90 0x5593c0b28710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 >> 172.21.15.31:0/202302185 conn(0x5593c0b16ca0 msgr2=0x5593c0b1c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.025+0000 7fd5d5ffb700 0 client.27305 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.032+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 shutdown_connections 2023-12-14T18:37:18.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.033+0000 7fd600561b80 1 -- 172.21.15.31:0/202302185 wait complete. 2023-12-14T18:37:18.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.035+0000 7fd600561b80 20 client.27305 trim_cache size 0 max 16384 2023-12-14T18:37:18.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 Processor -- start 2023-12-14T18:37:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 -- start start 2023-12-14T18:37:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b27a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b3b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0b24620 con 0x5593c0b20c90 2023-12-14T18:37:18.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0b23370 con 0x5593c0b2bb50 2023-12-14T18:37:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.056+0000 7fd600561b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0b29a60 con 0x5593c0b20380 2023-12-14T18:37:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44580/0 (socket says 172.21.15.31:44580) 2023-12-14T18:37:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 -- 172.21.15.31:0/213051238 learned_addr learned my addr 172.21.15.31:0/213051238 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f196e700 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b3b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f216f700 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b27a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 -- 172.21.15.31:0/213051238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 msgr2=0x5593c0b27a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b27a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 -- 172.21.15.31:0/213051238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 msgr2=0x5593c0b3b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b3b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 -- 172.21.15.31:0/213051238 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593c0a38ec0 con 0x5593c0b2bb50 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f216f700 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:18.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.057+0000 7fd5f2970700 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2af20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80972b0 tx=0x7fd5e8005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763ab693386802a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.059+0000 7fd5e15ad700 1 -- 172.21.15.31:0/213051238 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80a3170 con 0x5593c0b2bb50 2023-12-14T18:37:18.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.059+0000 7fd5e15ad700 1 -- 172.21.15.31:0/213051238 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e80974c0 con 0x5593c0b2bb50 2023-12-14T18:37:18.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.059+0000 7fd5e15ad700 1 -- 172.21.15.31:0/213051238 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e809b010 con 0x5593c0b2bb50 2023-12-14T18:37:18.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.060+0000 7fd600561b80 1 -- 172.21.15.31:0/213051238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 msgr2=0x5593c0b2af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.060+0000 7fd600561b80 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2af20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80972b0 tx=0x7fd5e8005b40 comp rx=0 tx=0).stop 2023-12-14T18:37:18.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.067+0000 7fd600561b80 1 -- 172.21.15.31:0/213051238 shutdown_connections 2023-12-14T18:37:18.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.067+0000 7fd600561b80 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b2af20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.067+0000 7fd600561b80 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.067+0000 7fd600561b80 1 --2- 172.21.15.31:0/213051238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b3b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.067+0000 7fd600561b80 1 -- 172.21.15.31:0/213051238 >> 172.21.15.31:0/213051238 conn(0x5593c0b16ca0 msgr2=0x5593c0b1af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.074+0000 7fd600561b80 1 -- 172.21.15.31:0/213051238 shutdown_connections 2023-12-14T18:37:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.075+0000 7fd600561b80 1 -- 172.21.15.31:0/213051238 wait complete. 2023-12-14T18:37:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.076+0000 7fd600561b80 1 -- start start 2023-12-14T18:37:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.076+0000 7fd600561b80 1 Processor -- start 2023-12-14T18:37:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.076+0000 7fd600561b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b258a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd600561b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd600561b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b26320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd600561b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0bb0210 con 0x5593c0b20c90 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd600561b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0bb0380 con 0x5593c0b2bb50 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd600561b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593c0bb0540 con 0x5593c0b20380 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd600561b80 10 client.0 _ll_register_callbacks cb 0x5593c0a32910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b25de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53108/0 (socket says 172.21.15.31:53108) 2023-12-14T18:37:18.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 -- 172.21.15.31:0/144110150 learned_addr learned my addr 172.21.15.31:0/144110150 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 -- 172.21.15.31:0/144110150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 msgr2=0x5593c0b258a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b258a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 -- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 msgr2=0x5593c0b26320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b26320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.077+0000 7fd5f196e700 1 -- 172.21.15.31:0/144110150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec073aa0 con 0x5593c0b20c90 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd5f196e700 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b25de0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec076570 tx=0x7fd5e4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14041109de209de server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd5d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:18.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/144110150 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400e040 con 0x5593c0b20c90 2023-12-14T18:37:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/144110150 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fd5e4008970 con 0x5593c0b20c90 2023-12-14T18:37:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5dc00ed00 con 0x5593c0b20c90 2023-12-14T18:37:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/144110150 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e4008b30 con 0x5593c0b20c90 2023-12-14T18:37:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd600561b80 10 client.17607 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.078+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5593c0a1f9a0 con 0x5593c0b20c90 2023-12-14T18:37:18.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.079+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/144110150 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc00ed00 con 0x5593c0b20c90 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/144110150 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd5e4027070 con 0x5593c0b20c90 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd600561b80 10 client.17607 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd600561b80 10 client.17607 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd600561b80 20 client.17607 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd600561b80 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0bb1c50 0x5593c0bb4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd600561b80 4 client.17607 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:37:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.080+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593c0a6ba00 con 0x5593c0bb1c50 2023-12-14T18:37:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.084+0000 7fd5f216f700 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0bb1c50 0x5593c0bb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.084+0000 7fd5d5ffb700 10 client.17607 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.084+0000 7fd5f216f700 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0bb1c50 0x5593c0bb4100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0b1df1df89a4fa4 server_cookie=7a081a796be5990a in_seq=0 out_seq=0 2023-12-14T18:37:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd5d5ffb700 1 -- 172.21.15.31:0/144110150 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593c0a6ba00 con 0x5593c0bb1c50 2023-12-14T18:37:18.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd5d5ffb700 10 client.17607 handle_command_reply: tid=0 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 client.17607 shutdown 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 msgr2=0x5593c0b25de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b25de0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec076570 tx=0x7fd5e4004b40 comp rx=0 tx=0).stop 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 shutdown_connections 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x5593c0b2bb50 0x5593c0b26320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x5593c0bb1c50 0x5593c0bb4100 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593c0b20380 0x5593c0b258a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 --2- 172.21.15.31:0/144110150 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x5593c0b20c90 0x5593c0b25de0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd5d5ffb700 0 client.17607 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:18.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.085+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 >> 172.21.15.31:0/144110150 conn(0x5593c0b16ca0 msgr2=0x5593c0b1c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:17 smithi204 ceph-mon[43866]: pgmap v294: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 544 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:37:18.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:17 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4057000671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.089+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 shutdown_connections 2023-12-14T18:37:18.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.089+0000 7fd600561b80 1 -- 172.21.15.31:0/144110150 wait complete. 2023-12-14T18:37:18.093 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:37:18.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.090+0000 7fd600561b80 20 client.17607 trim_cache size 0 max 16384 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.094+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd5d80a13c0 msgr2=0x7fd5d80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.094+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd5d80a13c0 0x7fd5d80a3870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4009550 tx=0x7fd5e400b040 comp rx=0 tx=0).stop 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.094+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 msgr2=0x7fd5ec074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.094+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec074df0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005950 tx=0x7fd5e800fa70 comp rx=0 tx=0).stop 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.096+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 shutdown_connections 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.096+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd5d80a13c0 0x7fd5d80a3870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.096+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec10eab0 0x7fd5ec074df0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.096+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5ec108b00 0x7fd5ec079ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.096+0000 7fd5f43d3700 1 --2- 172.21.15.31:0/1082274077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd5ec1094b0 0x7fd5ec07a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.096+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 >> 172.21.15.31:0/1082274077 conn(0x7fd5ec06d610 msgr2=0x7fd5ec06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.098+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 shutdown_connections 2023-12-14T18:37:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.098+0000 7fd5f43d3700 1 -- 172.21.15.31:0/1082274077 wait complete. 2023-12-14T18:37:18.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:37:18.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 -- 172.21.15.31:0/1465396036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107af0 msgr2=0x7f271010eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 --2- 172.21.15.31:0/1465396036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107af0 0x7f271010eb50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f270001a9a0 tx=0x7f270001acb0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 -- 172.21.15.31:0/1465396036 shutdown_connections 2023-12-14T18:37:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 --2- 172.21.15.31:0/1465396036 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710119a40 0x7f271011be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 --2- 172.21.15.31:0/1465396036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107af0 0x7f271010eb50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 --2- 172.21.15.31:0/1465396036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2710107ec0 0x7f2710108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 -- 172.21.15.31:0/1465396036 >> 172.21.15.31:0/1465396036 conn(0x7f271006b7e0 msgr2=0x7f2710071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 -- 172.21.15.31:0/1465396036 shutdown_connections 2023-12-14T18:37:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.373+0000 7f2716265700 1 -- 172.21.15.31:0/1465396036 wait complete. 2023-12-14T18:37:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.374+0000 7f2716265700 1 Processor -- start 2023-12-14T18:37:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.374+0000 7f2716265700 1 -- start start 2023-12-14T18:37:18.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f2716265700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710107af0 0x7f2710114ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f2716265700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107ec0 0x7f2710115430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f2716265700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2710119a40 0x7f2710073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f2716265700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710074330 con 0x7f2710119a40 2023-12-14T18:37:18.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f2716265700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100744a0 con 0x7f2710107af0 2023-12-14T18:37:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f2716265700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710074660 con 0x7f2710107ec0 2023-12-14T18:37:18.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f270ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710107af0 0x7f2710114ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f270ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710107af0 0x7f2710114ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44592/0 (socket says 172.21.15.31:44592) 2023-12-14T18:37:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f270ffff700 1 -- 172.21.15.31:0/744382265 learned_addr learned my addr 172.21.15.31:0/744382265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.375+0000 7f270f7fe700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107ec0 0x7f2710115430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.376+0000 7f270f7fe700 1 -- 172.21.15.31:0/744382265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710107af0 msgr2=0x7f2710114ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.376+0000 7f270f7fe700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710107af0 0x7f2710114ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.376+0000 7f270f7fe700 1 -- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2710119a40 msgr2=0x7f2710073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.376+0000 7f270f7fe700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2710119a40 0x7f2710073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.376+0000 7f270f7fe700 1 -- 172.21.15.31:0/744382265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270001a600 con 0x7f2710107ec0 2023-12-14T18:37:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.376+0000 7f270f7fe700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107ec0 0x7f2710115430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f270800bd70 tx=0x7f270800bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.381+0000 7f270d7fa700 1 -- 172.21.15.31:0/744382265 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270800cc90 con 0x7f2710107ec0 2023-12-14T18:37:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.381+0000 7f270d7fa700 1 -- 172.21.15.31:0/744382265 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f270800cdf0 con 0x7f2710107ec0 2023-12-14T18:37:18.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.381+0000 7f270d7fa700 1 -- 172.21.15.31:0/744382265 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708012760 con 0x7f2710107ec0 2023-12-14T18:37:18.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.381+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710111ab0 con 0x7f2710107ec0 2023-12-14T18:37:18.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.381+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2710111920 con 0x7f2710107ec0 2023-12-14T18:37:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.387+0000 7f27223f3b80 1 Processor -- start 2023-12-14T18:37:18.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 -- start start 2023-12-14T18:37:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x557234332750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 0x55723433d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x55723433ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723421c5e0 con 0x557234332c90 2023-12-14T18:37:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723428b870 con 0x55723433db50 2023-12-14T18:37:18.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.389+0000 7f27223f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557234284d00 con 0x557234332380 2023-12-14T18:37:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.390+0000 7f270d7fa700 1 -- 172.21.15.31:0/744382265 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f27080128c0 con 0x7f2710107ec0 2023-12-14T18:37:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.391+0000 7f270d7fa700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f26f80a1490 0x7f26f80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.391+0000 7f270d7fa700 1 -- 172.21.15.31:0/744382265 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2708006e40 con 0x7f2710107ec0 2023-12-14T18:37:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.392+0000 7f270ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 0x55723433d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.392+0000 7f270ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 0x55723433d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53124/0 (socket says 172.21.15.31:53124) 2023-12-14T18:37:18.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.392+0000 7f270ffff700 1 -- 172.21.15.31:0/892569711 learned_addr learned my addr 172.21.15.31:0/892569711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f270ffff700 1 -- 172.21.15.31:0/892569711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 msgr2=0x557234332750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f2714802700 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x557234332750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f270ffff700 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x557234332750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f270ffff700 1 -- 172.21.15.31:0/892569711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 msgr2=0x55723433ff30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f270ffff700 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x55723433ff30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f270ffff700 1 -- 172.21.15.31:0/892569711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572341063b0 con 0x557234332c90 2023-12-14T18:37:18.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f2714802700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f26f80a1490 0x7f26f80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f270ffff700 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 0x55723433d610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2700003760 tx=0x7f270001c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1fe2497004bffa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.393+0000 7f2714802700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f26f80a1490 0x7f26f80a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f270400a860 tx=0x7f27040048a0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.395+0000 7f26f67fc700 1 -- 172.21.15.31:0/892569711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700071050 con 0x557234332c90 2023-12-14T18:37:18.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.399+0000 7f26f67fc700 1 -- 172.21.15.31:0/892569711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2700072040 con 0x557234332c90 2023-12-14T18:37:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.399+0000 7f26f67fc700 1 -- 172.21.15.31:0/892569711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700004340 con 0x557234332c90 2023-12-14T18:37:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 -- 172.21.15.31:0/892569711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 msgr2=0x55723433d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 0x55723433d610 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2700003760 tx=0x7f270001c590 comp rx=0 tx=0).stop 2023-12-14T18:37:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 -- 172.21.15.31:0/892569711 shutdown_connections 2023-12-14T18:37:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x55723433ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x557234332750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 --2- 172.21.15.31:0/892569711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332c90 0x55723433d610 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.400+0000 7f27223f3b80 1 -- 172.21.15.31:0/892569711 >> 172.21.15.31:0/892569711 conn(0x557234328ca0 msgr2=0x55723432e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.403+0000 7f27223f3b80 1 -- 172.21.15.31:0/892569711 shutdown_connections 2023-12-14T18:37:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.405+0000 7f27223f3b80 1 -- 172.21.15.31:0/892569711 wait complete. 2023-12-14T18:37:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.405+0000 7f27223f3b80 1 -- start start 2023-12-14T18:37:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 Processor -- start 2023-12-14T18:37:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x55723433d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x55723433a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 0x55723433add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557234220f90 con 0x557234332380 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557234220c30 con 0x557234332c90 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572341086a0 con 0x55723433db50 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f27223f3b80 10 client.0 _ll_register_callbacks cb 0x557234244910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:18.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.406+0000 7f2714802700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 0x55723433add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 0x55723433add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45674/0 (socket says 172.21.15.31:45674) 2023-12-14T18:37:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 -- 172.21.15.31:0/560997456 learned_addr learned my addr 172.21.15.31:0/560997456 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 -- 172.21.15.31:0/560997456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 msgr2=0x55723433a810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x55723433a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 -- 172.21.15.31:0/560997456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 msgr2=0x55723433d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x55723433d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 -- 172.21.15.31:0/560997456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271011c6b0 con 0x55723433db50 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.407+0000 7f2714802700 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 0x55723433add0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2704011f10 tx=0x7f2704004910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759f9c822d3bc61f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.408+0000 7f26ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:37:18.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.408+0000 7f26ef7fe700 1 -- 172.21.15.31:0/560997456 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270401d450 con 0x55723433db50 2023-12-14T18:37:18.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.408+0000 7f26ef7fe700 1 -- 172.21.15.31:0/560997456 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f2704016030 con 0x55723433db50 2023-12-14T18:37:18.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.408+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572343262b0 con 0x55723433db50 2023-12-14T18:37:18.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.415+0000 7f26ef7fe700 1 -- 172.21.15.31:0/560997456 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f270401d5b0 con 0x55723433db50 2023-12-14T18:37:18.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.416+0000 7f27223f3b80 10 client.27388 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:18.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.416+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55723424b170 con 0x55723433db50 2023-12-14T18:37:18.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.416+0000 7f26ef7fe700 1 -- 172.21.15.31:0/560997456 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572343262b0 con 0x55723433db50 2023-12-14T18:37:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f26ef7fe700 1 -- 172.21.15.31:0/560997456 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f270401c820 con 0x55723433db50 2023-12-14T18:37:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f27223f3b80 10 client.27388 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:18.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f27223f3b80 10 client.27388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f27223f3b80 20 client.27388 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f27223f3b80 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55723434d5e0 0x55723434f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f27223f3b80 4 client.27388 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:18.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.417+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55723427dce0 con 0x55723434d5e0 2023-12-14T18:37:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.422+0000 7f270ffff700 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55723434d5e0 0x55723434f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.424+0000 7f270ffff700 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55723434d5e0 0x55723434f9f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=765d41c6ef9f9a2e server_cookie=4ed85411fd8edaad in_seq=0 out_seq=0 2023-12-14T18:37:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.427+0000 7f26ef7fe700 10 client.27388 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.428+0000 7f26ef7fe700 1 -- 172.21.15.31:0/560997456 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55723427dce0 con 0x55723434d5e0 2023-12-14T18:37:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.428+0000 7f26ef7fe700 10 client.27388 handle_command_reply: tid=0 2023-12-14T18:37:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.429+0000 7f27223f3b80 1 client.27388 shutdown 2023-12-14T18:37:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.436+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 msgr2=0x55723433add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.436+0000 7f27223f3b80 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 0x55723433add0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2704011f10 tx=0x7f2704004910 comp rx=0 tx=0).stop 2023-12-14T18:37:18.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.448+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 shutdown_connections 2023-12-14T18:37:18.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.448+0000 7f27223f3b80 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55723434d5e0 0x55723434f9f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.448+0000 7f27223f3b80 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55723433db50 0x55723433add0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.448+0000 7f27223f3b80 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x55723433d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.448+0000 7f27223f3b80 1 --2- 172.21.15.31:0/560997456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x55723433a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.448+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 >> 172.21.15.31:0/560997456 conn(0x557234328ca0 msgr2=0x55723432e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.450+0000 7f26ef7fe700 0 client.27388 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:18.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.451+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 shutdown_connections 2023-12-14T18:37:18.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.456+0000 7f27223f3b80 1 -- 172.21.15.31:0/560997456 wait complete. 2023-12-14T18:37:18.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.459+0000 7f27223f3b80 20 client.27388 trim_cache size 0 max 16384 2023-12-14T18:37:18.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.486+0000 7f27223f3b80 1 Processor -- start 2023-12-14T18:37:18.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 -- start start 2023-12-14T18:37:18.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x5572343446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x557234335b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55723433db50 0x557234336050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572343335b0 con 0x55723433db50 2023-12-14T18:37:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723433b800 con 0x557234332c90 2023-12-14T18:37:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f27223f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572343300e0 con 0x557234332380 2023-12-14T18:37:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f270ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x5572343446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f270ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x5572343446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45676/0 (socket says 172.21.15.31:45676) 2023-12-14T18:37:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f270ffff700 1 -- 172.21.15.31:0/71875904 learned_addr learned my addr 172.21.15.31:0/71875904 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.489+0000 7f270f7fe700 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x557234335b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.492+0000 7f270f7fe700 1 -- 172.21.15.31:0/71875904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 msgr2=0x5572343446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.492+0000 7f270f7fe700 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x5572343446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.492+0000 7f270f7fe700 1 -- 172.21.15.31:0/71875904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55723433db50 msgr2=0x557234336050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.492+0000 7f270f7fe700 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55723433db50 0x557234336050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.492+0000 7f270f7fe700 1 -- 172.21.15.31:0/71875904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723424aec0 con 0x557234332c90 2023-12-14T18:37:18.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.493+0000 7f270f7fe700 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x557234335b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27080c9bd0 tx=0x7f2708099430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec35ac5a3eb9a471 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.498+0000 7f26f6ffd700 1 -- 172.21.15.31:0/71875904 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27080995e0 con 0x557234332c90 2023-12-14T18:37:18.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.498+0000 7f26f6ffd700 1 -- 172.21.15.31:0/71875904 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f270809d610 con 0x557234332c90 2023-12-14T18:37:18.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.498+0000 7f26f6ffd700 1 -- 172.21.15.31:0/71875904 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708099aa0 con 0x557234332c90 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 -- 172.21.15.31:0/71875904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 msgr2=0x557234335b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x557234335b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27080c9bd0 tx=0x7f2708099430 comp rx=0 tx=0).stop 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 -- 172.21.15.31:0/71875904 shutdown_connections 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55723433db50 0x557234336050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234332380 0x5572343446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 --2- 172.21.15.31:0/71875904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x557234332c90 0x557234335b10 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f27080c9bd0 tx=0x7f2708099430 comp rx=0 tx=0).stop 2023-12-14T18:37:18.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.500+0000 7f27223f3b80 1 -- 172.21.15.31:0/71875904 >> 172.21.15.31:0/71875904 conn(0x557234328ca0 msgr2=0x55723434f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.509+0000 7f27223f3b80 1 -- 172.21.15.31:0/71875904 shutdown_connections 2023-12-14T18:37:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.511+0000 7f27223f3b80 1 -- 172.21.15.31:0/71875904 wait complete. 2023-12-14T18:37:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.511+0000 7f27223f3b80 1 -- start start 2023-12-14T18:37:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 Processor -- start 2023-12-14T18:37:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x557234399860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x557234399da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234395470 0x5572343958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x557234395e90 con 0x557234332380 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x557234396000 con 0x55723433db50 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572343961c0 con 0x557234395470 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.512+0000 7f27223f3b80 10 client.0 _ll_register_callbacks cb 0x7f27040118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x557234399da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x557234399da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:44630/0 (socket says 172.21.15.31:44630) 2023-12-14T18:37:18.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 -- 172.21.15.31:0/2193664788 learned_addr learned my addr 172.21.15.31:0/2193664788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270ffff700 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x557234399860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 -- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234395470 msgr2=0x5572343958e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234395470 0x5572343958e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 -- 172.21.15.31:0/2193664788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 msgr2=0x557234399860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x557234399860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.513+0000 7f270f7fe700 1 -- 172.21.15.31:0/2193664788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2710111920 con 0x55723433db50 2023-12-14T18:37:18.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.514+0000 7f270f7fe700 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x557234399da0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27080c9c30 tx=0x7f270800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114ce9afdbd0f046 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.515+0000 7f26ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.176:3300/0 2023-12-14T18:37:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.515+0000 7f26ef7fe700 1 -- 172.21.15.31:0/2193664788 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708098cd0 con 0x55723433db50 2023-12-14T18:37:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.515+0000 7f26ef7fe700 1 -- 172.21.15.31:0/2193664788 <== mon.2 v2:172.21.15.176:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f27080a19a0 con 0x55723433db50 2023-12-14T18:37:18.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.515+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2700072040 con 0x55723433db50 2023-12-14T18:37:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.517+0000 7f26ef7fe700 1 -- 172.21.15.31:0/2193664788 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2708084250 con 0x55723433db50 2023-12-14T18:37:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.519+0000 7f27223f3b80 10 client.27317 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.519+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5572342319a0 con 0x55723433db50 2023-12-14T18:37:18.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.519+0000 7f26ef7fe700 1 -- 172.21.15.31:0/2193664788 <== mon.2 v2:172.21.15.176:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2700072040 con 0x55723433db50 2023-12-14T18:37:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f26ef7fe700 1 -- 172.21.15.31:0/2193664788 <== mon.2 v2:172.21.15.176:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f27080cc020 con 0x55723433db50 2023-12-14T18:37:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f27223f3b80 10 client.27317 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f27223f3b80 10 client.27317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f27223f3b80 20 client.27317 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f27223f3b80 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5572343c4850 0x5572343c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:18.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f27223f3b80 4 client.27317 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:37:18.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55723427da00 con 0x5572343c4850 2023-12-14T18:37:18.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.527+0000 7f270ffff700 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5572343c4850 0x5572343c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:18.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.528+0000 7f270ffff700 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5572343c4850 0x5572343c6c30 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca44bf3ba5ce419 server_cookie=43bc17c1d4882e82 in_seq=0 out_seq=0 2023-12-14T18:37:18.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.528+0000 7f26ef7fe700 10 client.27317 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:18.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.528+0000 7f26ef7fe700 1 -- 172.21.15.31:0/2193664788 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55723427da00 con 0x5572343c4850 2023-12-14T18:37:18.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.528+0000 7f26ef7fe700 10 client.27317 handle_command_reply: tid=0 2023-12-14T18:37:18.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.528+0000 7f27223f3b80 1 client.27317 shutdown 2023-12-14T18:37:18.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 msgr2=0x557234399da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x557234399da0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27080c9c30 tx=0x7f270800c680 comp rx=0 tx=0).stop 2023-12-14T18:37:18.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 shutdown_connections 2023-12-14T18:37:18.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5572343c4850 0x5572343c6c30 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557234395470 0x5572343958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x557234332380 0x557234399860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 --2- 172.21.15.31:0/2193664788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55723433db50 0x557234399da0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 >> 172.21.15.31:0/2193664788 conn(0x557234328ca0 msgr2=0x557234335b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.531+0000 7f26ef7fe700 0 client.27317 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:18.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.532+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 shutdown_connections 2023-12-14T18:37:18.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.532+0000 7f27223f3b80 1 -- 172.21.15.31:0/2193664788 wait complete. 2023-12-14T18:37:18.535 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:37:18.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.533+0000 7f27223f3b80 20 client.27317 trim_cache size 0 max 16384 2023-12-14T18:37:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.548+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f26f80a1490 msgr2=0x7f26f80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.548+0000 7f2716265700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f26f80a1490 0x7f26f80a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f270400a860 tx=0x7f27040048a0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.548+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107ec0 msgr2=0x7f2710115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.548+0000 7f2716265700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107ec0 0x7f2710115430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f270800bd70 tx=0x7f270800bda0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.550+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 shutdown_connections 2023-12-14T18:37:18.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.550+0000 7f2716265700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2710119a40 0x7f2710073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.550+0000 7f2716265700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2710107af0 0x7f2710114ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.550+0000 7f2716265700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f26f80a1490 0x7f26f80a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.550+0000 7f2716265700 1 --2- 172.21.15.31:0/744382265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2710107ec0 0x7f2710115430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:18.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.550+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 >> 172.21.15.31:0/744382265 conn(0x7f271006b7e0 msgr2=0x7f271011bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:18.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.554+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 shutdown_connections 2023-12-14T18:37:18.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:18.554+0000 7f2716265700 1 -- 172.21.15.31:0/744382265 wait complete. 2023-12-14T18:37:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:37:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:37:18.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:37:18.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:37:18.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:37:18.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:37:18.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:18 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:37:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:37:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:37:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:37:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:37:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:37:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:37:18.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:18 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:37:19.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:18 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:37:19.581 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:37:19.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:37:19.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.836+0000 7fdeddff3700 1 -- 172.21.15.31:0/1652216485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8119a40 msgr2=0x7fded811be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.836+0000 7fdeddff3700 1 --2- 172.21.15.31:0/1652216485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8119a40 0x7fded811be20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fded000b0a0 tx=0x7fded000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.845+0000 7fdeddff3700 1 -- 172.21.15.31:0/1652216485 shutdown_connections 2023-12-14T18:37:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.845+0000 7fdeddff3700 1 --2- 172.21.15.31:0/1652216485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8119a40 0x7fded811be20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.845+0000 7fdeddff3700 1 --2- 172.21.15.31:0/1652216485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8107af0 0x7fded810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.845+0000 7fdeddff3700 1 --2- 172.21.15.31:0/1652216485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8107ec0 0x7fded8108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.845+0000 7fdeddff3700 1 -- 172.21.15.31:0/1652216485 >> 172.21.15.31:0/1652216485 conn(0x7fded806b7e0 msgr2=0x7fded8071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.851+0000 7fdeddff3700 1 -- 172.21.15.31:0/1652216485 shutdown_connections 2023-12-14T18:37:19.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.852+0000 7fdeddff3700 1 -- 172.21.15.31:0/1652216485 wait complete. 2023-12-14T18:37:19.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.853+0000 7fdeddff3700 1 Processor -- start 2023-12-14T18:37:19.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 -- start start 2023-12-14T18:37:19.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8107af0 0x7fded81a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8107ec0 0x7fded81a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 0x7fded81a9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded81aa140 con 0x7fded8119a40 2023-12-14T18:37:19.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded81aa2b0 con 0x7fded8107ec0 2023-12-14T18:37:19.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.854+0000 7fdeddff3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded81aa470 con 0x7fded8107af0 2023-12-14T18:37:19.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 0x7fded81a9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 0x7fded81a9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53164/0 (socket says 172.21.15.31:53164) 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 -- 172.21.15.31:0/2470229933 learned_addr learned my addr 172.21.15.31:0/2470229933 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded6ffd700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8107ec0 0x7fded81a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 -- 172.21.15.31:0/2470229933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8107af0 msgr2=0x7fded81a5160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8107af0 0x7fded81a5160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 -- 172.21.15.31:0/2470229933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8107ec0 msgr2=0x7fded81a56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8107ec0 0x7fded81a56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 -- 172.21.15.31:0/2470229933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0009d00 con 0x7fded8119a40 2023-12-14T18:37:19.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.855+0000 7fded7fff700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 0x7fded81a9b00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fded000b070 tx=0x7fded0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:19.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.856+0000 7fded4ff9700 1 -- 172.21.15.31:0/2470229933 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000e040 con 0x7fded8119a40 2023-12-14T18:37:19.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.856+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded811f110 con 0x7fded8119a40 2023-12-14T18:37:19.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.856+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded811ef80 con 0x7fded8119a40 2023-12-14T18:37:19.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.856+0000 7fded4ff9700 1 -- 172.21.15.31:0/2470229933 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded00079d0 con 0x7fded8119a40 2023-12-14T18:37:19.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.856+0000 7fded4ff9700 1 -- 172.21.15.31:0/2470229933 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded001b790 con 0x7fded8119a40 2023-12-14T18:37:19.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.863+0000 7fdeea181b80 1 Processor -- start 2023-12-14T18:37:19.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.863+0000 7fded4ff9700 1 -- 172.21.15.31:0/2470229933 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fded001b8f0 con 0x7fded8119a40 2023-12-14T18:37:19.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.864+0000 7fded4ff9700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdec00a1380 0x7fdec00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.864+0000 7fded4ff9700 1 -- 172.21.15.31:0/2470229933 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fded00ce1b0 con 0x7fded8119a40 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 -- start start 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 0x558767d327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d3ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c1c660 con 0x558767d3dbd0 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c8b8f0 con 0x558767d32d10 2023-12-14T18:37:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.865+0000 7fdeea181b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c84d80 con 0x558767d32400 2023-12-14T18:37:19.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.866+0000 7fded7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d3ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.866+0000 7fded7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d3ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53176/0 (socket says 172.21.15.31:53176) 2023-12-14T18:37:19.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.866+0000 7fded7fff700 1 -- 172.21.15.31:0/2036935643 learned_addr learned my addr 172.21.15.31:0/2036935643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded7fff700 1 -- 172.21.15.31:0/2036935643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 msgr2=0x558767d327d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded7fff700 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 0x558767d327d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded7fff700 1 -- 172.21.15.31:0/2036935643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 msgr2=0x558767d3d690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded7fff700 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3d690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded7fff700 1 -- 172.21.15.31:0/2036935643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558767b06430 con 0x558767d3dbd0 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded7fff700 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d3ffb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fded009ae60 tx=0x7fded009ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a20dda4ff23935 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fded6ffd700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdec00a1380 0x7fdec00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fdebdffb700 1 -- 172.21.15.31:0/2036935643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00a3290 con 0x558767d3dbd0 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fdebdffb700 1 -- 172.21.15.31:0/2036935643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded00975e0 con 0x558767d3dbd0 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.868+0000 7fdebdffb700 1 -- 172.21.15.31:0/2036935643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded009a620 con 0x558767d3dbd0 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.869+0000 7fdeea181b80 1 -- 172.21.15.31:0/2036935643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 msgr2=0x558767d3ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.869+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d3ffb0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fded009ae60 tx=0x7fded009ab00 comp rx=0 tx=0).stop 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.870+0000 7fdeea181b80 1 -- 172.21.15.31:0/2036935643 shutdown_connections 2023-12-14T18:37:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.870+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d3ffb0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.870+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 0x558767d327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.870+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2036935643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.870+0000 7fdeea181b80 1 -- 172.21.15.31:0/2036935643 >> 172.21.15.31:0/2036935643 conn(0x558767d28d20 msgr2=0x558767d2e860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.870+0000 7fdeea181b80 1 -- 172.21.15.31:0/2036935643 shutdown_connections 2023-12-14T18:37:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.871+0000 7fdeea181b80 1 -- 172.21.15.31:0/2036935643 wait complete. 2023-12-14T18:37:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.871+0000 7fdeea181b80 1 -- start start 2023-12-14T18:37:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.871+0000 7fdeea181b80 1 Processor -- start 2023-12-14T18:37:19.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.872+0000 7fded6ffd700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdec00a1380 0x7fdec00a3830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558767d2fc10 tx=0x7fdec8009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 0x558767d3d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d3dbd0 0x558767d3ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c21010 con 0x558767d32400 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767c20cb0 con 0x558767d32d10 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767b08720 con 0x558767d3dbd0 2023-12-14T18:37:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.874+0000 7fdeea181b80 10 client.0 _ll_register_callbacks cb 0x558767c44990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.875+0000 7fded7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d3dbd0 0x558767d3ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.875+0000 7fded77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 0x558767d3d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.875+0000 7fded77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 0x558767d3d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53188/0 (socket says 172.21.15.31:53188) 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.875+0000 7fded77fe700 1 -- 172.21.15.31:0/3334497108 learned_addr learned my addr 172.21.15.31:0/3334497108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded77fe700 1 -- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d3dbd0 msgr2=0x558767d3ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded77fe700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d3dbd0 0x558767d3ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded77fe700 1 -- 172.21.15.31:0/3334497108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 msgr2=0x558767d3a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded6ffd700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded77fe700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded77fe700 1 -- 172.21.15.31:0/3334497108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded811ef80 con 0x558767d32400 2023-12-14T18:37:19.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.878+0000 7fded6ffd700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.879+0000 7fded77fe700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 0x558767d3d510 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009ec0 tx=0x7fdecc00b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36004f5144ae5cc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.879+0000 7fdeb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.879+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/3334497108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc011420 con 0x558767d32400 2023-12-14T18:37:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.879+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/3334497108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fdecc011580 con 0x558767d32400 2023-12-14T18:37:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.879+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558767d26330 con 0x558767d32400 2023-12-14T18:37:19.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.880+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/3334497108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdecc011740 con 0x558767d32400 2023-12-14T18:37:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.880+0000 7fdeea181b80 10 client.17622 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.880+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558767c4b1f0 con 0x558767d32400 2023-12-14T18:37:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/3334497108 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558767d26330 con 0x558767d32400 2023-12-14T18:37:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/3334497108 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdecc01aa00 con 0x558767d32400 2023-12-14T18:37:19.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeea181b80 10 client.17622 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeea181b80 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeea181b80 20 client.17622 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeea181b80 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767d5e440 0x558767d60850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeea181b80 4 client.17622 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:19.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558767c7dd60 con 0x558767d5e440 2023-12-14T18:37:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.881+0000 7fded6ffd700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767d5e440 0x558767d60850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.882+0000 7fded6ffd700 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767d5e440 0x558767d60850 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b040c35e16c8d91 server_cookie=b9028d7e49276d08 in_seq=0 out_seq=0 2023-12-14T18:37:19.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.883+0000 7fdeb6ffd700 10 client.17622 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:19.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.883+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/3334497108 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558767c7dd60 con 0x558767d5e440 2023-12-14T18:37:19.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.883+0000 7fdeb6ffd700 10 client.17622 handle_command_reply: tid=0 2023-12-14T18:37:19.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.883+0000 7fdeea181b80 1 client.17622 shutdown 2023-12-14T18:37:19.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.884+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 msgr2=0x558767d3d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.884+0000 7fdeea181b80 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 0x558767d3d510 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009ec0 tx=0x7fdecc00b3c0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.887+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 shutdown_connections 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.887+0000 7fdeea181b80 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767d5e440 0x558767d60850 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.887+0000 7fdeea181b80 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d3dbd0 0x558767d3ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.887+0000 7fdeea181b80 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d32400 0x558767d3d510 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.887+0000 7fdeea181b80 1 --2- 172.21.15.31:0/3334497108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.887+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 >> 172.21.15.31:0/3334497108 conn(0x558767d28d20 msgr2=0x558767d2e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.888+0000 7fdeb6ffd700 0 client.17622 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.888+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 shutdown_connections 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.888+0000 7fdeea181b80 1 -- 172.21.15.31:0/3334497108 wait complete. 2023-12-14T18:37:19.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.888+0000 7fdeea181b80 20 client.17622 trim_cache size 0 max 16384 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.917+0000 7fdeea181b80 1 Processor -- start 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.917+0000 7fdeea181b80 1 -- start start 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.917+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 0x558767d49010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.917+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d41fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.918+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d34ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.918+0000 7fdeea181b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d330e0 con 0x558767d3dbd0 2023-12-14T18:37:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.918+0000 7fdeea181b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d33410 con 0x558767d32d10 2023-12-14T18:37:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.918+0000 7fdeea181b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767d5ba80 con 0x558767d32400 2023-12-14T18:37:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d34ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d34ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53194/0 (socket says 172.21.15.31:53194) 2023-12-14T18:37:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 -- 172.21.15.31:0/723890896 learned_addr learned my addr 172.21.15.31:0/723890896 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 -- 172.21.15.31:0/723890896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 msgr2=0x558767d49010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 0x558767d49010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 -- 172.21.15.31:0/723890896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 msgr2=0x558767d41fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d41fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.919+0000 7fded7fff700 1 -- 172.21.15.31:0/723890896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558767d2a560 con 0x558767d3dbd0 2023-12-14T18:37:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.920+0000 7fded7fff700 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d34ae0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fded001de40 tx=0x7fded009aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6e442ca4c4632 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.920+0000 7fdebe7fc700 1 -- 172.21.15.31:0/723890896 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00a3290 con 0x558767d3dbd0 2023-12-14T18:37:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.920+0000 7fdebe7fc700 1 -- 172.21.15.31:0/723890896 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded009a3c0 con 0x558767d3dbd0 2023-12-14T18:37:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.921+0000 7fdebe7fc700 1 -- 172.21.15.31:0/723890896 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00a2e00 con 0x558767d3dbd0 2023-12-14T18:37:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.921+0000 7fdeea181b80 1 -- 172.21.15.31:0/723890896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 msgr2=0x558767d34ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.921+0000 7fdeea181b80 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d34ae0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fded001de40 tx=0x7fded009aa10 comp rx=0 tx=0).stop 2023-12-14T18:37:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.922+0000 7fdeea181b80 1 -- 172.21.15.31:0/723890896 shutdown_connections 2023-12-14T18:37:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.922+0000 7fdeea181b80 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d34ae0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.922+0000 7fdeea181b80 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d32400 0x558767d49010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.922+0000 7fdeea181b80 1 --2- 172.21.15.31:0/723890896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d32d10 0x558767d41fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.922+0000 7fdeea181b80 1 -- 172.21.15.31:0/723890896 >> 172.21.15.31:0/723890896 conn(0x558767d28d20 msgr2=0x558767d343d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:19.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.926+0000 7fdeea181b80 1 -- 172.21.15.31:0/723890896 shutdown_connections 2023-12-14T18:37:19.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.931+0000 7fdeea181b80 1 -- 172.21.15.31:0/723890896 wait complete. 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.931+0000 7fdeea181b80 1 -- start start 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 Processor -- start 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d420f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d403e0 0x558767d40850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 0x558767db2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767db2870 con 0x558767d3dbd0 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767db29b0 con 0x558767d403e0 2023-12-14T18:37:19.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558767db2b20 con 0x558767d40d90 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fdeea181b80 10 client.0 _ll_register_callbacks cb 0x7fdecc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fded7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 0x558767db2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fded7fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 0x558767db2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45740/0 (socket says 172.21.15.31:45740) 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.932+0000 7fded7fff700 1 -- 172.21.15.31:0/2517923205 learned_addr learned my addr 172.21.15.31:0/2517923205 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fded7fff700 1 -- 172.21.15.31:0/2517923205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d403e0 msgr2=0x558767d40850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fded7fff700 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d403e0 0x558767d40850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fded7fff700 1 -- 172.21.15.31:0/2517923205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 msgr2=0x558767d420f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fded7fff700 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d420f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fded7fff700 1 -- 172.21.15.31:0/2517923205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558767b06430 con 0x558767d40d90 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fded7fff700 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 0x558767db2330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fded00cba40 tx=0x7fded009b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b480a4c981858516 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fdeb6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/2517923205 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00b91e0 con 0x558767d40d90 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/2517923205 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fded00b9020 con 0x558767d40d90 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.933+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fded00975e0 con 0x558767d40d90 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/2517923205 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fded00a3010 con 0x558767d40d90 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 10 client.27400 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558767ab2be0 con 0x558767d40d90 2023-12-14T18:37:19.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/2517923205 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded00975e0 con 0x558767d40d90 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/2517923205 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fded00b32a0 con 0x558767d40d90 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 10 client.27400 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 10 client.27400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 20 client.27400 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767db3d10 0x558767db60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 4 client.27400 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:37:19.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.934+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558767c7da80 con 0x558767db3d10 2023-12-14T18:37:19.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.935+0000 7fded77fe700 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767db3d10 0x558767db60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:19.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.938+0000 7fdeb6ffd700 10 client.27400 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.938+0000 7fded77fe700 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767db3d10 0x558767db60f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8855e3c5dc643efa server_cookie=2e52873b9f0f3c83 in_seq=0 out_seq=0 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.939+0000 7fdeb6ffd700 1 -- 172.21.15.31:0/2517923205 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558767c7da80 con 0x558767db3d10 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.939+0000 7fdeb6ffd700 10 client.27400 handle_command_reply: tid=0 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.939+0000 7fdeea181b80 1 client.27400 shutdown 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 msgr2=0x558767db2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 0x558767db2330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fded00cba40 tx=0x7fded009b4a0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 shutdown_connections 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x558767db3d10 0x558767db60f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558767d40d90 0x558767db2330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x558767d3dbd0 0x558767d420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 --2- 172.21.15.31:0/2517923205 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x558767d403e0 0x558767d40850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 >> 172.21.15.31:0/2517923205 conn(0x558767d28d20 msgr2=0x558767d34c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:19.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.942+0000 7fdeb6ffd700 0 client.27400 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:19.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.950+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 shutdown_connections 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.957+0000 7fdeea181b80 1 -- 172.21.15.31:0/2517923205 wait complete. 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.958+0000 7fdeea181b80 20 client.27400 trim_cache size 0 max 16384 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "dbf8fdc5-e5ad-40ae-b452-bb22aed8d5da", 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:37:19.959 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:37:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.958+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdec00a1380 msgr2=0x7fdec00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.958+0000 7fdeddff3700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdec00a1380 0x7fdec00a3830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x558767d2fc10 tx=0x7fdec8009260 comp rx=0 tx=0).stop 2023-12-14T18:37:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.958+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 msgr2=0x7fded81a9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.958+0000 7fdeddff3700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 0x7fded81a9b00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fded000b070 tx=0x7fded0004c80 comp rx=0 tx=0).stop 2023-12-14T18:37:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.962+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 shutdown_connections 2023-12-14T18:37:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.962+0000 7fdeddff3700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fded8119a40 0x7fded81a9b00 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.962+0000 7fdeddff3700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdec00a1380 0x7fdec00a3830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.962+0000 7fdeddff3700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded8107af0 0x7fded81a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.962+0000 7fdeddff3700 1 --2- 172.21.15.31:0/2470229933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fded8107ec0 0x7fded81a56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:19.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.962+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 >> 172.21.15.31:0/2470229933 conn(0x7fded806b7e0 msgr2=0x7fded8071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:19.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:19 smithi031 ceph-mon[42085]: pgmap v295: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 5 op/s 2023-12-14T18:37:19.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.964+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 shutdown_connections 2023-12-14T18:37:19.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:19.965+0000 7fdeddff3700 1 -- 172.21.15.31:0/2470229933 wait complete. 2023-12-14T18:37:19.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:37:20.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:19 smithi204 ceph-mon[43866]: pgmap v295: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 5 op/s 2023-12-14T18:37:20.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:19 smithi176 ceph-mon[43506]: pgmap v295: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 5 op/s 2023-12-14T18:37:20.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.269+0000 7fcbb331e700 1 -- 172.21.15.31:0/1075354117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac106920 msgr2=0x7fcbac106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.269+0000 7fcbb331e700 1 --2- 172.21.15.31:0/1075354117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac106920 0x7fcbac106d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcba80098f0 tx=0x7fcba8009c00 comp rx=0 tx=0).stop 2023-12-14T18:37:20.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.283+0000 7fcbb331e700 1 -- 172.21.15.31:0/1075354117 shutdown_connections 2023-12-14T18:37:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.283+0000 7fcbb331e700 1 --2- 172.21.15.31:0/1075354117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac1072d0 0x7fcbac113640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.283+0000 7fcbb331e700 1 --2- 172.21.15.31:0/1075354117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac10c8a0 0x7fcbac10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.283+0000 7fcbb331e700 1 --2- 172.21.15.31:0/1075354117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac106920 0x7fcbac106d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.283+0000 7fcbb331e700 1 -- 172.21.15.31:0/1075354117 >> 172.21.15.31:0/1075354117 conn(0x7fcbac074830 msgr2=0x7fcbac076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.293+0000 7fcbb331e700 1 -- 172.21.15.31:0/1075354117 shutdown_connections 2023-12-14T18:37:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.294+0000 7fcbb331e700 1 -- 172.21.15.31:0/1075354117 wait complete. 2023-12-14T18:37:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.294+0000 7fcbb331e700 1 Processor -- start 2023-12-14T18:37:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.294+0000 7fcbb331e700 1 -- start start 2023-12-14T18:37:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.295+0000 7fcbb331e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac106920 0x7fcbac0720a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.295+0000 7fcbb331e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac1072d0 0x7fcbac072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.295+0000 7fcbb331e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac10c8a0 0x7fcbac06b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.295+0000 7fcbb331e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06b900 con 0x7fcbac1072d0 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.295+0000 7fcbb331e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06ba70 con 0x7fcbac106920 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.295+0000 7fcbb331e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac06bc30 con 0x7fcbac10c8a0 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb18bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac10c8a0 0x7fcbac06b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb08b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac1072d0 0x7fcbac072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb08b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac1072d0 0x7fcbac072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53214/0 (socket says 172.21.15.31:53214) 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb08b9700 1 -- 172.21.15.31:0/3303538122 learned_addr learned my addr 172.21.15.31:0/3303538122 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac106920 0x7fcbac0720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3303538122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac10c8a0 msgr2=0x7fcbac06b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac10c8a0 0x7fcbac06b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac1072d0 msgr2=0x7fcbac072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac1072d0 0x7fcbac072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.298+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3303538122 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8009520 con 0x7fcbac106920 2023-12-14T18:37:20.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.299+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac106920 0x7fcbac0720a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00e9c0 tx=0x7fcb9c00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:20.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.302+0000 7fcba27fc700 1 -- 172.21.15.31:0/3303538122 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00cbd0 con 0x7fcbac106920 2023-12-14T18:37:20.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.302+0000 7fcba27fc700 1 -- 172.21.15.31:0/3303538122 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb9c009e50 con 0x7fcbac106920 2023-12-14T18:37:20.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.302+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac116960 con 0x7fcbac106920 2023-12-14T18:37:20.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.302+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac1167d0 con 0x7fcbac106920 2023-12-14T18:37:20.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.304+0000 7fcba27fc700 1 -- 172.21.15.31:0/3303538122 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c010ea0 con 0x7fcbac106920 2023-12-14T18:37:20.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.306+0000 7fcbbf4acb80 1 Processor -- start 2023-12-14T18:37:20.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 -- start start 2023-12-14T18:37:20.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f5133750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f513e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5140f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f501d5e0 con 0x55e6f513eb50 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f508c870 con 0x55e6f5133c90 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f5085d00 con 0x55e6f5133380 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbb08b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbb08b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5140f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53226/0 (socket says 172.21.15.31:53226) 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2068997586 learned_addr learned my addr 172.21.15.31:0/2068997586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:20.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.307+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f5133750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.308+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2068997586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 msgr2=0x55e6f5133750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:20.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.308+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f5133750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.308+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2068997586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 msgr2=0x55e6f513e610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:20.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.308+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f513e610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.308+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2068997586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6f4f073b0 con 0x55e6f513eb50 2023-12-14T18:37:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.308+0000 7fcba27fc700 1 -- 172.21.15.31:0/3303538122 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcb9c018740 con 0x7fcbac106920 2023-12-14T18:37:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.309+0000 7fcba27fc700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcb980a1410 0x7fcb980a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.309+0000 7fcba27fc700 1 -- 172.21.15.31:0/3303538122 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fcb9c014070 con 0x7fcbac106920 2023-12-14T18:37:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.309+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcb980a1410 0x7fcb980a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.309+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5140f30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcba800bf70 tx=0x7fcba800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d5fe7fbf181875 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:20.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.310+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcb980a1410 0x7fcb980a38c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcbac072cf0 tx=0x7fcba40075c0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.313+0000 7fcb977fe700 1 -- 172.21.15.31:0/2068997586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba801c040 con 0x55e6f513eb50 2023-12-14T18:37:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.313+0000 7fcb977fe700 1 -- 172.21.15.31:0/2068997586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba800f7e0 con 0x55e6f513eb50 2023-12-14T18:37:20.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.313+0000 7fcb977fe700 1 -- 172.21.15.31:0/2068997586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80056e0 con 0x55e6f513eb50 2023-12-14T18:37:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.314+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2068997586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 msgr2=0x55e6f5140f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.314+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5140f30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcba800bf70 tx=0x7fcba800bfa0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2068997586 shutdown_connections 2023-12-14T18:37:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5140f30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f5133750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2068997586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f513e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2068997586 >> 172.21.15.31:0/2068997586 conn(0x55e6f5129ca0 msgr2=0x55e6f512f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:20.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2068997586 shutdown_connections 2023-12-14T18:37:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.315+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2068997586 wait complete. 2023-12-14T18:37:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.316+0000 7fcbbf4acb80 1 -- start start 2023-12-14T18:37:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.316+0000 7fcbbf4acb80 1 Processor -- start 2023-12-14T18:37:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.316+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 0x55e6f513e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 0x55e6f513b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f513be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f5021f90 con 0x55e6f513eb50 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f5021c30 con 0x55e6f5133380 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f4f096a0 con 0x55e6f5133c90 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbbf4acb80 10 client.0 _ll_register_callbacks cb 0x55e6f5045910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbb08b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 0x55e6f513b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbb08b9700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 0x55e6f513b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45770/0 (socket says 172.21.15.31:45770) 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2842108722 learned_addr learned my addr 172.21.15.31:0/2842108722 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f513be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.317+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2842108722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 msgr2=0x55e6f513e560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 0x55e6f513e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2842108722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 msgr2=0x55e6f513be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f513be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcbb08b9700 1 -- 172.21.15.31:0/2842108722 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac113f40 con 0x55e6f5133c90 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 0x55e6f513b810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c010 tx=0x7fcba800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912464ad940c4313 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcb94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:37:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcb94ff9700 1 -- 172.21.15.31:0/2842108722 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba801c050 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6f51272b0 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.318+0000 7fcb94ff9700 1 -- 172.21.15.31:0/2842108722 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fcba800f950 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcb94ff9700 1 -- 172.21.15.31:0/2842108722 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba8005720 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 10 client.27409 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e6f504c170 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcb94ff9700 1 -- 172.21.15.31:0/2842108722 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6f51272b0 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcb94ff9700 1 -- 172.21.15.31:0/2842108722 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcba8015020 con 0x55e6f5133c90 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 10 client.27409 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 10 client.27409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 20 client.27409 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f514e070 0x55e6f5150480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 4 client.27409 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.319+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6f507ece0 con 0x55e6f514e070 2023-12-14T18:37:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.320+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f514e070 0x55e6f5150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.320+0000 7fcb94ff9700 10 client.27409 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.320+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f514e070 0x55e6f5150480 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3002323180528b5a server_cookie=f6112639678847a9 in_seq=0 out_seq=0 2023-12-14T18:37:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.328+0000 7fcb94ff9700 1 -- 172.21.15.31:0/2842108722 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e6f507ece0 con 0x55e6f514e070 2023-12-14T18:37:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.328+0000 7fcb94ff9700 10 client.27409 handle_command_reply: tid=0 2023-12-14T18:37:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.328+0000 7fcbbf4acb80 1 client.27409 shutdown 2023-12-14T18:37:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.332+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 msgr2=0x55e6f513b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.332+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 0x55e6f513b810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c010 tx=0x7fcba800b840 comp rx=0 tx=0).stop 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 shutdown_connections 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f514e070 0x55e6f5150480 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f513be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 0x55e6f513e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/2842108722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133c90 0x55e6f513b810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcb94ff9700 0 client.27409 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:20.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.335+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 >> 172.21.15.31:0/2842108722 conn(0x55e6f5129ca0 msgr2=0x55e6f512f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:20.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.343+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 shutdown_connections 2023-12-14T18:37:20.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.345+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/2842108722 wait complete. 2023-12-14T18:37:20.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.346+0000 7fcbbf4acb80 20 client.27409 trim_cache size 0 max 16384 2023-12-14T18:37:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.378+0000 7fcbbf4acb80 1 Processor -- start 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.378+0000 7fcbbf4acb80 1 -- start start 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 0x55e6f513e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f5133c90 0x55e6f513e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 0x55e6f5136b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f51376c0 con 0x55e6f5133c90 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f513c440 con 0x55e6f5133380 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f5133750 con 0x55e6f513eb50 2023-12-14T18:37:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbb18bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 0x55e6f5136b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbb18bb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 0x55e6f5136b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45772/0 (socket says 172.21.15.31:45772) 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbb18bb700 1 -- 172.21.15.31:0/619475112 learned_addr learned my addr 172.21.15.31:0/619475112 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.379+0000 7fcbb18bb700 1 -- 172.21.15.31:0/619475112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 msgr2=0x55e6f513e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 0x55e6f513e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcbb18bb700 1 -- 172.21.15.31:0/619475112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f5133c90 msgr2=0x55e6f513e560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f5133c90 0x55e6f513e560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcbb18bb700 1 -- 172.21.15.31:0/619475112 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6f504bec0 con 0x55e6f513eb50 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 0x55e6f5136b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcba4007cb0 tx=0x7fcba4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fc46baf91e347a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcb97fff700 1 -- 172.21.15.31:0/619475112 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4020620 con 0x55e6f513eb50 2023-12-14T18:37:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcb97fff700 1 -- 172.21.15.31:0/619475112 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba4012e30 con 0x55e6f513eb50 2023-12-14T18:37:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.380+0000 7fcb97fff700 1 -- 172.21.15.31:0/619475112 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba401fa80 con 0x55e6f513eb50 2023-12-14T18:37:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.381+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/619475112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 msgr2=0x55e6f5136b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.381+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 0x55e6f5136b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcba4007cb0 tx=0x7fcba4012000 comp rx=0 tx=0).stop 2023-12-14T18:37:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.382+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/619475112 shutdown_connections 2023-12-14T18:37:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.382+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f513eb50 0x55e6f5136b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.382+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133380 0x55e6f513e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.382+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/619475112 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f5133c90 0x55e6f513e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.382+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/619475112 >> 172.21.15.31:0/619475112 conn(0x55e6f5129ca0 msgr2=0x55e6f5149e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:20.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.388+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/619475112 shutdown_connections 2023-12-14T18:37:20.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.389+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/619475112 wait complete. 2023-12-14T18:37:20.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 -- start start 2023-12-14T18:37:20.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 Processor -- start 2023-12-14T18:37:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f519d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f519d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f51455e0 con 0x55e6f513eb50 2023-12-14T18:37:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f5145750 con 0x55e6f5133c90 2023-12-14T18:37:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.390+0000 7fcbbf4acb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6f5145910 con 0x55e6f5133380 2023-12-14T18:37:20.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbbf4acb80 10 client.0 _ll_register_callbacks cb 0x7fcba4008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:20.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f519d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f519d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:45784/0 (socket says 172.21.15.31:45784) 2023-12-14T18:37:20.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3900984980 learned_addr learned my addr 172.21.15.31:0/3900984980 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:20.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f519d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb18bb700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3900984980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 msgr2=0x55e6f519d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:20.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f519d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3900984980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 msgr2=0x55e6f5144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.391+0000 7fcbb10ba700 1 -- 172.21.15.31:0/3900984980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba400fd10 con 0x55e6f5133380 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.392+0000 7fcbb10ba700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f519d350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0963a0 tx=0x7fcb9c009690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e284595534104e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.392+0000 7fcb94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.392+0000 7fcb94ff9700 1 -- 172.21.15.31:0/3900984980 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c09a300 con 0x55e6f5133380 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.392+0000 7fcb94ff9700 1 -- 172.21.15.31:0/3900984980 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fcb9c09ded0 con 0x55e6f5133380 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.392+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba800f7e0 con 0x55e6f5133380 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.392+0000 7fcb94ff9700 1 -- 172.21.15.31:0/3900984980 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb9c0a22f0 con 0x55e6f5133380 2023-12-14T18:37:20.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.393+0000 7fcbbf4acb80 10 client.27415 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:20.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.393+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e6f50329a0 con 0x55e6f5133380 2023-12-14T18:37:20.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.394+0000 7fcb94ff9700 1 -- 172.21.15.31:0/3900984980 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba800f7e0 con 0x55e6f5133380 2023-12-14T18:37:20.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.394+0000 7fcb94ff9700 1 -- 172.21.15.31:0/3900984980 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcb9c0a1e60 con 0x55e6f5133380 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbbf4acb80 10 client.27415 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbbf4acb80 10 client.27415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbbf4acb80 20 client.27415 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f519a180 0x55e6f519a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbbf4acb80 4 client.27415 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e6f507ea00 con 0x55e6f519a180 2023-12-14T18:37:20.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.395+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f519a180 0x55e6f519a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.396+0000 7fcbb08b9700 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f519a180 0x55e6f519a580 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1b2014ad43ca91 server_cookie=4b32b4e22f39430f in_seq=0 out_seq=0 2023-12-14T18:37:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.396+0000 7fcb94ff9700 10 client.27415 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.396+0000 7fcb94ff9700 1 -- 172.21.15.31:0/3900984980 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e6f507ea00 con 0x55e6f519a180 2023-12-14T18:37:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.396+0000 7fcb94ff9700 10 client.27415 handle_command_reply: tid=0 2023-12-14T18:37:20.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.396+0000 7fcbbf4acb80 1 client.27415 shutdown 2023-12-14T18:37:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.403+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 msgr2=0x55e6f519d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.403+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f519d350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0963a0 tx=0x7fcb9c009690 comp rx=0 tx=0).stop 2023-12-14T18:37:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.404+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 shutdown_connections 2023-12-14T18:37:20.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.404+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55e6f519a180 0x55e6f519a580 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.404+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55e6f513eb50 0x55e6f5144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.404+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e6f5133380 0x55e6f519d350 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.404+0000 7fcbbf4acb80 1 --2- 172.21.15.31:0/3900984980 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55e6f5133c90 0x55e6f519d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.404+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 >> 172.21.15.31:0/3900984980 conn(0x55e6f5129ca0 msgr2=0x55e6f5136b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:20.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.408+0000 7fcb94ff9700 0 client.27415 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:20.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.409+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 shutdown_connections 2023-12-14T18:37:20.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.410+0000 7fcbbf4acb80 1 -- 172.21.15.31:0/3900984980 wait complete. 2023-12-14T18:37:20.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.414+0000 7fcbbf4acb80 20 client.27415 trim_cache size 0 max 16384 2023-12-14T18:37:20.418 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:37:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:37:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:37:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "dbf8fdc5-e5ad-40ae-b452-bb22aed8d5da": { 2023-12-14T18:37:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:37:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "dbf8fdc5-e5ad-40ae-b452-bb22aed8d5da", 2023-12-14T18:37:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:37:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:37:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:37:20.419 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:37:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.418+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcb980a1410 msgr2=0x7fcb980a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.418+0000 7fcbb331e700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcb980a1410 0x7fcb980a38c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcbac072cf0 tx=0x7fcba40075c0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.418+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac106920 msgr2=0x7fcbac0720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:20.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.418+0000 7fcbb331e700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac106920 0x7fcbac0720a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00e9c0 tx=0x7fcb9c00ed80 comp rx=0 tx=0).stop 2023-12-14T18:37:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.421+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 shutdown_connections 2023-12-14T18:37:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.421+0000 7fcbb331e700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcb980a1410 0x7fcb980a38c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.421+0000 7fcbb331e700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbac10c8a0 0x7fcbac06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.421+0000 7fcbb331e700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcbac106920 0x7fcbac0720a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.421+0000 7fcbb331e700 1 --2- 172.21.15.31:0/3303538122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcbac1072d0 0x7fcbac072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:20.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.421+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 >> 172.21.15.31:0/3303538122 conn(0x7fcbac074830 msgr2=0x7fcbac076340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:20.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.427+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 shutdown_connections 2023-12-14T18:37:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:20.432+0000 7fcbb331e700 1 -- 172.21.15.31:0/3303538122 wait complete. 2023-12-14T18:37:20.466 INFO:tasks.cephfs.filesystem:scrub status for tag:dbf8fdc5-e5ad-40ae-b452-bb22aed8d5da - {'path': '/', 'tag': 'dbf8fdc5-e5ad-40ae-b452-bb22aed8d5da', 'options': 'recursive,force'} 2023-12-14T18:37:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:20 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:37:20.772 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:20 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:20 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:37:21.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:20 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:21.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:20 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:37:21.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:20 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:21 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:37:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:21 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:21 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:37:22.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:21 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:21 smithi204 ceph-mon[43866]: pgmap v296: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2023-12-14T18:37:22.217 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:21 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:37:22.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:21 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:21 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:37:22.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:21 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.218 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:21 smithi031 ceph-mon[42085]: pgmap v296: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2023-12-14T18:37:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:21 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:37:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:21 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:21 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:37:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:21 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:22.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:21 smithi176 ceph-mon[43506]: pgmap v296: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 189 B/s wr, 5 op/s 2023-12-14T18:37:24.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:23 smithi204 ceph-mon[43866]: pgmap v297: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-12-14T18:37:24.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:23 smithi031 ceph-mon[42085]: pgmap v297: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-12-14T18:37:24.243 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:23 smithi176 ceph-mon[43506]: pgmap v297: 129 pgs: 22 active+clean+snaptrim, 37 active+clean, 70 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-12-14T18:37:26.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:25 smithi204 ceph-mon[43866]: pgmap v298: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-14T18:37:26.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:25 smithi031 ceph-mon[42085]: pgmap v298: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-14T18:37:26.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:25 smithi176 ceph-mon[43506]: pgmap v298: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 8 op/s 2023-12-14T18:37:28.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:27 smithi204 ceph-mon[43866]: pgmap v299: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:37:28.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:27 smithi031 ceph-mon[42085]: pgmap v299: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:37:28.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:27 smithi176 ceph-mon[43506]: pgmap v299: 129 pgs: 21 active+clean+snaptrim, 41 active+clean, 67 active+clean+snaptrim_wait; 544 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:37:30.215 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:29 smithi031 ceph-mon[42085]: pgmap v300: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-12-14T18:37:30.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:29 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:37:30.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:29 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:37:30.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:29 smithi176 ceph-mon[43506]: pgmap v300: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-12-14T18:37:30.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:29 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:37:30.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:29 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:37:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:29 smithi204 ceph-mon[43866]: pgmap v300: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 8 op/s 2023-12-14T18:37:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:29 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:37:30.340 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:29 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:37:32.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:31 smithi031 ceph-mon[42085]: pgmap v301: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:32.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:31 smithi176 ceph-mon[43506]: pgmap v301: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:32.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:31 smithi204 ceph-mon[43866]: pgmap v301: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:34.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:33 smithi031 ceph-mon[42085]: pgmap v302: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:34.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:33 smithi176 ceph-mon[43506]: pgmap v302: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:34.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:33 smithi204 ceph-mon[43866]: pgmap v302: 129 pgs: 16 active+clean+snaptrim, 58 active+clean, 55 active+clean+snaptrim_wait; 544 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:36.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:35 smithi031 ceph-mon[42085]: pgmap v303: 129 pgs: 14 active+clean+snaptrim, 75 active+clean, 40 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:37:36.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:35 smithi176 ceph-mon[43506]: pgmap v303: 129 pgs: 14 active+clean+snaptrim, 75 active+clean, 40 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:37:36.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:35 smithi204 ceph-mon[43866]: pgmap v303: 129 pgs: 14 active+clean+snaptrim, 75 active+clean, 40 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2023-12-14T18:37:38.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:37 smithi031 ceph-mon[42085]: pgmap v304: 129 pgs: 14 active+clean+snaptrim, 75 active+clean, 40 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:38.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:37 smithi176 ceph-mon[43506]: pgmap v304: 129 pgs: 14 active+clean+snaptrim, 75 active+clean, 40 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:38.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:37 smithi204 ceph-mon[43866]: pgmap v304: 129 pgs: 14 active+clean+snaptrim, 75 active+clean, 40 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:40.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:39 smithi031 ceph-mon[42085]: pgmap v305: 129 pgs: 4 active+clean+snaptrim, 113 active+clean, 12 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-14T18:37:40.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:39 smithi176 ceph-mon[43506]: pgmap v305: 129 pgs: 4 active+clean+snaptrim, 113 active+clean, 12 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-14T18:37:40.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:39 smithi204 ceph-mon[43866]: pgmap v305: 129 pgs: 4 active+clean+snaptrim, 113 active+clean, 12 active+clean+snaptrim_wait; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-12-14T18:37:42.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:41 smithi031 ceph-mon[42085]: pgmap v306: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:37:42.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:41 smithi176 ceph-mon[43506]: pgmap v306: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:37:42.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:41 smithi204 ceph-mon[43866]: pgmap v306: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:37:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:43 smithi031 ceph-mon[42085]: pgmap v307: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:44.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:43 smithi031 ceph-mon[42085]: osdmap e98: 12 total, 12 up, 12 in 2023-12-14T18:37:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:43 smithi176 ceph-mon[43506]: pgmap v307: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:44.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:43 smithi176 ceph-mon[43506]: osdmap e98: 12 total, 12 up, 12 in 2023-12-14T18:37:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:43 smithi204 ceph-mon[43866]: pgmap v307: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:37:44.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:43 smithi204 ceph-mon[43866]: osdmap e98: 12 total, 12 up, 12 in 2023-12-14T18:37:46.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:45 smithi031 ceph-mon[42085]: pgmap v309: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:46.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:45 smithi176 ceph-mon[43506]: pgmap v309: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:46.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:45 smithi204 ceph-mon[43866]: pgmap v309: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:48.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:47 smithi031 ceph-mon[42085]: pgmap v310: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:48.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:47 smithi176 ceph-mon[43506]: pgmap v310: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:48.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:47 smithi204 ceph-mon[43866]: pgmap v310: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:50.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:49 smithi031 ceph-mon[42085]: pgmap v311: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:50.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:49 smithi176 ceph-mon[43506]: pgmap v311: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:50.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:49 smithi204 ceph-mon[43866]: pgmap v311: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:50.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 -- 172.21.15.31:0/2332028873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13101015f0 msgr2=0x7f1310104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 --2- 172.21.15.31:0/2332028873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13101015f0 0x7f1310104e20 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f12f80099c0 tx=0x7f12f8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 -- 172.21.15.31:0/2332028873 shutdown_connections 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 --2- 172.21.15.31:0/2332028873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1310105550 0x7f131010d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 --2- 172.21.15.31:0/2332028873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1310100c50 0x7f1310101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 --2- 172.21.15.31:0/2332028873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13101015f0 0x7f1310104e20 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 -- 172.21.15.31:0/2332028873 >> 172.21.15.31:0/2332028873 conn(0x7f1310077570 msgr2=0x7f1310077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 -- 172.21.15.31:0/2332028873 shutdown_connections 2023-12-14T18:37:50.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.615+0000 7f131664d700 1 -- 172.21.15.31:0/2332028873 wait complete. 2023-12-14T18:37:50.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.616+0000 7f131664d700 1 Processor -- start 2023-12-14T18:37:50.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.616+0000 7f131664d700 1 -- start start 2023-12-14T18:37:50.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f131664d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1310100c50 0x7f1310196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f131664d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101015f0 0x7f13101969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f131664d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 0x7f131019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f131664d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131019b440 con 0x7f1310105550 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f131664d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131019b5b0 con 0x7f13101015f0 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f131664d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131019b770 con 0x7f1310100c50 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 0x7f131019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 0x7f131019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50132/0 (socket says 172.21.15.31:50132) 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 -- 172.21.15.31:0/3521071491 learned_addr learned my addr 172.21.15.31:0/3521071491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f130ffff700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1310100c50 0x7f1310196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1307fff700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101015f0 0x7f13101969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 -- 172.21.15.31:0/3521071491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1310100c50 msgr2=0x7f1310196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1310100c50 0x7f1310196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 -- 172.21.15.31:0/3521071491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101015f0 msgr2=0x7f13101969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101015f0 0x7f13101969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 -- 172.21.15.31:0/3521071491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80095f0 con 0x7f1310105550 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.617+0000 7f1314bea700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 0x7f131019ae00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1308007ee0 tx=0x7f130800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.618+0000 7f130dffb700 1 -- 172.21.15.31:0/3521071491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130800ddd0 con 0x7f1310105550 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.618+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13101102d0 con 0x7f1310105550 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.618+0000 7f130dffb700 1 -- 172.21.15.31:0/3521071491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f130800f040 con 0x7f1310105550 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.618+0000 7f130dffb700 1 -- 172.21.15.31:0/3521071491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080048e0 con 0x7f1310105550 2023-12-14T18:37:50.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.618+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310110140 con 0x7f1310105550 2023-12-14T18:37:50.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.620+0000 7f130dffb700 1 -- 172.21.15.31:0/3521071491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1308003a00 con 0x7f1310105550 2023-12-14T18:37:50.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.620+0000 7f130dffb700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12fc0a12c0 0x7f12fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.621+0000 7f130dffb700 1 -- 172.21.15.31:0/3521071491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f13080cc970 con 0x7f1310105550 2023-12-14T18:37:50.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.621+0000 7f130ffff700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12fc0a12c0 0x7f12fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.621+0000 7f130ffff700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12fc0a12c0 0x7f12fc0a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1300007be0 tx=0x7f13000073d0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.621+0000 7f13227dbb80 1 Processor -- start 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.621+0000 7f13227dbb80 1 -- start start 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x561044548750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044553610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044555f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13227dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610444325e0 con 0x561044548380 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13227dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610444a1870 con 0x561044548c90 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13227dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104449ad00 con 0x561044553b50 2023-12-14T18:37:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x561044548750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x561044548750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50136/0 (socket says 172.21.15.31:50136) 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 -- 172.21.15.31:0/2535333167 learned_addr learned my addr 172.21.15.31:0/2535333167 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f1314bea700 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044555f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f1307fff700 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044553610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 -- 172.21.15.31:0/2535333167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 msgr2=0x561044555f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044555f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 -- 172.21.15.31:0/2535333167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 msgr2=0x561044553610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044553610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 -- 172.21.15.31:0/2535333167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104431c3b0 con 0x561044548380 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f130ffff700 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x561044548750 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1300010e80 tx=0x7f1300013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba17d4a3fcccc09 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.622+0000 7f13067fc700 1 -- 172.21.15.31:0/2535333167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300010eb0 con 0x561044548380 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13067fc700 1 -- 172.21.15.31:0/2535333167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300003b40 con 0x561044548380 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13067fc700 1 -- 172.21.15.31:0/2535333167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300011c00 con 0x561044548380 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 -- 172.21.15.31:0/2535333167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 msgr2=0x561044548750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x561044548750 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1300010e80 tx=0x7f1300013000 comp rx=0 tx=0).stop 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 -- 172.21.15.31:0/2535333167 shutdown_connections 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044555f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x561044548750 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2535333167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044553610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 -- 172.21.15.31:0/2535333167 >> 172.21.15.31:0/2535333167 conn(0x56104453eca0 msgr2=0x5610445447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 -- 172.21.15.31:0/2535333167 shutdown_connections 2023-12-14T18:37:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.623+0000 7f13227dbb80 1 -- 172.21.15.31:0/2535333167 wait complete. 2023-12-14T18:37:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.624+0000 7f13227dbb80 1 -- start start 2023-12-14T18:37:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.624+0000 7f13227dbb80 1 Processor -- start 2023-12-14T18:37:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x5610445533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548c90 0x561044550760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044553b50 0x561044550ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561044436f90 con 0x561044548380 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561044436c30 con 0x561044553b50 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104431e6a0 con 0x561044548c90 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x5610445533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 10 client.0 _ll_register_callbacks cb 0x56104445a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x5610445533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50144/0 (socket says 172.21.15.31:50144) 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 -- 172.21.15.31:0/4223532890 learned_addr learned my addr 172.21.15.31:0/4223532890 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:50.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f1314bea700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044553b50 0x561044550ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f1307fff700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548c90 0x561044550760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 -- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548c90 msgr2=0x561044550760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548c90 0x561044550760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 -- 172.21.15.31:0/4223532890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044553b50 msgr2=0x561044550ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044553b50 0x561044550ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 -- 172.21.15.31:0/4223532890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1300010b60 con 0x561044548380 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f130ffff700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x5610445533d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1300011780 tx=0x7f1300003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892199e3d5783e77 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f12ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f12ef7fe700 1 -- 172.21.15.31:0/4223532890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300030070 con 0x561044548380 2023-12-14T18:37:50.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f12ef7fe700 1 -- 172.21.15.31:0/4223532890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f1300017070 con 0x561044548380 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.625+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56104453c2b0 con 0x561044548380 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f12ef7fe700 1 -- 172.21.15.31:0/4223532890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1300010eb0 con 0x561044548380 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 10 client.17652 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561044461170 con 0x561044548380 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f12ef7fe700 1 -- 172.21.15.31:0/4223532890 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56104453c2b0 con 0x561044548380 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f12ef7fe700 1 -- 172.21.15.31:0/4223532890 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1300033020 con 0x561044548380 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 10 client.17652 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:50.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 20 client.17652 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561044563310 0x561044565720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 4 client.17652 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561044493ce0 con 0x561044563310 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.626+0000 7f1307fff700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561044563310 0x561044565720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.627+0000 7f1307fff700 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561044563310 0x561044565720 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9aa07dc2d44716 server_cookie=175274f8efb1948d in_seq=0 out_seq=0 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.627+0000 7f12ef7fe700 10 client.17652 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.627+0000 7f12ef7fe700 1 -- 172.21.15.31:0/4223532890 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561044493ce0 con 0x561044563310 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.627+0000 7f12ef7fe700 10 client.17652 handle_command_reply: tid=0 2023-12-14T18:37:50.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 client.17652 shutdown 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 msgr2=0x5610445533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x5610445533d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1300011780 tx=0x7f1300003f60 comp rx=0 tx=0).stop 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 shutdown_connections 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x561044563310 0x561044565720 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044553b50 0x561044550ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548380 0x5610445533d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f12ef7fe700 0 client.17652 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 --2- 172.21.15.31:0/4223532890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548c90 0x561044550760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 >> 172.21.15.31:0/4223532890 conn(0x56104453eca0 msgr2=0x5610445443d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 shutdown_connections 2023-12-14T18:37:50.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 1 -- 172.21.15.31:0/4223532890 wait complete. 2023-12-14T18:37:50.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.628+0000 7f13227dbb80 20 client.17652 trim_cache size 0 max 16384 2023-12-14T18:37:50.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 Processor -- start 2023-12-14T18:37:50.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 -- start start 2023-12-14T18:37:50.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548380 0x56104455a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044554f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 0x561044555470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561044551cd0 con 0x561044553b50 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104453e820 con 0x561044548c90 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f13227dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104454c5f0 con 0x561044548380 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f1314bea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 0x561044555470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f1314bea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 0x561044555470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50158/0 (socket says 172.21.15.31:50158) 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f130ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548380 0x56104455a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.646+0000 7f1314bea700 1 -- 172.21.15.31:0/2420962562 learned_addr learned my addr 172.21.15.31:0/2420962562 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f1307fff700 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044554f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f130ffff700 1 -- 172.21.15.31:0/2420962562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 msgr2=0x561044554f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f130ffff700 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044554f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f130ffff700 1 -- 172.21.15.31:0/2420962562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 msgr2=0x561044555470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f130ffff700 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 0x561044555470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f130ffff700 1 -- 172.21.15.31:0/2420962562 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561044460ec0 con 0x561044548380 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f1314bea700 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 0x561044555470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f130ffff700 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548380 0x56104455a680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f130002dff0 tx=0x7f1300005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbfa417a05e788d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f1306ffd700 1 -- 172.21.15.31:0/2420962562 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300030070 con 0x561044548380 2023-12-14T18:37:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f1306ffd700 1 -- 172.21.15.31:0/2420962562 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300005670 con 0x561044548380 2023-12-14T18:37:50.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.647+0000 7f1306ffd700 1 -- 172.21.15.31:0/2420962562 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130001f430 con 0x561044548380 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 -- 172.21.15.31:0/2420962562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548380 msgr2=0x56104455a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548380 0x56104455a680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f130002dff0 tx=0x7f1300005980 comp rx=0 tx=0).stop 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 -- 172.21.15.31:0/2420962562 shutdown_connections 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044553b50 0x561044555470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044548380 0x56104455a680 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2420962562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548c90 0x561044554f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 -- 172.21.15.31:0/2420962562 >> 172.21.15.31:0/2420962562 conn(0x56104453eca0 msgr2=0x561044542f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 -- 172.21.15.31:0/2420962562 shutdown_connections 2023-12-14T18:37:50.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 -- 172.21.15.31:0/2420962562 wait complete. 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.648+0000 7f13227dbb80 1 -- start start 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 Processor -- start 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548380 0x56104455a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 0x56104455a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044558670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561044558c40 con 0x561044548c90 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x561044558db0 con 0x561044548380 2023-12-14T18:37:50.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561044558f70 con 0x561044553b50 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f13227dbb80 10 client.0 _ll_register_callbacks cb 0x7f1308007f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 0x56104455a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 0x56104455a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50164/0 (socket says 172.21.15.31:50164) 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 -- 172.21.15.31:0/2466366013 learned_addr learned my addr 172.21.15.31:0/2466366013 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f130ffff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548380 0x56104455a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1314bea700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044558670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 -- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 msgr2=0x561044558670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044558670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 -- 172.21.15.31:0/2466366013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548380 msgr2=0x56104455a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548380 0x56104455a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f1307fff700 1 -- 172.21.15.31:0/2466366013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1310110140 con 0x561044548c90 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.649+0000 7f130ffff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548380 0x56104455a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f1307fff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 0x56104455a550 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b580 tx=0x7f12f8027ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356955c2d7564d74 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f12ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f12ef7fe700 1 -- 172.21.15.31:0/2466366013 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f801f030 con 0x561044548c90 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f12ef7fe700 1 -- 172.21.15.31:0/2466366013 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f12f8005400 con 0x561044548c90 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1300003b40 con 0x561044548c90 2023-12-14T18:37:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f12ef7fe700 1 -- 172.21.15.31:0/2466366013 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12f801d040 con 0x561044548c90 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 10 client.17658 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561044461170 con 0x561044548c90 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f12ef7fe700 1 -- 172.21.15.31:0/2466366013 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1300003b40 con 0x561044548c90 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f12ef7fe700 1 -- 172.21.15.31:0/2466366013 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12f800b8f0 con 0x561044548c90 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 10 client.17658 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 20 client.17658 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5610445b83c0 0x5610445ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.650+0000 7f13227dbb80 4 client.17658 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561044493a00 con 0x5610445b83c0 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f130ffff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5610445b83c0 0x5610445ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f130ffff700 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5610445b83c0 0x5610445ba7d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958aa3e12bf3b29 server_cookie=2b769899405f171b in_seq=0 out_seq=0 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f12ef7fe700 10 client.17658 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f12ef7fe700 1 -- 172.21.15.31:0/2466366013 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561044493a00 con 0x5610445b83c0 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f12ef7fe700 10 client.17658 handle_command_reply: tid=0 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.651+0000 7f13227dbb80 1 client.17658 shutdown 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 msgr2=0x56104455a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 0x56104455a550 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f12f800b580 tx=0x7f12f8027ad0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 shutdown_connections 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5610445b83c0 0x5610445ba7d0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561044553b50 0x561044558670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f12ef7fe700 0 client.17658 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x561044548380 0x56104455a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 --2- 172.21.15.31:0/2466366013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561044548c90 0x56104455a550 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 >> 172.21.15.31:0/2466366013 conn(0x56104453eca0 msgr2=0x561044551e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 shutdown_connections 2023-12-14T18:37:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 1 -- 172.21.15.31:0/2466366013 wait complete. 2023-12-14T18:37:50.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f13227dbb80 20 client.17658 trim_cache size 0 max 16384 2023-12-14T18:37:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:37:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:37:50.663 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:37:50.663 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:37:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12fc0a12c0 msgr2=0x7f12fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12fc0a12c0 0x7f12fc0a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1300007be0 tx=0x7f13000073d0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 msgr2=0x7f131019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 0x7f131019ae00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f1308007ee0 tx=0x7f130800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 shutdown_connections 2023-12-14T18:37:50.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1310105550 0x7f131019ae00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f12fc0a12c0 0x7f12fc0a3770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1310100c50 0x7f1310196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 --2- 172.21.15.31:0/3521071491 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f13101015f0 0x7f13101969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 >> 172.21.15.31:0/3521071491 conn(0x7f1310077570 msgr2=0x7f13100febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 shutdown_connections 2023-12-14T18:37:50.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.652+0000 7f131664d700 1 -- 172.21.15.31:0/3521071491 wait complete. 2023-12-14T18:37:50.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:37:50.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:37:50.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 -- 172.21.15.31:0/588455834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e001009e0 msgr2=0x7f9e00100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 --2- 172.21.15.31:0/588455834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e001009e0 0x7f9e00100e50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009990 tx=0x7f9de8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 -- 172.21.15.31:0/588455834 shutdown_connections 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 --2- 172.21.15.31:0/588455834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e00101390 0x7f9e0010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 --2- 172.21.15.31:0/588455834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e001051b0 0x7f9e00105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 --2- 172.21.15.31:0/588455834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e001009e0 0x7f9e00100e50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 -- 172.21.15.31:0/588455834 >> 172.21.15.31:0/588455834 conn(0x7f9e00077520 msgr2=0x7f9e00077920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 -- 172.21.15.31:0/588455834 shutdown_connections 2023-12-14T18:37:50.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.804+0000 7f9e066b1700 1 -- 172.21.15.31:0/588455834 wait complete. 2023-12-14T18:37:50.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.805+0000 7f9e066b1700 1 Processor -- start 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.805+0000 7f9e066b1700 1 -- start start 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.805+0000 7f9e066b1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e001009e0 0x7f9e00072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 0x7f9e00073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e001051b0 0x7f9e001a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e00073b60 con 0x7f9e00101390 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001a4b20 con 0x7f9e001009e0 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001a4c60 con 0x7f9e001051b0 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 0x7f9e00073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 0x7f9e00073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50180/0 (socket says 172.21.15.31:50180) 2023-12-14T18:37:50.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 -- 172.21.15.31:0/3192206104 learned_addr learned my addr 172.21.15.31:0/3192206104 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dfffff700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e001009e0 0x7f9e00072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e04c4e700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e001051b0 0x7f9e001a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 -- 172.21.15.31:0/3192206104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e001051b0 msgr2=0x7f9e001a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e001051b0 0x7f9e001a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 -- 172.21.15.31:0/3192206104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e001009e0 msgr2=0x7f9e00072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e001009e0 0x7f9e00072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 -- 172.21.15.31:0/3192206104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de80095f0 con 0x7f9e00101390 2023-12-14T18:37:50.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dff7fe700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 0x7f9e00073330 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098e0 tx=0x7f9de800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de801d070 con 0x7f9e00101390 2023-12-14T18:37:50.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9de8005050 con 0x7f9e00101390 2023-12-14T18:37:50.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e00110320 con 0x7f9e00101390 2023-12-14T18:37:50.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8004b30 con 0x7f9e00101390 2023-12-14T18:37:50.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.806+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e00110190 con 0x7f9e00101390 2023-12-14T18:37:50.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.807+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e001104b0 con 0x7f9e00101390 2023-12-14T18:37:50.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.809+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9de80051c0 con 0x7f9e00101390 2023-12-14T18:37:50.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.809+0000 7f9dfd7fa700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dec0a12c0 0x7f9dec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:50.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.810+0000 7f9dfffff700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dec0a12c0 0x7f9dec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:50.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.810+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f9de80ce060 con 0x7f9e00101390 2023-12-14T18:37:50.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.810+0000 7f9dfffff700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dec0a12c0 0x7f9dec0a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005950 tx=0x7f9df000b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:50.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:50.814+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9de8097110 con 0x7f9e00101390 2023-12-14T18:37:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.074+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e00067160 con 0x7f9e00101390 2023-12-14T18:37:51.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.074+0000 7f9dfd7fa700 1 -- 172.21.15.31:0/3192206104 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f9de809a730 con 0x7f9e00101390 2023-12-14T18:37:51.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:37:51.119 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:29.509045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:37:51.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dec0a12c0 msgr2=0x7f9dec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dec0a12c0 0x7f9dec0a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005950 tx=0x7f9df000b410 comp rx=0 tx=0).stop 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 msgr2=0x7f9e00073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 0x7f9e00073330 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9de80098e0 tx=0x7f9de800f670 comp rx=0 tx=0).stop 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 shutdown_connections 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f9dec0a12c0 0x7f9dec0a3770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e001051b0 0x7f9e001a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e001009e0 0x7f9e00072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 --2- 172.21.15.31:0/3192206104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e00101390 0x7f9e00073330 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 >> 172.21.15.31:0/3192206104 conn(0x7f9e00077520 msgr2=0x7f9e000ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 shutdown_connections 2023-12-14T18:37:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.077+0000 7f9e066b1700 1 -- 172.21.15.31:0/3192206104 wait complete. 2023-12-14T18:37:51.122 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2023-12-14T18:37:51.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.261+0000 7f832d25b700 1 -- 172.21.15.31:0/3126138667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8328104410 msgr2=0x7f832810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.261+0000 7f832d25b700 1 --2- 172.21.15.31:0/3126138667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8328104410 0x7f832810bfe0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f832000b0d0 tx=0x7f832000b3e0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 -- 172.21.15.31:0/3126138667 shutdown_connections 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 --2- 172.21.15.31:0/3126138667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8328104410 0x7f832810bfe0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 --2- 172.21.15.31:0/3126138667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 0x7f83280ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 --2- 172.21.15.31:0/3126138667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83280ffd70 0x7f8328103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 -- 172.21.15.31:0/3126138667 >> 172.21.15.31:0/3126138667 conn(0x7f8328076000 msgr2=0x7f8328076400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 -- 172.21.15.31:0/3126138667 shutdown_connections 2023-12-14T18:37:51.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 -- 172.21.15.31:0/3126138667 wait complete. 2023-12-14T18:37:51.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.262+0000 7f832d25b700 1 Processor -- start 2023-12-14T18:37:51.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 -- start start 2023-12-14T18:37:51.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 0x7f8328100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 0x7f8328101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8328104410 0x7f83281a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8328101b40 con 0x7f83280ffd70 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83281a4c20 con 0x7f8328104410 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832d25b700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83281a4d60 con 0x7f83280ff3d0 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 0x7f8328101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 0x7f8328101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50196/0 (socket says 172.21.15.31:50196) 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 -- 172.21.15.31:0/367766796 learned_addr learned my addr 172.21.15.31:0/367766796 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832759e700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8328104410 0x7f83281a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f8326d9d700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 0x7f8328100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 -- 172.21.15.31:0/367766796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 msgr2=0x7f8328100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 0x7f8328100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 -- 172.21.15.31:0/367766796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8328104410 msgr2=0x7f83281a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.263+0000 7f832659c700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8328104410 0x7f83281a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f832659c700 1 -- 172.21.15.31:0/367766796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320009d00 con 0x7f83280ffd70 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f832759e700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8328104410 0x7f83281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f8326d9d700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 0x7f8328100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f832659c700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 0x7f8328101310 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e940 tx=0x7f831c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f830ffff700 1 -- 172.21.15.31:0/367766796 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c009730 con 0x7f83280ffd70 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832810f230 con 0x7f83280ffd70 2023-12-14T18:37:51.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f830ffff700 1 -- 172.21.15.31:0/367766796 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831c010430 con 0x7f83280ffd70 2023-12-14T18:37:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f830ffff700 1 -- 172.21.15.31:0/367766796 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00cea0 con 0x7f83280ffd70 2023-12-14T18:37:51.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.264+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832810f0a0 con 0x7f83280ffd70 2023-12-14T18:37:51.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.267+0000 7f830ffff700 1 -- 172.21.15.31:0/367766796 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f831c010810 con 0x7f83280ffd70 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.267+0000 7f830ffff700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f83100a12c0 0x7f83100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.268+0000 7f8326d9d700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f83100a12c0 0x7f83100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.268+0000 7f830ffff700 1 -- 172.21.15.31:0/367766796 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f831c014070 con 0x7f83280ffd70 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.268+0000 7f8326d9d700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f83100a12c0 0x7f83100a3770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8318011490 tx=0x7f8318009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.268+0000 7f83393e9b80 1 Processor -- start 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.268+0000 7f83393e9b80 1 -- start start 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6380 0x55bb2f0b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f83393e9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2efa05e0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f83393e9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f00f870 con 0x55bb2f0b6380 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f83393e9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f008d00 con 0x55bb2f0b6c90 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50206/0 (socket says 172.21.15.31:50206) 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 -- 172.21.15.31:0/2846835033 learned_addr learned my addr 172.21.15.31:0/2846835033 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832659c700 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f8326d9d700 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6380 0x55bb2f0b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 -- 172.21.15.31:0/2846835033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 msgr2=0x55bb2f0c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 -- 172.21.15.31:0/2846835033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6380 msgr2=0x55bb2f0b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6380 0x55bb2f0b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 -- 172.21.15.31:0/2846835033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2ee8a3b0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832659c700 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f832759e700 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c3f30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8320015000 tx=0x7f8320009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a3fa6c5e8e605d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f830d7fa700 1 -- 172.21.15.31:0/2846835033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000e040 con 0x55bb2f0c1b50 2023-12-14T18:37:51.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f830d7fa700 1 -- 172.21.15.31:0/2846835033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83200074e0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.269+0000 7f830d7fa700 1 -- 172.21.15.31:0/2846835033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8320003ea0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 -- 172.21.15.31:0/2846835033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 msgr2=0x55bb2f0c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c3f30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8320015000 tx=0x7f8320009ba0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 -- 172.21.15.31:0/2846835033 shutdown_connections 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c3f30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6380 0x55bb2f0b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2846835033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 -- 172.21.15.31:0/2846835033 >> 172.21.15.31:0/2846835033 conn(0x55bb2f0acca0 msgr2=0x55bb2f0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 -- 172.21.15.31:0/2846835033 shutdown_connections 2023-12-14T18:37:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.270+0000 7f83393e9b80 1 -- 172.21.15.31:0/2846835033 wait complete. 2023-12-14T18:37:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 -- start start 2023-12-14T18:37:51.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 Processor -- start 2023-12-14T18:37:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0be710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2efa4f90 con 0x55bb2f0c1b50 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2efa4c30 con 0x55bb2f0b6c90 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2ee8c6a0 con 0x55bb2f0b6380 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f83393e9b80 10 client.0 _ll_register_callbacks cb 0x55bb2efc8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f832759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f832759e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0bec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50218/0 (socket says 172.21.15.31:50218) 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f832759e700 1 -- 172.21.15.31:0/25755251 learned_addr learned my addr 172.21.15.31:0/25755251 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f8326d9d700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.271+0000 7f832659c700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0be710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f832759e700 1 -- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6380 msgr2=0x55bb2f0c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f832759e700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f832759e700 1 -- 172.21.15.31:0/25755251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 msgr2=0x55bb2f0be710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f832759e700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0be710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f832759e700 1 -- 172.21.15.31:0/25755251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832810c820 con 0x55bb2f0c1b50 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f832759e700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0bec50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f832000b770 tx=0x7f832000bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb3e07cebe14977 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f830aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:51.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f830aff5700 1 -- 172.21.15.31:0/25755251 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832000e070 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f830aff5700 1 -- 172.21.15.31:0/25755251 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f83200050d0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb2f0aa2b0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f830aff5700 1 -- 172.21.15.31:0/25755251 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8320009490 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f83393e9b80 10 client.17676 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.272+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bb2efcf170 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f830aff5700 1 -- 172.21.15.31:0/25755251 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb2f0aa2b0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f830aff5700 1 -- 172.21.15.31:0/25755251 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f832000b100 con 0x55bb2f0c1b50 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f83393e9b80 10 client.17676 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:51.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f83393e9b80 10 client.17676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f83393e9b80 20 client.17676 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f83393e9b80 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f0d1090 0x55bb2f0d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f83393e9b80 4 client.17676 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb2f001ce0 con 0x55bb2f0d1090 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f8326d9d700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f0d1090 0x55bb2f0d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f8326d9d700 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f0d1090 0x55bb2f0d3470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db94d0fc1ff91037 server_cookie=dbea047c51f4738e in_seq=0 out_seq=0 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.273+0000 7f830aff5700 10 client.17676 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f830aff5700 1 -- 172.21.15.31:0/25755251 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb2f001ce0 con 0x55bb2f0d1090 2023-12-14T18:37:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f830aff5700 10 client.17676 handle_command_reply: tid=0 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 client.17676 shutdown 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 msgr2=0x55bb2f0bec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0bec50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f832000b770 tx=0x7f832000bcd0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 shutdown_connections 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f0d1090 0x55bb2f0d3470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0bec50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f830aff5700 0 client.17676 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:51.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 --2- 172.21.15.31:0/25755251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0be710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 >> 172.21.15.31:0/25755251 conn(0x55bb2f0acca0 msgr2=0x55bb2f0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 shutdown_connections 2023-12-14T18:37:51.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.274+0000 7f83393e9b80 1 -- 172.21.15.31:0/25755251 wait complete. 2023-12-14T18:37:51.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.275+0000 7f83393e9b80 20 client.17676 trim_cache size 0 max 16384 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 Processor -- start 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 -- start start 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f0ba5d0 con 0x55bb2f0b6380 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f0b94b0 con 0x55bb2f0c1b50 2023-12-14T18:37:51.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.288+0000 7f83393e9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f0ae210 con 0x55bb2f0b6c90 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50234/0 (socket says 172.21.15.31:50234) 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 -- 172.21.15.31:0/2473030242 learned_addr learned my addr 172.21.15.31:0/2473030242 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f832759e700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f832659c700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 -- 172.21.15.31:0/2473030242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 msgr2=0x55bb2f0c0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 -- 172.21.15.31:0/2473030242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0c1b50 msgr2=0x55bb2f0c1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 -- 172.21.15.31:0/2473030242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb2efceec0 con 0x55bb2f0b6380 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f832759e700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f832659c700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f8326d9d700 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c3f30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8318013010 tx=0x7f8318014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba9bd091ad013f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f830dffb700 1 -- 172.21.15.31:0/2473030242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318024040 con 0x55bb2f0b6380 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f830dffb700 1 -- 172.21.15.31:0/2473030242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831802ae10 con 0x55bb2f0b6380 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.289+0000 7f830dffb700 1 -- 172.21.15.31:0/2473030242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831801f660 con 0x55bb2f0b6380 2023-12-14T18:37:51.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 -- 172.21.15.31:0/2473030242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 msgr2=0x55bb2f0c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c3f30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8318013010 tx=0x7f8318014000 comp rx=0 tx=0).stop 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 -- 172.21.15.31:0/2473030242 shutdown_connections 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c3f30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 --2- 172.21.15.31:0/2473030242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 -- 172.21.15.31:0/2473030242 >> 172.21.15.31:0/2473030242 conn(0x55bb2f0acca0 msgr2=0x55bb2f0cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 -- 172.21.15.31:0/2473030242 shutdown_connections 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 -- 172.21.15.31:0/2473030242 wait complete. 2023-12-14T18:37:51.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 -- start start 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.290+0000 7f83393e9b80 1 Processor -- start 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f0c5220 con 0x55bb2f0b6380 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f0c5390 con 0x55bb2f0b6c90 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb2f0c5550 con 0x55bb2f0c1b50 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f83393e9b80 10 client.0 _ll_register_callbacks cb 0x7f8320009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:51.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50236/0 (socket says 172.21.15.31:50236) 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 -- 172.21.15.31:0/3730623900 learned_addr learned my addr 172.21.15.31:0/3730623900 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f832659c700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f832759e700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 -- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0c1b50 msgr2=0x55bb2f0c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 -- 172.21.15.31:0/3730623900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 msgr2=0x55bb2f0c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 -- 172.21.15.31:0/3730623900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8318017070 con 0x55bb2f0b6380 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f832759e700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f832659c700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f8326d9d700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c8460 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8318005c30 tx=0x7f831802a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9630ba77d983b44a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f830aff5700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.291+0000 7f830aff5700 1 -- 172.21.15.31:0/3730623900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8318024050 con 0x55bb2f0b6380 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f830aff5700 1 -- 172.21.15.31:0/3730623900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f8318004750 con 0x55bb2f0b6380 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83200074e0 con 0x55bb2f0b6380 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f830aff5700 1 -- 172.21.15.31:0/3730623900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8318004950 con 0x55bb2f0b6380 2023-12-14T18:37:51.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 10 client.17682 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bb2efb59a0 con 0x55bb2f0b6380 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f830aff5700 1 -- 172.21.15.31:0/3730623900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83200074e0 con 0x55bb2f0b6380 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f830aff5700 1 -- 172.21.15.31:0/3730623900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8318031070 con 0x55bb2f0b6380 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 10 client.17682 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 10 client.17682 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 20 client.17682 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f1262a0 0x55bb2f1286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 4 client.17682 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:37:51.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb2f001a00 con 0x55bb2f1262a0 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.292+0000 7f832659c700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f1262a0 0x55bb2f1286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f832659c700 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f1262a0 0x55bb2f1286b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d01c10d284bc3c0 server_cookie=132268b123852b71 in_seq=0 out_seq=0 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f830aff5700 10 client.17682 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f830aff5700 1 -- 172.21.15.31:0/3730623900 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb2f001a00 con 0x55bb2f1262a0 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f830aff5700 10 client.17682 handle_command_reply: tid=0 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 client.17682 shutdown 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 msgr2=0x55bb2f0c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c8460 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8318005c30 tx=0x7f831802a6e0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 shutdown_connections 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55bb2f1262a0 0x55bb2f1286b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bb2f0c1b50 0x55bb2f0c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bb2f0b6380 0x55bb2f0c8460 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 --2- 172.21.15.31:0/3730623900 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55bb2f0b6c90 0x55bb2f0c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 >> 172.21.15.31:0/3730623900 conn(0x55bb2f0acca0 msgr2=0x55bb2f0be6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f830aff5700 0 client.17682 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 shutdown_connections 2023-12-14T18:37:51.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 1 -- 172.21.15.31:0/3730623900 wait complete. 2023-12-14T18:37:51.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.293+0000 7f83393e9b80 20 client.17682 trim_cache size 0 max 16384 2023-12-14T18:37:51.306 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:37:51.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f83100a12c0 msgr2=0x7f83100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f83100a12c0 0x7f83100a3770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8318011490 tx=0x7f8318009260 comp rx=0 tx=0).stop 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 msgr2=0x7f8328101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 0x7f8328101310 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e940 tx=0x7f831c00ec50 comp rx=0 tx=0).stop 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 shutdown_connections 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8328104410 0x7f83281a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f83100a12c0 0x7f83100a3770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83280ff3d0 0x7f8328100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 --2- 172.21.15.31:0/367766796 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83280ffd70 0x7f8328101310 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 >> 172.21.15.31:0/367766796 conn(0x7f8328076000 msgr2=0x7f8328068fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 shutdown_connections 2023-12-14T18:37:51.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.294+0000 7f832d25b700 1 -- 172.21.15.31:0/367766796 wait complete. 2023-12-14T18:37:51.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:37:51.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 -- 172.21.15.31:0/833880874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c10cbe0 msgr2=0x7fe81c10efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 --2- 172.21.15.31:0/833880874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c10cbe0 0x7fe81c10efc0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe814009990 tx=0x7fe814009ca0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 -- 172.21.15.31:0/833880874 shutdown_connections 2023-12-14T18:37:51.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 --2- 172.21.15.31:0/833880874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c10cbe0 0x7fe81c10efc0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 --2- 172.21.15.31:0/833880874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe81c075e70 0x7fe81c0ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 --2- 172.21.15.31:0/833880874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c0ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 -- 172.21.15.31:0/833880874 >> 172.21.15.31:0/833880874 conn(0x7fe81c074a60 msgr2=0x7fe81c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 -- 172.21.15.31:0/833880874 shutdown_connections 2023-12-14T18:37:51.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.449+0000 7fe8238c1700 1 -- 172.21.15.31:0/833880874 wait complete. 2023-12-14T18:37:51.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 Processor -- start 2023-12-14T18:37:51.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 -- start start 2023-12-14T18:37:51.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe81c075e70 0x7fe81c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c10cbe0 0x7fe81c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c19d410 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c19d580 con 0x7fe81c075e70 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe8238c1700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c19d740 con 0x7fe81c10cbe0 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe820e5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe820e5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50248/0 (socket says 172.21.15.31:50248) 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe820e5c700 1 -- 172.21.15.31:0/1536396959 learned_addr learned my addr 172.21.15.31:0/1536396959 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe821e5e700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c10cbe0 0x7fe81c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.450+0000 7fe82165d700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe81c075e70 0x7fe81c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe820e5c700 1 -- 172.21.15.31:0/1536396959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c10cbe0 msgr2=0x7fe81c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c10cbe0 0x7fe81c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe820e5c700 1 -- 172.21.15.31:0/1536396959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe81c075e70 msgr2=0x7fe81c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe81c075e70 0x7fe81c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe820e5c700 1 -- 172.21.15.31:0/1536396959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140095f0 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c198a10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe80800d950 tx=0x7fe80800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe81a7fc700 1 -- 172.21.15.31:0/1536396959 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800f890 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe81a7fc700 1 -- 172.21.15.31:0/1536396959 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe808010460 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c112440 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe81a7fc700 1 -- 172.21.15.31:0/1536396959 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80800e610 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.451+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1122b0 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.454+0000 7fe81a7fc700 1 -- 172.21.15.31:0/1536396959 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe8080108d0 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.454+0000 7fe81a7fc700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe80c0a1340 0x7fe80c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.454+0000 7fe81a7fc700 1 -- 172.21.15.31:0/1536396959 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe8080cce50 con 0x7fe81c0ff9b0 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.454+0000 7fe82165d700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe80c0a1340 0x7fe80c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82165d700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe80c0a1340 0x7fe80c0a37f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe81000f540 tx=0x7fe81000b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 Processor -- start 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 -- start start 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a901f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a7de5e0 con 0x55f23a8f4c90 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a84d870 con 0x55f23a8ffb50 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.455+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a846d00 con 0x55f23a8f4380 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50256/0 (socket says 172.21.15.31:50256) 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 -- 172.21.15.31:0/1734092340 learned_addr learned my addr 172.21.15.31:0/1734092340 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe82165d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe821e5e700 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a901f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 -- 172.21.15.31:0/1734092340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 msgr2=0x55f23a8f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 -- 172.21.15.31:0/1734092340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 msgr2=0x55f23a901f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a901f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 -- 172.21.15.31:0/1734092340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f23a6c83b0 con 0x55f23a8f4c90 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe82165d700 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8ff610 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe808096030 tx=0x7fe808099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6827cd10799cfd0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe8077fe700 1 -- 172.21.15.31:0/1734092340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808096240 con 0x55f23a8f4c90 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe8077fe700 1 -- 172.21.15.31:0/1734092340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8080963a0 con 0x55f23a8f4c90 2023-12-14T18:37:51.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.456+0000 7fe8077fe700 1 -- 172.21.15.31:0/1734092340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8080a1af0 con 0x55f23a8f4c90 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1734092340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 msgr2=0x55f23a8ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8ff610 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe808096030 tx=0x7fe808099bb0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1734092340 shutdown_connections 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a901f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1734092340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8ff610 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1734092340 >> 172.21.15.31:0/1734092340 conn(0x55f23a8eaca0 msgr2=0x55f23a8f07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1734092340 shutdown_connections 2023-12-14T18:37:51.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1734092340 wait complete. 2023-12-14T18:37:51.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.457+0000 7fe82fa4fb80 1 -- start start 2023-12-14T18:37:51.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 Processor -- start 2023-12-14T18:37:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a8fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a7e2f90 con 0x55f23a8f4c90 2023-12-14T18:37:51.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a7e2c30 con 0x55f23a8ffb50 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a6ca6a0 con 0x55f23a8f4380 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82fa4fb80 10 client.0 _ll_register_callbacks cb 0x55f23a806910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe820e5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe820e5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50272/0 (socket says 172.21.15.31:50272) 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe820e5c700 1 -- 172.21.15.31:0/1720354223 learned_addr learned my addr 172.21.15.31:0/1720354223 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe82165d700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.458+0000 7fe821e5e700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a8fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe820e5c700 1 -- 172.21.15.31:0/1720354223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 msgr2=0x55f23a8ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe820e5c700 1 -- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 msgr2=0x55f23a8fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a8fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe820e5c700 1 -- 172.21.15.31:0/1720354223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1122b0 con 0x55f23a8f4c90 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe820e5c700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fc7c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe8080ca730 tx=0x7fe808099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553339c557602195 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe804ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe804ff9700 1 -- 172.21.15.31:0/1720354223 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808096240 con 0x55f23a8f4c90 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe804ff9700 1 -- 172.21.15.31:0/1720354223 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe8080b22e0 con 0x55f23a8f4c90 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f23a8e82b0 con 0x55f23a8f4c90 2023-12-14T18:37:51.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe804ff9700 1 -- 172.21.15.31:0/1720354223 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe808099020 con 0x55f23a8f4c90 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe82fa4fb80 10 client.17694 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f23a80d170 con 0x55f23a8f4c90 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe804ff9700 1 -- 172.21.15.31:0/1720354223 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f23a8e82b0 con 0x55f23a8f4c90 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.459+0000 7fe804ff9700 1 -- 172.21.15.31:0/1720354223 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe808098a90 con 0x55f23a8f4c90 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82fa4fb80 10 client.17694 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82fa4fb80 10 client.17694 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82fa4fb80 20 client.17694 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a90eea0 0x55f23a911280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82fa4fb80 4 client.17694 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f23a83fce0 con 0x55f23a90eea0 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82165d700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a90eea0 0x55f23a911280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe804ff9700 10 client.17694 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.460+0000 7fe82165d700 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a90eea0 0x55f23a911280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=838b039e290b3825 server_cookie=ed82f3522a611177 in_seq=0 out_seq=0 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe804ff9700 1 -- 172.21.15.31:0/1720354223 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f23a83fce0 con 0x55f23a90eea0 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe804ff9700 10 client.17694 handle_command_reply: tid=0 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 client.17694 shutdown 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 msgr2=0x55f23a8fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fc7c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe8080ca730 tx=0x7fe808099fc0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 shutdown_connections 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8ffb50 0x55f23a8fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a90eea0 0x55f23a911280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4380 0x55f23a8ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe804ff9700 0 client.17694 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/1720354223 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fc7c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 >> 172.21.15.31:0/1720354223 conn(0x55f23a8eaca0 msgr2=0x55f23a8f0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 shutdown_connections 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.461+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/1720354223 wait complete. 2023-12-14T18:37:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.462+0000 7fe82fa4fb80 20 client.17694 trim_cache size 0 max 16384 2023-12-14T18:37:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 Processor -- start 2023-12-14T18:37:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 -- start start 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8eaca0 0x55f23a90b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a8fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a8f5d70 con 0x55f23a8f4c90 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a8f83e0 con 0x55f23a8f4380 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a90bc40 con 0x55f23a8eaca0 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.475+0000 7fe821e5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50284/0 (socket says 172.21.15.31:50284) 2023-12-14T18:37:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 -- 172.21.15.31:0/2965731903 learned_addr learned my addr 172.21.15.31:0/2965731903 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe82165d700 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8eaca0 0x55f23a90b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe820e5c700 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a8fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 -- 172.21.15.31:0/2965731903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8eaca0 msgr2=0x55f23a90b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8eaca0 0x55f23a90b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 -- 172.21.15.31:0/2965731903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 msgr2=0x55f23a8fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a8fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 -- 172.21.15.31:0/2965731903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f23a80cec0 con 0x55f23a8f4c90 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe820e5c700 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a8fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe821e5e700 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fbfe0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe8140098e0 tx=0x7fe814005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba1e830969ef1a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe807fff700 1 -- 172.21.15.31:0/2965731903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81401d070 con 0x55f23a8f4c90 2023-12-14T18:37:51.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe807fff700 1 -- 172.21.15.31:0/2965731903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe81401e040 con 0x55f23a8f4c90 2023-12-14T18:37:51.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.476+0000 7fe807fff700 1 -- 172.21.15.31:0/2965731903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814004920 con 0x55f23a8f4c90 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/2965731903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 msgr2=0x55f23a8fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fbfe0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe8140098e0 tx=0x7fe814005960 comp rx=0 tx=0).stop 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/2965731903 shutdown_connections 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8f4c90 0x55f23a8fbfe0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8eaca0 0x55f23a90b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/2965731903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a8fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/2965731903 >> 172.21.15.31:0/2965731903 conn(0x55f23a8f5390 msgr2=0x55f23a8eef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/2965731903 shutdown_connections 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/2965731903 wait complete. 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 -- start start 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.477+0000 7fe82fa4fb80 1 Processor -- start 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 0x55f23a906510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a904960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4c90 0x55f23a904ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a905580 con 0x55f23a8eaca0 2023-12-14T18:37:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a9056f0 con 0x55f23a8f4380 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23a962c70 con 0x55f23a8f4c90 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82fa4fb80 10 client.0 _ll_register_callbacks cb 0x7fe8100097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 0x55f23a906510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 0x55f23a906510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50286/0 (socket says 172.21.15.31:50286) 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 -- 172.21.15.31:0/12897385 learned_addr learned my addr 172.21.15.31:0/12897385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe820e5c700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a904960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe821e5e700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4c90 0x55f23a904ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 -- 172.21.15.31:0/12897385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4c90 msgr2=0x55f23a904ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4c90 0x55f23a904ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 -- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 msgr2=0x55f23a904960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a904960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe82165d700 1 -- 172.21.15.31:0/12897385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c10faf0 con 0x55f23a8eaca0 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe820e5c700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a904960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.478+0000 7fe821e5e700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4c90 0x55f23a904ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82165d700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 0x55f23a906510 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b010 tx=0x7fe810004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb46aaebcbf283b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe804ff9700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe804ff9700 1 -- 172.21.15.31:0/12897385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81002a040 con 0x55f23a8eaca0 2023-12-14T18:37:51.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe804ff9700 1 -- 172.21.15.31:0/12897385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fe810004390 con 0x55f23a8eaca0 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8080963a0 con 0x55f23a8eaca0 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe804ff9700 1 -- 172.21.15.31:0/12897385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe81000e040 con 0x55f23a8eaca0 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 10 client.17700 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f23a7f39a0 con 0x55f23a8eaca0 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe804ff9700 1 -- 172.21.15.31:0/12897385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8080963a0 con 0x55f23a8eaca0 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe804ff9700 1 -- 172.21.15.31:0/12897385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe810015ac0 con 0x55f23a8eaca0 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 10 client.17700 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 10 client.17700 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 20 client.17700 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a8ffb50 0x55f23a966320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 4 client.17700 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.479+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f23a83fa00 con 0x55f23a8ffb50 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe820e5c700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a8ffb50 0x55f23a966320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe820e5c700 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a8ffb50 0x55f23a966320 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=813b31b7258f5400 server_cookie=68297a0b5573e39 in_seq=0 out_seq=0 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe804ff9700 10 client.17700 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe804ff9700 1 -- 172.21.15.31:0/12897385 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f23a83fa00 con 0x55f23a8ffb50 2023-12-14T18:37:51.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe804ff9700 10 client.17700 handle_command_reply: tid=0 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 client.17700 shutdown 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 msgr2=0x55f23a906510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 0x55f23a906510 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b010 tx=0x7fe810004b40 comp rx=0 tx=0).stop 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 shutdown_connections 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x55f23a8ffb50 0x55f23a966320 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f23a8f4c90 0x55f23a904ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55f23a8eaca0 0x55f23a906510 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe804ff9700 0 client.17700 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 --2- 172.21.15.31:0/12897385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55f23a8f4380 0x55f23a904960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.480+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 >> 172.21.15.31:0/12897385 conn(0x55f23a8f5390 msgr2=0x55f23a8ef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 shutdown_connections 2023-12-14T18:37:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe82fa4fb80 1 -- 172.21.15.31:0/12897385 wait complete. 2023-12-14T18:37:51.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe82fa4fb80 20 client.17700 trim_cache size 0 max 16384 2023-12-14T18:37:51.492 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:37:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe80c0a1340 msgr2=0x7fe80c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe80c0a1340 0x7fe80c0a37f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe81000f540 tx=0x7fe81000b040 comp rx=0 tx=0).stop 2023-12-14T18:37:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 msgr2=0x7fe81c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c198a10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe80800d950 tx=0x7fe80800dc60 comp rx=0 tx=0).stop 2023-12-14T18:37:51.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 shutdown_connections 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe80c0a1340 0x7fe80c0a37f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c10cbe0 0x7fe81c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe81c075e70 0x7fe81c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 --2- 172.21.15.31:0/1536396959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe81c0ff9b0 0x7fe81c198a10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 >> 172.21.15.31:0/1536396959 conn(0x7fe81c074a60 msgr2=0x7fe81c10cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 shutdown_connections 2023-12-14T18:37:51.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.481+0000 7fe8238c1700 1 -- 172.21.15.31:0/1536396959 wait complete. 2023-12-14T18:37:51.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:37:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 -- 172.21.15.31:0/3170376647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe14100c50 msgr2=0x7efe14101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 --2- 172.21.15.31:0/3170376647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe14100c50 0x7efe14101020 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0099c0 tx=0x7efdfc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 -- 172.21.15.31:0/3170376647 shutdown_connections 2023-12-14T18:37:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 --2- 172.21.15.31:0/3170376647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe14105550 0x7efe1410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 --2- 172.21.15.31:0/3170376647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe14100c50 0x7efe14101020 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 --2- 172.21.15.31:0/3170376647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe141015f0 0x7efe14104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 -- 172.21.15.31:0/3170376647 >> 172.21.15.31:0/3170376647 conn(0x7efe14077570 msgr2=0x7efe14077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 -- 172.21.15.31:0/3170376647 shutdown_connections 2023-12-14T18:37:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.633+0000 7efe19348700 1 -- 172.21.15.31:0/3170376647 wait complete. 2023-12-14T18:37:51.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 Processor -- start 2023-12-14T18:37:51.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 -- start start 2023-12-14T18:37:51.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe14100c50 0x7efe14196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 0x7efe141969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe14105550 0x7efe1419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419b440 con 0x7efe141015f0 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419b5b0 con 0x7efe14100c50 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe19348700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419b770 con 0x7efe14105550 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 0x7efe141969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 0x7efe141969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50302/0 (socket says 172.21.15.31:50302) 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.634+0000 7efe127fc700 1 -- 172.21.15.31:0/601275891 learned_addr learned my addr 172.21.15.31:0/601275891 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe12ffd700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe14100c50 0x7efe14196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe137fe700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe14105550 0x7efe1419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe127fc700 1 -- 172.21.15.31:0/601275891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe14105550 msgr2=0x7efe1419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe127fc700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe14105550 0x7efe1419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe127fc700 1 -- 172.21.15.31:0/601275891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe14100c50 msgr2=0x7efe14196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe127fc700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe14100c50 0x7efe14196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe127fc700 1 -- 172.21.15.31:0/601275891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0095f0 con 0x7efe141015f0 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe127fc700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 0x7efe141969a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d980 tx=0x7efe0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe0bfff700 1 -- 172.21.15.31:0/601275891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400f890 con 0x7efe141015f0 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe141102c0 con 0x7efe141015f0 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe0bfff700 1 -- 172.21.15.31:0/601275891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe04010460 con 0x7efe141015f0 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe0bfff700 1 -- 172.21.15.31:0/601275891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400e610 con 0x7efe141015f0 2023-12-14T18:37:51.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.635+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14110130 con 0x7efe141015f0 2023-12-14T18:37:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.638+0000 7efe0bfff700 1 -- 172.21.15.31:0/601275891 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efe040105d0 con 0x7efe141015f0 2023-12-14T18:37:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.638+0000 7efe0bfff700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efe000a12c0 0x7efe000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.638+0000 7efe0bfff700 1 -- 172.21.15.31:0/601275891 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7efe040ccbf0 con 0x7efe141015f0 2023-12-14T18:37:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.638+0000 7efe12ffd700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efe000a12c0 0x7efe000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.639+0000 7efe12ffd700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efe000a12c0 0x7efe000a3770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c00 tx=0x7efdfc003680 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.639+0000 7efe254d6b80 1 Processor -- start 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.639+0000 7efe254d6b80 1 -- start start 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46380 0x55c33cb46750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb51610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb53f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe254d6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33ca305e0 con 0x55c33cb51b50 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe254d6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33ca9f870 con 0x55c33cb46380 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe254d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33ca98d00 con 0x55c33cb46c90 2023-12-14T18:37:51.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb53f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb53f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50304/0 (socket says 172.21.15.31:50304) 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 -- 172.21.15.31:0/3749662816 learned_addr learned my addr 172.21.15.31:0/3749662816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe12ffd700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46380 0x55c33cb46750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe127fc700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb51610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 -- 172.21.15.31:0/3749662816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 msgr2=0x55c33cb51610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb51610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 -- 172.21.15.31:0/3749662816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46380 msgr2=0x55c33cb46750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46380 0x55c33cb46750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 -- 172.21.15.31:0/3749662816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c33c91a3b0 con 0x55c33cb51b50 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe12ffd700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46380 0x55c33cb46750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe127fc700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb51610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe137fe700 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb53f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efe0c003fa0 tx=0x7efe0c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bdb82068109658 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.640+0000 7efe097fa700 1 -- 172.21.15.31:0/3749662816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c019070 con 0x55c33cb51b50 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe097fa700 1 -- 172.21.15.31:0/3749662816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0c00f040 con 0x55c33cb51b50 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe097fa700 1 -- 172.21.15.31:0/3749662816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c007b60 con 0x55c33cb51b50 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 -- 172.21.15.31:0/3749662816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 msgr2=0x55c33cb53f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb53f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efe0c003fa0 tx=0x7efe0c009ba0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 -- 172.21.15.31:0/3749662816 shutdown_connections 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb53f30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46380 0x55c33cb46750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 --2- 172.21.15.31:0/3749662816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb51610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 -- 172.21.15.31:0/3749662816 >> 172.21.15.31:0/3749662816 conn(0x55c33cb3cca0 msgr2=0x55c33cb427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 -- 172.21.15.31:0/3749662816 shutdown_connections 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.641+0000 7efe254d6b80 1 -- 172.21.15.31:0/3749662816 wait complete. 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 -- start start 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 Processor -- start 2023-12-14T18:37:51.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46c90 0x55c33cb4e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb4ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33ca34f90 con 0x55c33cb51b50 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33ca34c30 con 0x55c33cb46c90 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33c91c6a0 con 0x55c33cb46380 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe254d6b80 10 client.0 _ll_register_callbacks cb 0x55c33ca58910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb4ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe12ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe12ffd700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:47560/0 (socket says 172.21.15.31:47560) 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe12ffd700 1 -- 172.21.15.31:0/1444791907 learned_addr learned my addr 172.21.15.31:0/1444791907 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb4ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50314/0 (socket says 172.21.15.31:50314) 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.642+0000 7efe127fc700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46c90 0x55c33cb4e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe137fe700 1 -- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 msgr2=0x55c33cb514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe137fe700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe137fe700 1 -- 172.21.15.31:0/1444791907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46c90 msgr2=0x55c33cb4e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe137fe700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46c90 0x55c33cb4e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe137fe700 1 -- 172.21.15.31:0/1444791907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c003bd0 con 0x55c33cb51b50 2023-12-14T18:37:51.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe12ffd700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe137fe700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb4ed20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000f80 tx=0x7efe0c004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c9cfe1b9ba2d37 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efdfa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efdfa7fc700 1 -- 172.21.15.31:0/1444791907 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c023070 con 0x55c33cb51b50 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efdfa7fc700 1 -- 172.21.15.31:0/1444791907 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7efe0c009460 con 0x55c33cb51b50 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c33cb3a2b0 con 0x55c33cb51b50 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efdfa7fc700 1 -- 172.21.15.31:0/1444791907 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe0c019040 con 0x55c33cb51b50 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe254d6b80 10 client.17712 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.643+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c33ca5f170 con 0x55c33cb51b50 2023-12-14T18:37:51.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efdfa7fc700 1 -- 172.21.15.31:0/1444791907 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c33cb3a2b0 con 0x55c33cb51b50 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efdfa7fc700 1 -- 172.21.15.31:0/1444791907 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe0c026020 con 0x55c33cb51b50 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe254d6b80 10 client.17712 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe254d6b80 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe254d6b80 20 client.17712 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb613a0 0x55c33cb63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe254d6b80 4 client.17712 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c33ca91ce0 con 0x55c33cb613a0 2023-12-14T18:37:51.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.644+0000 7efe12ffd700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb613a0 0x55c33cb63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.645+0000 7efe12ffd700 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb613a0 0x55c33cb63780 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397eea049071f96 server_cookie=4df176f997af672b in_seq=0 out_seq=0 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.645+0000 7efdfa7fc700 10 client.17712 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.645+0000 7efdfa7fc700 1 -- 172.21.15.31:0/1444791907 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c33ca91ce0 con 0x55c33cb613a0 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.645+0000 7efdfa7fc700 10 client.17712 handle_command_reply: tid=0 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.645+0000 7efe254d6b80 1 client.17712 shutdown 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 msgr2=0x55c33cb4ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb4ed20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000f80 tx=0x7efe0c004780 comp rx=0 tx=0).stop 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 shutdown_connections 2023-12-14T18:37:51.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb613a0 0x55c33cb63780 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb51b50 0x55c33cb4ed20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1444791907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb46c90 0x55c33cb4e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efdfa7fc700 0 client.17712 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 >> 172.21.15.31:0/1444791907 conn(0x55c33cb3cca0 msgr2=0x55c33cb423d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 shutdown_connections 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 1 -- 172.21.15.31:0/1444791907 wait complete. 2023-12-14T18:37:51.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.646+0000 7efe254d6b80 20 client.17712 trim_cache size 0 max 16384 2023-12-14T18:37:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.660+0000 7efe254d6b80 1 Processor -- start 2023-12-14T18:37:51.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.660+0000 7efe254d6b80 1 -- start start 2023-12-14T18:37:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb58680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 0x55c33cb50f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe254d6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33cb4a690 con 0x55c33cb46380 2023-12-14T18:37:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe254d6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33cb47d60 con 0x55c33cb3cca0 2023-12-14T18:37:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe254d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33cb4f030 con 0x55c33cb46c90 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 0x55c33cb50f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 0x55c33cb50f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50330/0 (socket says 172.21.15.31:50330) 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 -- 172.21.15.31:0/1894726636 learned_addr learned my addr 172.21.15.31:0/1894726636 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe137fe700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe12ffd700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb58680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 -- 172.21.15.31:0/1894726636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 msgr2=0x55c33cb514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 -- 172.21.15.31:0/1894726636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 msgr2=0x55c33cb58680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb58680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 -- 172.21.15.31:0/1894726636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c33ca5eec0 con 0x55c33cb46380 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe137fe700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe12ffd700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.661+0000 7efe127fc700 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 0x55c33cb50f90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efe040ca550 tx=0x7efe04099230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e5f1cfa73a501 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe09ffb700 1 -- 172.21.15.31:0/1894726636 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040a1d20 con 0x55c33cb46380 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe09ffb700 1 -- 172.21.15.31:0/1894726636 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe040108b0 con 0x55c33cb46380 2023-12-14T18:37:51.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe09ffb700 1 -- 172.21.15.31:0/1894726636 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040a1890 con 0x55c33cb46380 2023-12-14T18:37:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 -- 172.21.15.31:0/1894726636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 msgr2=0x55c33cb50f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 0x55c33cb50f90 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efe040ca550 tx=0x7efe04099230 comp rx=0 tx=0).stop 2023-12-14T18:37:51.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 -- 172.21.15.31:0/1894726636 shutdown_connections 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46c90 0x55c33cb514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb58680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 --2- 172.21.15.31:0/1894726636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46380 0x55c33cb50f90 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 -- 172.21.15.31:0/1894726636 >> 172.21.15.31:0/1894726636 conn(0x55c33cb54200 msgr2=0x55c33cb5d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 -- 172.21.15.31:0/1894726636 shutdown_connections 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.662+0000 7efe254d6b80 1 -- 172.21.15.31:0/1894726636 wait complete. 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 -- start start 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 Processor -- start 2023-12-14T18:37:51.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb57d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb58260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 0x55c33cb561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33cb567e0 con 0x55c33cb46c90 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33cb56950 con 0x55c33cb3cca0 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c33cb56b10 con 0x55c33cb46380 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe254d6b80 10 client.0 _ll_register_callbacks cb 0x7efe0c003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 0x55c33cb561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 0x55c33cb561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50344/0 (socket says 172.21.15.31:50344) 2023-12-14T18:37:51.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe137fe700 1 -- 172.21.15.31:0/4149392846 learned_addr learned my addr 172.21.15.31:0/4149392846 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.663+0000 7efe12ffd700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb57d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe127fc700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb58260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe137fe700 1 -- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 msgr2=0x55c33cb58260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe137fe700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb58260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe137fe700 1 -- 172.21.15.31:0/4149392846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 msgr2=0x55c33cb57d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe137fe700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb57d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe137fe700 1 -- 172.21.15.31:0/4149392846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14110130 con 0x55c33cb46c90 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe137fe700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 0x55c33cb561c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009be0 tx=0x7efe0c009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457fd5a681494384 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:51.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efdfa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efdfa7fc700 1 -- 172.21.15.31:0/4149392846 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c036070 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efdfa7fc700 1 -- 172.21.15.31:0/4149392846 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7efe0c0073e0 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe0c00f040 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efdfa7fc700 1 -- 172.21.15.31:0/4149392846 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe0c023040 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe254d6b80 10 client.17718 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c33ca459a0 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.664+0000 7efdfa7fc700 1 -- 172.21.15.31:0/4149392846 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe0c00f040 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efdfa7fc700 1 -- 172.21.15.31:0/4149392846 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efe0c039020 con 0x55c33cb46c90 2023-12-14T18:37:51.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe254d6b80 10 client.17718 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe254d6b80 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe254d6b80 20 client.17718 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe254d6b80 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb51b50 0x55c33cbb7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe254d6b80 4 client.17718 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c33ca91a00 con 0x55c33cb51b50 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe12ffd700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb51b50 0x55c33cbb7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efdfa7fc700 10 client.17718 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.665+0000 7efe12ffd700 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb51b50 0x55c33cbb7e20 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321b22e6deb50278 server_cookie=664c46e21a53d481 in_seq=0 out_seq=0 2023-12-14T18:37:51.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efdfa7fc700 1 -- 172.21.15.31:0/4149392846 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c33ca91a00 con 0x55c33cb51b50 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efdfa7fc700 10 client.17718 handle_command_reply: tid=0 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 client.17718 shutdown 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 msgr2=0x55c33cb561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 0x55c33cb561c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009be0 tx=0x7efe0c009180 comp rx=0 tx=0).stop 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 shutdown_connections 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55c33cb51b50 0x55c33cbb7e20 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55c33cb46c90 0x55c33cb561c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55c33cb3cca0 0x55c33cb57d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efdfa7fc700 0 client.17718 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 --2- 172.21.15.31:0/4149392846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c33cb46380 0x55c33cb58260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 >> 172.21.15.31:0/4149392846 conn(0x55c33cb54200 msgr2=0x55c33cb49ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 shutdown_connections 2023-12-14T18:37:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 1 -- 172.21.15.31:0/4149392846 wait complete. 2023-12-14T18:37:51.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.666+0000 7efe254d6b80 20 client.17718 trim_cache size 0 max 16384 2023-12-14T18:37:51.677 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:37:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efe000a12c0 msgr2=0x7efe000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efe000a12c0 0x7efe000a3770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c00 tx=0x7efdfc003680 comp rx=0 tx=0).stop 2023-12-14T18:37:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 msgr2=0x7efe141969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:51.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 0x7efe141969a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d980 tx=0x7efe0400dc90 comp rx=0 tx=0).stop 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 shutdown_connections 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efe000a12c0 0x7efe000a3770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe14105550 0x7efe1419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efe14100c50 0x7efe14196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 --2- 172.21.15.31:0/601275891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe141015f0 0x7efe141969a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 >> 172.21.15.31:0/601275891 conn(0x7efe14077570 msgr2=0x7efe140feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 shutdown_connections 2023-12-14T18:37:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:51.667+0000 7efe19348700 1 -- 172.21.15.31:0/601275891 wait complete. 2023-12-14T18:37:52.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:51 smithi031 ceph-mon[42085]: pgmap v312: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:52.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:51 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3192206104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:51 smithi176 ceph-mon[43506]: pgmap v312: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:52.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:51 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3192206104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:51 smithi204 ceph-mon[43866]: pgmap v312: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:52.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:51 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3192206104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:52.682 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-12-14T18:37:52.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-12-14T18:37:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 -- 172.21.15.31:0/2632335149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ff390 msgr2=0x7fd4dc0ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 --2- 172.21.15.31:0/2632335149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ff390 0x7fd4dc0ff760 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00b0d0 tx=0x7fd4cc00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 -- 172.21.15.31:0/2632335149 shutdown_connections 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 --2- 172.21.15.31:0/2632335149 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 0x7fd4dc10bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 --2- 172.21.15.31:0/2632335149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ff390 0x7fd4dc0ff760 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 --2- 172.21.15.31:0/2632335149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 -- 172.21.15.31:0/2632335149 >> 172.21.15.31:0/2632335149 conn(0x7fd4dc075fd0 msgr2=0x7fd4dc0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 -- 172.21.15.31:0/2632335149 shutdown_connections 2023-12-14T18:37:52.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.832+0000 7fd4e3208700 1 -- 172.21.15.31:0/2632335149 wait complete. 2023-12-14T18:37:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 Processor -- start 2023-12-14T18:37:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 -- start start 2023-12-14T18:37:52.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc0ff390 0x7fd4dc196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 0x7fd4dc19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc19b4f0 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc19b660 con 0x7fd4dc1043d0 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e3208700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc19b820 con 0x7fd4dc0ff390 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50374/0 (socket says 172.21.15.31:50374) 2023-12-14T18:37:52.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4dbfff700 1 -- 172.21.15.31:0/1829277821 learned_addr learned my addr 172.21.15.31:0/1829277821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.833+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc0ff390 0x7fd4dc196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 0x7fd4dc19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4dbfff700 1 -- 172.21.15.31:0/1829277821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc0ff390 msgr2=0x7fd4dc196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc0ff390 0x7fd4dc196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4dbfff700 1 -- 172.21.15.31:0/1829277821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 msgr2=0x7fd4dc19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 0x7fd4dc19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4dbfff700 1 -- 172.21.15.31:0/1829277821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc009d00 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 0x7fd4dc19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc196a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400e940 tx=0x7fd4c400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:52.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4d9ffb700 1 -- 172.21.15.31:0/1829277821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009730 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4d9ffb700 1 -- 172.21.15.31:0/1829277821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4c400ee90 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc110300 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4d9ffb700 1 -- 172.21.15.31:0/1829277821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4010ea0 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.834+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc110170 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.837+0000 7fd4d9ffb700 1 -- 172.21.15.31:0/1829277821 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd4c400cbd0 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.837+0000 7fd4d9ffb700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4c80a12c0 0x7fd4c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.837+0000 7fd4d9ffb700 1 -- 172.21.15.31:0/1829277821 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd4c4014070 con 0x7fd4dc0ffd30 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.837+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4c80a12c0 0x7fd4c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4c80a12c0 0x7fd4c80a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009fd0 tx=0x7fd4cc00b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 Processor -- start 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 -- start start 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b3a4660 con 0x56036b4c5bd0 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b4138f0 con 0x56036b4ba400 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b40cd80 con 0x56036b4bad10 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50390/0 (socket says 172.21.15.31:50390) 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.838+0000 7fd4e17a5700 1 -- 172.21.15.31:0/942882519 learned_addr learned my addr 172.21.15.31:0/942882519 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e17a5700 1 -- 172.21.15.31:0/942882519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 msgr2=0x56036b4c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e17a5700 1 -- 172.21.15.31:0/942882519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 msgr2=0x56036b4ba7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4ba7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e17a5700 1 -- 172.21.15.31:0/942882519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036b28e430 con 0x56036b4c5bd0 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4ba7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c7fb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009c20 tx=0x7fd4d4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c98398e8975e3db server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4d2ffd700 1 -- 172.21.15.31:0/942882519 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4004220 con 0x56036b4c5bd0 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4d2ffd700 1 -- 172.21.15.31:0/942882519 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d400fdd0 con 0x56036b4c5bd0 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.839+0000 7fd4d2ffd700 1 -- 172.21.15.31:0/942882519 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4003dd0 con 0x56036b4c5bd0 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 -- 172.21.15.31:0/942882519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 msgr2=0x56036b4c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c7fb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009c20 tx=0x7fd4d4009f30 comp rx=0 tx=0).stop 2023-12-14T18:37:52.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 -- 172.21.15.31:0/942882519 shutdown_connections 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c7fb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4ba7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/942882519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 -- 172.21.15.31:0/942882519 >> 172.21.15.31:0/942882519 conn(0x56036b4b0d20 msgr2=0x56036b4b6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 -- 172.21.15.31:0/942882519 shutdown_connections 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 -- 172.21.15.31:0/942882519 wait complete. 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.840+0000 7fd4ef396b80 1 -- start start 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 Processor -- start 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b3a9010 con 0x56036b4c5bd0 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b3a8cb0 con 0x56036b4ba400 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b290720 con 0x56036b4bad10 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4ef396b80 10 client.0 _ll_register_callbacks cb 0x56036b3cc990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50402/0 (socket says 172.21.15.31:50402) 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4dbfff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4e17a5700 1 -- 172.21.15.31:0/2634918867 learned_addr learned my addr 172.21.15.31:0/2634918867 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.841+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e17a5700 1 -- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 msgr2=0x56036b4c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e17a5700 1 -- 172.21.15.31:0/2634918867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 msgr2=0x56036b4c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e17a5700 1 -- 172.21.15.31:0/2634918867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d4009900 con 0x56036b4c5bd0 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c2d10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400f640 tx=0x7fd4d400fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a6c87ed2dabf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4bbfff700 1 -- 172.21.15.31:0/2634918867 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4021070 con 0x56036b4c5bd0 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56036b4ae330 con 0x56036b4c5bd0 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4bbfff700 1 -- 172.21.15.31:0/2634918867 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd4d4005250 con 0x56036b4c5bd0 2023-12-14T18:37:52.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4bbfff700 1 -- 172.21.15.31:0/2634918867 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4d4005d50 con 0x56036b4c5bd0 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4ef396b80 10 client.17730 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56036b3d31f0 con 0x56036b4c5bd0 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4bbfff700 1 -- 172.21.15.31:0/2634918867 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56036b4ae330 con 0x56036b4c5bd0 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.842+0000 7fd4bbfff700 1 -- 172.21.15.31:0/2634918867 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4d4011460 con 0x56036b4c5bd0 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4ef396b80 10 client.17730 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4ef396b80 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4ef396b80 20 client.17730 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4d5430 0x56036b4d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4ef396b80 4 client.17730 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:52.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56036b405d60 con 0x56036b4d5430 2023-12-14T18:37:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4d5430 0x56036b4d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4bbfff700 10 client.17730 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:52.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.843+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4d5430 0x56036b4d7840 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1c5ca9799dbe06d server_cookie=e0ed1e06daebba3b in_seq=0 out_seq=0 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4bbfff700 1 -- 172.21.15.31:0/2634918867 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56036b405d60 con 0x56036b4d5430 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4bbfff700 10 client.17730 handle_command_reply: tid=0 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 client.17730 shutdown 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 msgr2=0x56036b4c2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c2d10 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400f640 tx=0x7fd4d400fb40 comp rx=0 tx=0).stop 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 shutdown_connections 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4d5430 0x56036b4d7840 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4c5bd0 0x56036b4c2d10 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/2634918867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4bad10 0x56036b4c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4bbfff700 0 client.17730 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.844+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 >> 172.21.15.31:0/2634918867 conn(0x56036b4b0d20 msgr2=0x56036b4b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.845+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 shutdown_connections 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.845+0000 7fd4ef396b80 1 -- 172.21.15.31:0/2634918867 wait complete. 2023-12-14T18:37:52.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.845+0000 7fd4ef396b80 20 client.17730 trim_cache size 0 max 16384 2023-12-14T18:37:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 Processor -- start 2023-12-14T18:37:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 -- start start 2023-12-14T18:37:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4b0d20 0x56036b4bc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4c5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b4bb680 con 0x56036b4bad10 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b4be630 con 0x56036b4ba400 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b4b6d40 con 0x56036b4b0d20 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4c5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50414/0 (socket says 172.21.15.31:50414) 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4e17a5700 1 -- 172.21.15.31:0/4034761850 learned_addr learned my addr 172.21.15.31:0/4034761850 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4b0d20 0x56036b4bc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.863+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4e17a5700 1 -- 172.21.15.31:0/4034761850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4b0d20 msgr2=0x56036b4bc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4b0d20 0x56036b4bc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4e17a5700 1 -- 172.21.15.31:0/4034761850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 msgr2=0x56036b4c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4e17a5700 1 -- 172.21.15.31:0/4034761850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56036b3d2f40 con 0x56036b4bad10 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4c5500 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40094f0 tx=0x7fd4d400ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cfd0380fa089e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4d37fe700 1 -- 172.21.15.31:0/4034761850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4021040 con 0x56036b4bad10 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4d37fe700 1 -- 172.21.15.31:0/4034761850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d400fc60 con 0x56036b4bad10 2023-12-14T18:37:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.864+0000 7fd4d37fe700 1 -- 172.21.15.31:0/4034761850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4003e40 con 0x56036b4bad10 2023-12-14T18:37:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 -- 172.21.15.31:0/4034761850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 msgr2=0x56036b4c5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4c5500 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40094f0 tx=0x7fd4d400ff40 comp rx=0 tx=0).stop 2023-12-14T18:37:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 -- 172.21.15.31:0/4034761850 shutdown_connections 2023-12-14T18:37:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4c5500 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4b0d20 0x56036b4bc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/4034761850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4ba400 0x56036b4c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 -- 172.21.15.31:0/4034761850 >> 172.21.15.31:0/4034761850 conn(0x56036b4c8280 msgr2=0x56036b4bc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 -- 172.21.15.31:0/4034761850 shutdown_connections 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 -- 172.21.15.31:0/4034761850 wait complete. 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 -- start start 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.865+0000 7fd4ef396b80 1 Processor -- start 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4b0d20 0x56036b4caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4ba400 0x56036b4cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b529400 con 0x56036b4bad10 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b529570 con 0x56036b4b0d20 2023-12-14T18:37:52.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56036b529730 con 0x56036b4ba400 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4ef396b80 10 client.0 _ll_register_callbacks cb 0x7fd4c4095c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50424/0 (socket says 172.21.15.31:50424) 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 -- 172.21.15.31:0/200628886 learned_addr learned my addr 172.21.15.31:0/200628886 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4b0d20 0x56036b4caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4dbfff700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4ba400 0x56036b4cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 -- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4ba400 msgr2=0x56036b4cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4ba400 0x56036b4cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 -- 172.21.15.31:0/200628886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4b0d20 msgr2=0x56036b4caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4b0d20 0x56036b4caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 -- 172.21.15.31:0/200628886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56036b3d31f0 con 0x56036b4bad10 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4e17a5700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4cb980 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009460 tx=0x7fd4d4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78e75b4556bdb17 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.866+0000 7fd4bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4bbfff700 1 -- 172.21.15.31:0/200628886 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4021040 con 0x56036b4bad10 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4bbfff700 1 -- 172.21.15.31:0/200628886 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fd4d4000e40 con 0x56036b4bad10 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4d400fdd0 con 0x56036b4bad10 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4bbfff700 1 -- 172.21.15.31:0/200628886 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4d4005540 con 0x56036b4bad10 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 10 client.17736 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56036b3b9a20 con 0x56036b4bad10 2023-12-14T18:37:52.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4bbfff700 1 -- 172.21.15.31:0/200628886 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d400fdd0 con 0x56036b4bad10 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4bbfff700 1 -- 172.21.15.31:0/200628886 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd4d4026420 con 0x56036b4bad10 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 10 client.17736 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 20 client.17736 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4c5bd0 0x56036b52caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 4 client.17736 mds_command: new command op to 24538 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56036b405a80 con 0x56036b4c5bd0 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.867+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4c5bd0 0x56036b52caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:52.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.868+0000 7fd4e0fa4700 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4c5bd0 0x56036b52caa0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f10f9f3cafeb73 server_cookie=a1052b4730a689e1 in_seq=0 out_seq=0 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.868+0000 7fd4bbfff700 10 client.17736 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4bbfff700 1 -- 172.21.15.31:0/200628886 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56036b405a80 con 0x56036b4c5bd0 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4bbfff700 10 client.17736 handle_command_reply: tid=0 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 client.17736 shutdown 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 msgr2=0x56036b4cb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4cb980 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009460 tx=0x7fd4d4003910 comp rx=0 tx=0).stop 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 shutdown_connections 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x56036b4c5bd0 0x56036b52caa0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x56036b4bad10 0x56036b4cb980 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x56036b4b0d20 0x56036b4caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 --2- 172.21.15.31:0/200628886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56036b4ba400 0x56036b4cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 >> 172.21.15.31:0/200628886 conn(0x56036b4c8280 msgr2=0x56036b4d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4bbfff700 0 client.17736 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 shutdown_connections 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.869+0000 7fd4ef396b80 1 -- 172.21.15.31:0/200628886 wait complete. 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4ef396b80 20 client.17736 trim_cache size 0 max 16384 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:37:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "return_code": 0, 2023-12-14T18:37:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_tag": "455ad569-9238-46c8-ae92-f37f7026eb36", 2023-12-14T18:37:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": "asynchronous" 2023-12-14T18:37:52.880 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:37:52.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4c80a12c0 msgr2=0x7fd4c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4c80a12c0 0x7fd4c80a3770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009fd0 tx=0x7fd4cc00b040 comp rx=0 tx=0).stop 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 msgr2=0x7fd4dc196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc196a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400e940 tx=0x7fd4c400ec50 comp rx=0 tx=0).stop 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 shutdown_connections 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4dc1043d0 0x7fd4dc19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fd4c80a12c0 0x7fd4c80a3770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4dc0ff390 0x7fd4dc196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 --2- 172.21.15.31:0/1829277821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0ffd30 0x7fd4dc196a50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 >> 172.21.15.31:0/1829277821 conn(0x7fd4dc075fd0 msgr2=0x7fd4dc102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 shutdown_connections 2023-12-14T18:37:52.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:52.870+0000 7fd4e3208700 1 -- 172.21.15.31:0/1829277821 wait complete. 2023-12-14T18:37:52.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:37:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 -- 172.21.15.31:0/3999168096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ffd50 msgr2=0x7fb26c103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 --2- 172.21.15.31:0/3999168096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ffd50 0x7fb26c103d10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb2540099c0 tx=0x7fb254009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 -- 172.21.15.31:0/3999168096 shutdown_connections 2023-12-14T18:37:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 --2- 172.21.15.31:0/3999168096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c1043f0 0x7fb26c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 --2- 172.21.15.31:0/3999168096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c0ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 --2- 172.21.15.31:0/3999168096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ffd50 0x7fb26c103d10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 -- 172.21.15.31:0/3999168096 >> 172.21.15.31:0/3999168096 conn(0x7fb26c075fa0 msgr2=0x7fb26c0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 -- 172.21.15.31:0/3999168096 shutdown_connections 2023-12-14T18:37:53.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.023+0000 7fb2710ef700 1 -- 172.21.15.31:0/3999168096 wait complete. 2023-12-14T18:37:53.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 Processor -- start 2023-12-14T18:37:53.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 -- start start 2023-12-14T18:37:53.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c0ffd50 0x7fb26c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb26c1043f0 0x7fb26c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19b400 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19b570 con 0x7fb26c0ffd50 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.024+0000 7fb2710ef700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19b730 con 0x7fb26c1043f0 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50442/0 (socket says 172.21.15.31:50442) 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 -- 172.21.15.31:0/909162126 learned_addr learned my addr 172.21.15.31:0/909162126 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26a59c700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c0ffd50 0x7fb26c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26b59e700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb26c1043f0 0x7fb26c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 -- 172.21.15.31:0/909162126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb26c1043f0 msgr2=0x7fb26c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb26c1043f0 0x7fb26c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 -- 172.21.15.31:0/909162126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c0ffd50 msgr2=0x7fb26c196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c0ffd50 0x7fb26c196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 -- 172.21.15.31:0/909162126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2540095f0 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c196420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00d980 tx=0x7fb25c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb263fff700 1 -- 172.21.15.31:0/909162126 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c004220 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb263fff700 1 -- 172.21.15.31:0/909162126 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25c007640 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c10f170 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb263fff700 1 -- 172.21.15.31:0/909162126 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c005690 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.025+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c10efe0 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.028+0000 7fb263fff700 1 -- 172.21.15.31:0/909162126 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb25c004750 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.029+0000 7fb263fff700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2580a12c0 0x7fb2580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.029+0000 7fb26a59c700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2580a12c0 0x7fb2580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.029+0000 7fb263fff700 1 -- 172.21.15.31:0/909162126 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb25c0ccee0 con 0x7fb26c0ff3b0 2023-12-14T18:37:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb26a59c700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2580a12c0 0x7fb2580a3770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2540058e0 tx=0x7fb254005fb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 Processor -- start 2023-12-14T18:37:53.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 -- start start 2023-12-14T18:37:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908957750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x555908969340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559088415e0 con 0x555908957c90 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559088b0870 con 0x555908966f60 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb27d27db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559088a9d00 con 0x555908957380 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50452/0 (socket says 172.21.15.31:50452) 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb26a59c700 1 -- 172.21.15.31:0/4062058917 learned_addr learned my addr 172.21.15.31:0/4062058917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.030+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908957750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26b59e700 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x555908969340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26a59c700 1 -- 172.21.15.31:0/4062058917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 msgr2=0x555908957750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26a59c700 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908957750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26a59c700 1 -- 172.21.15.31:0/4062058917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 msgr2=0x555908969340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26a59c700 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x555908969340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26a59c700 1 -- 172.21.15.31:0/4062058917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55590872b3b0 con 0x555908957c90 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26b59e700 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x555908969340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb26a59c700 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966a20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb2540099c0 tx=0x7fb25401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7266f939c744ae server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb2617fa700 1 -- 172.21.15.31:0/4062058917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400b990 con 0x555908957c90 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb2617fa700 1 -- 172.21.15.31:0/4062058917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25401e040 con 0x555908957c90 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.031+0000 7fb2617fa700 1 -- 172.21.15.31:0/4062058917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2540306e0 con 0x555908957c90 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 -- 172.21.15.31:0/4062058917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 msgr2=0x555908966a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966a20 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fb2540099c0 tx=0x7fb25401f000 comp rx=0 tx=0).stop 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 -- 172.21.15.31:0/4062058917 shutdown_connections 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x555908969340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908957750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4062058917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966a20 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 -- 172.21.15.31:0/4062058917 >> 172.21.15.31:0/4062058917 conn(0x55590894dca0 msgr2=0x5559089537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 -- 172.21.15.31:0/4062058917 shutdown_connections 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 -- 172.21.15.31:0/4062058917 wait complete. 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 -- start start 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.032+0000 7fb27d27db80 1 Processor -- start 2023-12-14T18:37:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x5559089668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908963b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x5559089640f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555908845f90 con 0x555908957c90 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555908845c30 con 0x555908966f60 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590872d6a0 con 0x555908957380 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb27d27db80 10 client.0 _ll_register_callbacks cb 0x555908869910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908963b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908963b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50458/0 (socket says 172.21.15.31:50458) 2023-12-14T18:37:53.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 -- 172.21.15.31:0/3051387225 learned_addr learned my addr 172.21.15.31:0/3051387225 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26b59e700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x5559089640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x5559089668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 -- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 msgr2=0x5559089668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x5559089668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 -- 172.21.15.31:0/3051387225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 msgr2=0x5559089640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x5559089640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.033+0000 7fb26a59c700 1 -- 172.21.15.31:0/3051387225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c10c800 con 0x555908957c90 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb26a59c700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908963b70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb254004670 tx=0x7fb254005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21e1addc476d69a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb2527fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:53.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb2527fc700 1 -- 172.21.15.31:0/3051387225 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254034070 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55590894b2b0 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb2527fc700 1 -- 172.21.15.31:0/3051387225 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb2540309a0 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb2527fc700 1 -- 172.21.15.31:0/3051387225 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb254030b60 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 10 client.17748 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555908870170 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb2527fc700 1 -- 172.21.15.31:0/3051387225 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55590894b2b0 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb2527fc700 1 -- 172.21.15.31:0/3051387225 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb25400f580 con 0x555908957c90 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 10 client.17748 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 20 client.17748 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089762b0 0x5559089786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 4 client.17748 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.034+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559088a2ce0 con 0x5559089762b0 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.035+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089762b0 0x5559089786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.035+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089762b0 0x5559089786c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a94324f6497932c server_cookie=1b6e2e994fb5768 in_seq=0 out_seq=0 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.035+0000 7fb2527fc700 10 client.17748 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb2527fc700 1 -- 172.21.15.31:0/3051387225 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5559088a2ce0 con 0x5559089762b0 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb2527fc700 10 client.17748 handle_command_reply: tid=0 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 client.17748 shutdown 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 msgr2=0x555908963b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908963b70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb254004670 tx=0x7fb254005a00 comp rx=0 tx=0).stop 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 shutdown_connections 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089762b0 0x5559089786c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x5559089640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x5559089668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb2527fc700 0 client.17748 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:53.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 --2- 172.21.15.31:0/3051387225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908963b70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 >> 172.21.15.31:0/3051387225 conn(0x55590894dca0 msgr2=0x5559089533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 shutdown_connections 2023-12-14T18:37:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 1 -- 172.21.15.31:0/3051387225 wait complete. 2023-12-14T18:37:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.036+0000 7fb27d27db80 20 client.17748 trim_cache size 0 max 16384 2023-12-14T18:37:53.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 Processor -- start 2023-12-14T18:37:53.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 -- start start 2023-12-14T18:37:53.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908957380 0x55590896da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908966f60 0x5559089668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590895fa00 con 0x555908957c90 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55590896df70 con 0x555908957380 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb27d27db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555908963100 con 0x555908966f60 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50468/0 (socket says 172.21.15.31:50468) 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.054+0000 7fb26a59c700 1 -- 172.21.15.31:0/4292167177 learned_addr learned my addr 172.21.15.31:0/4292167177 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26b59e700 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908966f60 0x5559089668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908957380 0x55590896da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26a59c700 1 -- 172.21.15.31:0/4292167177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908966f60 msgr2=0x5559089668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26a59c700 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908966f60 0x5559089668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26a59c700 1 -- 172.21.15.31:0/4292167177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908957380 msgr2=0x55590896da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26a59c700 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908957380 0x55590896da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26a59c700 1 -- 172.21.15.31:0/4292167177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55590886fec0 con 0x555908957c90 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26b59e700 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908966f60 0x5559089668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb26a59c700 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966380 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb254003a70 tx=0x7fb254009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91607dc9e6451b0e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb261ffb700 1 -- 172.21.15.31:0/4292167177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254034070 con 0x555908957c90 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb261ffb700 1 -- 172.21.15.31:0/4292167177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb254003d80 con 0x555908957c90 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.055+0000 7fb261ffb700 1 -- 172.21.15.31:0/4292167177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254024770 con 0x555908957c90 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 -- 172.21.15.31:0/4292167177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 msgr2=0x555908966380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966380 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb254003a70 tx=0x7fb254009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 -- 172.21.15.31:0/4292167177 shutdown_connections 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908966f60 0x5559089668c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908957380 0x55590896da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 --2- 172.21.15.31:0/4292167177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908966380 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 -- 172.21.15.31:0/4292167177 >> 172.21.15.31:0/4292167177 conn(0x55590894dca0 msgr2=0x555908972bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 -- 172.21.15.31:0/4292167177 shutdown_connections 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 -- 172.21.15.31:0/4292167177 wait complete. 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 -- start start 2023-12-14T18:37:53.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.056+0000 7fb27d27db80 1 Processor -- start 2023-12-14T18:37:53.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908961310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908961850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x55590895fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559089604e0 con 0x555908957c90 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555908960650 con 0x555908966f60 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555908960810 con 0x555908957380 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb27d27db80 10 client.0 _ll_register_callbacks cb 0x7fb25c095f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908961850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908961850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50474/0 (socket says 172.21.15.31:50474) 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 -- 172.21.15.31:0/913142784 learned_addr learned my addr 172.21.15.31:0/913142784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26b59e700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x55590895fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908961310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 -- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 msgr2=0x555908961310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908961310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 -- 172.21.15.31:0/913142784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 msgr2=0x55590895fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x55590895fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 -- 172.21.15.31:0/913142784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555908870170 con 0x555908957c90 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb26a59c700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908961850 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb2540095b0 tx=0x7fb25400f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b292c124b74404d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:53.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb2527fc700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb2527fc700 1 -- 172.21.15.31:0/913142784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254034070 con 0x555908957c90 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.057+0000 7fb2527fc700 1 -- 172.21.15.31:0/913142784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7fb254030aa0 con 0x555908957c90 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559088a2a00 con 0x555908957c90 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb2527fc700 1 -- 172.21.15.31:0/913142784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb254003d80 con 0x555908957c90 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 10 client.17754 fetch_fsmap learned FSMap version 34 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5559088569a0 con 0x555908957c90 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb2527fc700 1 -- 172.21.15.31:0/913142784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559088a2a00 con 0x555908957c90 2023-12-14T18:37:53.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb2527fc700 1 -- 172.21.15.31:0/913142784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb254040070 con 0x555908957c90 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 10 client.17754 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 20 client.17754 populate_metadata read hostname 'smithi031' 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089cadd0 0x5559089cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 4 client.17754 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5559086f8cf0 con 0x5559089cadd0 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.058+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089cadd0 0x5559089cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb26ad9d700 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089cadd0 0x5559089cd1b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b796d1773eee9e7 server_cookie=8b0d304df18a3ab9 in_seq=0 out_seq=0 2023-12-14T18:37:53.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb2527fc700 10 client.17754 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb2527fc700 1 -- 172.21.15.31:0/913142784 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5559086f8cf0 con 0x5559089cadd0 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb2527fc700 10 client.17754 handle_command_reply: tid=0 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 client.17754 shutdown 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 msgr2=0x555908961850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908961850 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb2540095b0 tx=0x7fb25400f780 comp rx=0 tx=0).stop 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 shutdown_connections 2023-12-14T18:37:53.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x5559089cadd0 0x5559089cd1b0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555908966f60 0x55590895fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555908957380 0x555908961310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 --2- 172.21.15.31:0/913142784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555908957c90 0x555908961850 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb2527fc700 0 client.17754 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 >> 172.21.15.31:0/913142784 conn(0x55590894dca0 msgr2=0x55590895eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 shutdown_connections 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 1 -- 172.21.15.31:0/913142784 wait complete. 2023-12-14T18:37:53.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.059+0000 7fb27d27db80 20 client.17754 trim_cache size 0 max 16384 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "IDLE (waiting for more scrubs)", 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": { 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "455ad569-9238-46c8-ae92-f37f7026eb36": { 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/", 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tag": "455ad569-9238-46c8-ae92-f37f7026eb36", 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "options": "recursive,force" 2023-12-14T18:37:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:37:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:37:53.072 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:37:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2580a12c0 msgr2=0x7fb2580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2580a12c0 0x7fb2580a3770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2540058e0 tx=0x7fb254005fb0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 msgr2=0x7fb26c196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c196420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00d980 tx=0x7fb25c00dc90 comp rx=0 tx=0).stop 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 shutdown_connections 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2580a12c0 0x7fb2580a3770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb26c1043f0 0x7fb26c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb26c0ff3b0 0x7fb26c196420 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 --2- 172.21.15.31:0/909162126 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c0ffd50 0x7fb26c196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 >> 172.21.15.31:0/909162126 conn(0x7fb26c075fa0 msgr2=0x7fb26c068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:53.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 shutdown_connections 2023-12-14T18:37:53.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:53.060+0000 7fb2710ef700 1 -- 172.21.15.31:0/909162126 wait complete. 2023-12-14T18:37:53.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:52 smithi031 ceph-mon[42085]: scrub queued for path: / 2023-12-14T18:37:53.074 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:52 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:53.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:52 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:37:53.075 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:52 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:53.075 INFO:tasks.cephfs.filesystem:scrub status for tag:455ad569-9238-46c8-ae92-f37f7026eb36 - {'path': '/', 'tag': '455ad569-9238-46c8-ae92-f37f7026eb36', 'options': 'recursive,force'} 2023-12-14T18:37:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:52 smithi176 ceph-mon[43506]: scrub queued for path: / 2023-12-14T18:37:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:52 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:52 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:37:53.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:52 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:52 smithi204 ceph-mon[43866]: scrub queued for path: / 2023-12-14T18:37:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:52 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:52 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:37:53.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:52 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:53 smithi031 ceph-mon[42085]: pgmap v313: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:53 smithi031 ceph-mon[42085]: scrub summary: active paths [/] 2023-12-14T18:37:54.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:53 smithi031 ceph-mon[42085]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:53 smithi176 ceph-mon[43506]: pgmap v313: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:53 smithi176 ceph-mon[43506]: scrub summary: active paths [/] 2023-12-14T18:37:54.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:53 smithi176 ceph-mon[43506]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:53 smithi204 ceph-mon[43866]: pgmap v313: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:53 smithi204 ceph-mon[43866]: scrub summary: active paths [/] 2023-12-14T18:37:54.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:53 smithi204 ceph-mon[43866]: scrub summary: idle+waiting paths [/] 2023-12-14T18:37:54.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-14T18:37:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.916+0000 7f4e6cb07700 1 -- 172.21.15.31:0/2416443363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68100c50 msgr2=0x7f4e68101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.916+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/2416443363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68100c50 0x7f4e68101020 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4e500099c0 tx=0x7f4e50009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 -- 172.21.15.31:0/2416443363 shutdown_connections 2023-12-14T18:37:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/2416443363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e68105560 0x7f4e6810d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/2416443363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68100c50 0x7f4e68101020 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:54.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/2416443363 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e681015f0 0x7f4e68104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 -- 172.21.15.31:0/2416443363 >> 172.21.15.31:0/2416443363 conn(0x7f4e68077560 msgr2=0x7f4e68077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 -- 172.21.15.31:0/2416443363 shutdown_connections 2023-12-14T18:37:54.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.917+0000 7f4e6cb07700 1 -- 172.21.15.31:0/2416443363 wait complete. 2023-12-14T18:37:54.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 Processor -- start 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 -- start start 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e68100c50 0x7f4e68072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e681015f0 0x7f4e68073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 0x7f4e681a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e68073b60 con 0x7f4e68105560 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681a4ad0 con 0x7f4e68100c50 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6cb07700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681a4c10 con 0x7f4e681015f0 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 0x7f4e681a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:54.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 0x7f4e681a4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58740/0 (socket says 172.21.15.31:58740) 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 -- 172.21.15.31:0/3337202679 learned_addr learned my addr 172.21.15.31:0/3337202679 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e6659c700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e68100c50 0x7f4e68072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e65d9b700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e681015f0 0x7f4e68073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 -- 172.21.15.31:0/3337202679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e681015f0 msgr2=0x7f4e68073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e681015f0 0x7f4e68073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 -- 172.21.15.31:0/3337202679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e68100c50 msgr2=0x7f4e68072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e68100c50 0x7f4e68072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.918+0000 7f4e66d9d700 1 -- 172.21.15.31:0/3337202679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500095f0 con 0x7f4e68105560 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.919+0000 7f4e66d9d700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 0x7f4e681a4590 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4e60007ee0 tx=0x7f4e6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.919+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60017070 con 0x7f4e68105560 2023-12-14T18:37:54.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.919+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6000f040 con 0x7f4e68105560 2023-12-14T18:37:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.919+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e68110320 con 0x7f4e68105560 2023-12-14T18:37:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.919+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60005580 con 0x7f4e68105560 2023-12-14T18:37:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.919+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e68110190 con 0x7f4e68105560 2023-12-14T18:37:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.920+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e681104b0 con 0x7f4e68105560 2023-12-14T18:37:54.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.921+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e6000db20 con 0x7f4e68105560 2023-12-14T18:37:54.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.922+0000 7f4e5f7fe700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4e540a12c0 0x7f4e540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:54.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.922+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7f4e600cce50 con 0x7f4e68105560 2023-12-14T18:37:54.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.922+0000 7f4e6659c700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4e540a12c0 0x7f4e540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:54.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.923+0000 7f4e6659c700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4e540a12c0 0x7f4e540a3770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e50006010 tx=0x7f4e50009f10 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:54.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:54.926+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4e60095e10 con 0x7f4e68105560 2023-12-14T18:37:55.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.162+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 --> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e680ff000 con 0x7f4e540a12c0 2023-12-14T18:37:55.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.171+0000 7f4e5f7fe700 1 -- 172.21.15.31:0/3337202679 <== mgr.15585 v2:172.21.15.31:6800/1869071623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f4e680ff000 con 0x7f4e540a12c0 2023-12-14T18:37:55.173 INFO:teuthology.orchestra.run.smithi031.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-12-14T00:00:00", "created": "2023-12-14T18:27:53", "first": "2023-12-14T18:28:00", "last": "2023-12-14T18:37:00", "last_pruned": "2023-12-14T18:37:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4e540a12c0 msgr2=0x7f4e540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4e540a12c0 0x7f4e540a3770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4e50006010 tx=0x7f4e50009f10 comp rx=0 tx=0).stop 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 msgr2=0x7f4e681a4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 0x7f4e681a4590 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4e60007ee0 tx=0x7f4e6000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 shutdown_connections 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e68105560 0x7f4e681a4590 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4e68100c50 0x7f4e68072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f4e540a12c0 0x7f4e540a3770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 --2- 172.21.15.31:0/3337202679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e681015f0 0x7f4e68073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 >> 172.21.15.31:0/3337202679 conn(0x7f4e68077560 msgr2=0x7f4e680fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 shutdown_connections 2023-12-14T18:37:55.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.174+0000 7f4e6cb07700 1 -- 172.21.15.31:0/3337202679 wait complete. 2023-12-14T18:37:55.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-14T18:37:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 -- 172.21.15.31:0/3123812083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 msgr2=0x7efde81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 --2- 172.21.15.31:0/3123812083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde81016f0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efde000b0a0 tx=0x7efde000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 -- 172.21.15.31:0/3123812083 shutdown_connections 2023-12-14T18:37:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 --2- 172.21.15.31:0/3123812083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde810a840 0x7efde810cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 --2- 172.21.15.31:0/3123812083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde81016f0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 --2- 172.21.15.31:0/3123812083 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 0x7efde810a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 -- 172.21.15.31:0/3123812083 >> 172.21.15.31:0/3123812083 conn(0x7efde8075fa0 msgr2=0x7efde80763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 -- 172.21.15.31:0/3123812083 shutdown_connections 2023-12-14T18:37:55.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.327+0000 7efdef342700 1 -- 172.21.15.31:0/3123812083 wait complete. 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 Processor -- start 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 -- start start 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde8196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 0x7efde81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde810a840 0x7efde819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde819b450 con 0x7efde8101320 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde819b5c0 con 0x7efde8101c30 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdef342700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde819b780 con 0x7efde810a840 2023-12-14T18:37:55.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efded0de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde8196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efded0de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde8196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58760/0 (socket says 172.21.15.31:58760) 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efded0de700 1 -- 172.21.15.31:0/2448763448 learned_addr learned my addr 172.21.15.31:0/2448763448 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efdec8dd700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 0x7efde81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.328+0000 7efded8df700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde810a840 0x7efde819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efded0de700 1 -- 172.21.15.31:0/2448763448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde810a840 msgr2=0x7efde819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efded0de700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde810a840 0x7efde819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efded0de700 1 -- 172.21.15.31:0/2448763448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 msgr2=0x7efde81969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efded0de700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 0x7efde81969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efded0de700 1 -- 172.21.15.31:0/2448763448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0009d00 con 0x7efde8101320 2023-12-14T18:37:55.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efdec8dd700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 0x7efde81969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efded0de700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde8196470 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efde0006070 tx=0x7efde00079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000e040 con 0x7efde8101320 2023-12-14T18:37:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde0004ce0 con 0x7efde8101320 2023-12-14T18:37:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde81102b0 con 0x7efde8101320 2023-12-14T18:37:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde001d480 con 0x7efde8101320 2023-12-14T18:37:55.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.329+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde8110120 con 0x7efde8101320 2023-12-14T18:37:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.330+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde8110440 con 0x7efde8101320 2023-12-14T18:37:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.331+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efde0003a00 con 0x7efde8101320 2023-12-14T18:37:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.332+0000 7efde67fc700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efdd00a12c0 0x7efdd00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.332+0000 7efdec8dd700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efdd00a12c0 0x7efdd00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.332+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9525+0+0 (secure 0 0 0) 0x7efde00ce600 con 0x7efde8101320 2023-12-14T18:37:55.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.332+0000 7efdec8dd700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efdd00a12c0 0x7efdd00a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efde8197a80 tx=0x7efddc00b410 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:55.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.336+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efde00976b0 con 0x7efde8101320 2023-12-14T18:37:55.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.574+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 --> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efde8104a60 con 0x7efdd00a12c0 2023-12-14T18:37:55.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.582+0000 7efde67fc700 1 -- 172.21.15.31:0/2448763448 <== mgr.15585 v2:172.21.15.31:6800/1869071623 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7efde8104a60 con 0x7efdd00a12c0 2023-12-14T18:37:55.584 INFO:teuthology.orchestra.run.smithi031.stdout:/ 1M 6M3h 2023-12-14T18:37:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.584+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efdd00a12c0 msgr2=0x7efdd00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efdd00a12c0 0x7efdd00a3770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efde8197a80 tx=0x7efddc00b410 comp rx=0 tx=0).stop 2023-12-14T18:37:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 msgr2=0x7efde8196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde8196470 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7efde0006070 tx=0x7efde00079b0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 shutdown_connections 2023-12-14T18:37:55.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7efdd00a12c0 0x7efdd00a3770 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde810a840 0x7efde819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efde8101320 0x7efde8196470 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 --2- 172.21.15.31:0/2448763448 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efde8101c30 0x7efde81969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 >> 172.21.15.31:0/2448763448 conn(0x7efde8075fa0 msgr2=0x7efde8104350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 shutdown_connections 2023-12-14T18:37:55.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:37:55.585+0000 7efdef342700 1 -- 172.21.15.31:0/2448763448 wait complete. 2023-12-14T18:37:55.601 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-12-14T18:37:55.610 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-12-14T18:37:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:55 smithi031 ceph-mon[42085]: pgmap v314: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2023-12-14T18:37:56.214 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:55 smithi031 ceph-mon[42085]: from='client.17760 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:37:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:55 smithi176 ceph-mon[43506]: pgmap v314: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2023-12-14T18:37:56.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:55 smithi176 ceph-mon[43506]: from='client.17760 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:37:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:55 smithi204 ceph-mon[43866]: pgmap v314: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7 op/s 2023-12-14T18:37:56.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:55 smithi204 ceph-mon[43866]: from='client.17760 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:37:57.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:56 smithi031 ceph-mon[42085]: from='client.17766 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:37:57.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:56 smithi176 ceph-mon[43506]: from='client.17766 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:37:57.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:56 smithi204 ceph-mon[43866]: from='client.17766 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:37:58.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:57 smithi031 ceph-mon[42085]: pgmap v315: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:58.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:57 smithi176 ceph-mon[43506]: pgmap v315: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:58.339 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:57 smithi204 ceph-mon[43866]: pgmap v315: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2023-12-14T18:37:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:59 smithi204 ceph-mon[43866]: pgmap v316: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8 op/s 2023-12-14T18:37:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:59 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:37:59.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:37:59 smithi204 ceph-mon[43866]: scrub summary: idle 2023-12-14T18:37:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:59 smithi031 ceph-mon[42085]: pgmap v316: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8 op/s 2023-12-14T18:37:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:59 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:37:59.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:37:59 smithi031 ceph-mon[42085]: scrub summary: idle 2023-12-14T18:37:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:59 smithi176 ceph-mon[43506]: pgmap v316: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8 op/s 2023-12-14T18:37:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:59 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:37:59.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:37:59 smithi176 ceph-mon[43506]: scrub summary: idle 2023-12-14T18:38:01.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:01 smithi204 ceph-mon[43866]: pgmap v317: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:38:01.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:01 smithi031 ceph-mon[42085]: pgmap v317: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:38:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:01 smithi176 ceph-mon[43506]: pgmap v317: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:38:03.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:03 smithi204 ceph-mon[43866]: pgmap v318: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:38:03.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:03 smithi031 ceph-mon[42085]: pgmap v318: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:38:03.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:03 smithi176 ceph-mon[43506]: pgmap v318: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:38:05.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:05 smithi204 ceph-mon[43866]: pgmap v319: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-12-14T18:38:05.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:05 smithi031 ceph-mon[42085]: pgmap v319: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-12-14T18:38:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:05 smithi176 ceph-mon[43506]: pgmap v319: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 25 op/s 2023-12-14T18:38:07.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:07 smithi204 ceph-mon[43866]: pgmap v320: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:38:07.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:07 smithi031 ceph-mon[42085]: pgmap v320: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:38:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:07 smithi176 ceph-mon[43506]: pgmap v320: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:38:09.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:09 smithi204 ceph-mon[43866]: pgmap v321: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-12-14T18:38:09.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:09 smithi031 ceph-mon[42085]: pgmap v321: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-12-14T18:38:09.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:09 smithi176 ceph-mon[43506]: pgmap v321: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2023-12-14T18:38:10.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:10 smithi204 ceph-mon[43866]: osdmap e99: 12 total, 12 up, 12 in 2023-12-14T18:38:10.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:10 smithi031 ceph-mon[42085]: osdmap e99: 12 total, 12 up, 12 in 2023-12-14T18:38:10.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:10 smithi176 ceph-mon[43506]: osdmap e99: 12 total, 12 up, 12 in 2023-12-14T18:38:11.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:11 smithi204 ceph-mon[43866]: pgmap v323: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:38:11.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:11 smithi031 ceph-mon[42085]: pgmap v323: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:38:11.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:11 smithi176 ceph-mon[43506]: pgmap v323: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:38:13.840 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:13 smithi204 ceph-mon[43866]: pgmap v324: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:38:13.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:13 smithi031 ceph-mon[42085]: pgmap v324: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:38:13.993 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:13 smithi176 ceph-mon[43506]: pgmap v324: 129 pgs: 129 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:38:15.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:15 smithi031 ceph-mon[42085]: pgmap v325: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:38:15.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:15 smithi176 ceph-mon[43506]: pgmap v325: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:38:16.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:15 smithi204 ceph-mon[43866]: pgmap v325: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:38:17.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:17 smithi031 ceph-mon[42085]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:38:17.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:17 smithi176 ceph-mon[43506]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:38:18.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:17 smithi204 ceph-mon[43866]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:38:18.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:18 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:38:18.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:18 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:38:19.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:18 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:38:19.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:19 smithi176 ceph-mon[43506]: pgmap v327: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 102 B/s wr, 5 op/s 2023-12-14T18:38:19.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:19 smithi031 ceph-mon[42085]: pgmap v327: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 102 B/s wr, 5 op/s 2023-12-14T18:38:20.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:19 smithi204 ceph-mon[43866]: pgmap v327: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 102 B/s wr, 5 op/s 2023-12-14T18:38:21.352 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:21.352 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.353 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:21 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:21 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.499 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.500 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:21.500 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:21 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:22.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:22 smithi031 ceph-mon[42085]: pgmap v328: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 5 op/s 2023-12-14T18:38:22.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:22 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi204 to 182.4M 2023-12-14T18:38:22.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:22 smithi031 ceph-mon[42085]: Unable to set osd_memory_target on smithi204 to 191346892: error parsing value: Value '191346892' is below minimum 939524096 2023-12-14T18:38:22.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:22 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:22.466 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:22 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:22.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:22 smithi176 ceph-mon[43506]: pgmap v328: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 5 op/s 2023-12-14T18:38:22.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:22 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi204 to 182.4M 2023-12-14T18:38:22.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:22 smithi176 ceph-mon[43506]: Unable to set osd_memory_target on smithi204 to 191346892: error parsing value: Value '191346892' is below minimum 939524096 2023-12-14T18:38:22.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:22 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:22.494 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:22 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:22.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:22 smithi204 ceph-mon[43866]: pgmap v328: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 187 B/s wr, 5 op/s 2023-12-14T18:38:22.591 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:22 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi204 to 182.4M 2023-12-14T18:38:22.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:22 smithi204 ceph-mon[43866]: Unable to set osd_memory_target on smithi204 to 191346892: error parsing value: Value '191346892' is below minimum 939524096 2023-12-14T18:38:22.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:22 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:22.592 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:22 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:23.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-12-14T18:38:23.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/3608185973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 msgr2=0x7f8a14107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/3608185973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 0x7f8a14107f70 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c010480 tx=0x7f8a0c010790 comp rx=0 tx=0).stop 2023-12-14T18:38:23.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/3608185973 shutdown_connections 2023-12-14T18:38:23.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/3608185973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 0x7f8a14107f70 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/3608185973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a14071e00 0x7f8a1410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/3608185973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14072260 0x7f8a1410d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.367+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/3608185973 >> 172.21.15.31:0/3608185973 conn(0x7f8a1406b7e0 msgr2=0x7f8a14071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.372+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/3608185973 shutdown_connections 2023-12-14T18:38:23.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.373+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/3608185973 wait complete. 2023-12-14T18:38:23.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.373+0000 7f8a1aa6d700 1 Processor -- start 2023-12-14T18:38:23.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 -- start start 2023-12-14T18:38:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 0x7f8a1419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a14072260 0x7f8a1419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 0x7f8a141a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141a1910 con 0x7f8a14107af0 2023-12-14T18:38:23.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141a1a80 con 0x7f8a14071e00 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a1aa6d700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141a1c40 con 0x7f8a14072260 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a18809700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 0x7f8a1419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a18809700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 0x7f8a1419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:52460/0 (socket says 172.21.15.31:52460) 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.374+0000 7f8a18809700 1 -- 172.21.15.31:0/2964115856 learned_addr learned my addr 172.21.15.31:0/2964115856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.375+0000 7f8a1900a700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 0x7f8a141a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.375+0000 7f8a18809700 1 -- 172.21.15.31:0/2964115856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a14072260 msgr2=0x7f8a1419ceb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:38:23.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.375+0000 7f8a18809700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a14072260 0x7f8a1419ceb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.375+0000 7f8a18809700 1 -- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 msgr2=0x7f8a141a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.375+0000 7f8a18809700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 0x7f8a141a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.375+0000 7f8a18809700 1 -- 172.21.15.31:0/2964115856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c00bc60 con 0x7f8a14071e00 2023-12-14T18:38:23.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.377+0000 7f8a18809700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 0x7f8a1419c970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c970 tx=0x7f8a0400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.380+0000 7f8a11ffb700 1 -- 172.21.15.31:0/2964115856 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04004220 con 0x7f8a14071e00 2023-12-14T18:38:23.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.380+0000 7f8a11ffb700 1 -- 172.21.15.31:0/2964115856 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a04007d60 con 0x7f8a14071e00 2023-12-14T18:38:23.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.380+0000 7f8a11ffb700 1 -- 172.21.15.31:0/2964115856 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003c90 con 0x7f8a14071e00 2023-12-14T18:38:23.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.380+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a14116920 con 0x7f8a14071e00 2023-12-14T18:38:23.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.380+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a14116790 con 0x7f8a14071e00 2023-12-14T18:38:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.388+0000 7f8a11ffb700 1 -- 172.21.15.31:0/2964115856 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8a04004380 con 0x7f8a14071e00 2023-12-14T18:38:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.389+0000 7f8a26bfbb80 1 Processor -- start 2023-12-14T18:38:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.389+0000 7f8a11ffb700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89fc0a14d0 0x7f89fc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.389+0000 7f8a11ffb700 1 -- 172.21.15.31:0/2964115856 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f8a040ccbc0 con 0x7f8a14071e00 2023-12-14T18:38:23.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.389+0000 7f8a1900a700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89fc0a14d0 0x7f89fc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.389+0000 7f8a26bfbb80 1 -- start start 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769d8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 0x559a769f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769f4b60 0x559a769f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a768c25e0 con 0x559a769d8c90 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76931870 con 0x559a769f4b60 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a7692ad00 con 0x559a769d8380 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a18809700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769d8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a18809700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769d8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:34892/0 (socket says 172.21.15.31:34892) 2023-12-14T18:38:23.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a18809700 1 -- 172.21.15.31:0/1348102879 learned_addr learned my addr 172.21.15.31:0/1348102879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.390+0000 7f8a1900a700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89fc0a14d0 0x7f89fc0a3980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009430 tx=0x7f8a0c005040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.393+0000 7f8a18809700 1 -- 172.21.15.31:0/1348102879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769f4b60 msgr2=0x559a769f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.393+0000 7f8a18809700 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769f4b60 0x559a769f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.393+0000 7f8a18809700 1 -- 172.21.15.31:0/1348102879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 msgr2=0x559a769f4620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.393+0000 7f8a18809700 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 0x559a769f4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.393+0000 7f8a18809700 1 -- 172.21.15.31:0/1348102879 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a767ac3b0 con 0x559a769d8380 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.397+0000 7f8a18809700 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769d8750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a040a1710 tx=0x7f8a04099690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e157d559959431a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.400+0000 7f89faffd700 1 -- 172.21.15.31:0/1348102879 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0409d820 con 0x559a769d8380 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.400+0000 7f89faffd700 1 -- 172.21.15.31:0/1348102879 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a040a1c40 con 0x559a769d8380 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.400+0000 7f89faffd700 1 -- 172.21.15.31:0/1348102879 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b9320 con 0x559a769d8380 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/1348102879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 msgr2=0x559a769d8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769d8750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8a040a1710 tx=0x7f8a04099690 comp rx=0 tx=0).stop 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/1348102879 shutdown_connections 2023-12-14T18:38:23.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769f4b60 0x559a769f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769d8750 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/1348102879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 0x559a769f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.401+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/1348102879 >> 172.21.15.31:0/1348102879 conn(0x559a769ceca0 msgr2=0x559a769d47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.402+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/1348102879 shutdown_connections 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.402+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/1348102879 wait complete. 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.402+0000 7f8a26bfbb80 1 -- start start 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.402+0000 7f8a26bfbb80 1 Processor -- start 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 0x559a769e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8c90 0x559a769e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a768c6f90 con 0x559a769d8380 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a768c6c30 con 0x559a769d8c90 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a767ae6a0 con 0x559a769f4b60 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.403+0000 7f8a26bfbb80 10 client.0 _ll_register_callbacks cb 0x559a768ea910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.404+0000 7f8a18809700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 0x559a769e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.404+0000 7f8a18809700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 0x559a769e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56070/0 (socket says 172.21.15.31:56070) 2023-12-14T18:38:23.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.404+0000 7f8a18809700 1 -- 172.21.15.31:0/2715599294 learned_addr learned my addr 172.21.15.31:0/2715599294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.407+0000 7f8a18809700 1 -- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 msgr2=0x559a769e0a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:38:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.407+0000 7f8a18809700 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769e0a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.407+0000 7f8a18809700 1 -- 172.21.15.31:0/2715599294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8c90 msgr2=0x559a769e5b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:38:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.407+0000 7f8a18809700 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8c90 0x559a769e5b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.407+0000 7f8a18809700 1 -- 172.21.15.31:0/2715599294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a14116790 con 0x559a769d8380 2023-12-14T18:38:23.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.408+0000 7f8a18809700 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 0x559a769e5650 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c940 tx=0x7f8a040c9ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b47599a65baf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.408+0000 7f89f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:38:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.408+0000 7f89f3fff700 1 -- 172.21.15.31:0/2715599294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b92e0 con 0x559a769d8380 2023-12-14T18:38:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.408+0000 7f89f3fff700 1 -- 172.21.15.31:0/2715599294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f8a04099a50 con 0x559a769d8380 2023-12-14T18:38:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.408+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a769cc2b0 con 0x559a769d8380 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.409+0000 7f89f3fff700 1 -- 172.21.15.31:0/2715599294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a04099c10 con 0x559a769d8380 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.410+0000 7f8a26bfbb80 10 client.17775 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.410+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559a768f1170 con 0x559a769d8380 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.414+0000 7f89f3fff700 1 -- 172.21.15.31:0/2715599294 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a769cc2b0 con 0x559a769d8380 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.414+0000 7f89f3fff700 1 -- 172.21.15.31:0/2715599294 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a040955e0 con 0x559a769d8380 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.415+0000 7f8a26bfbb80 10 client.17775 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.415+0000 7f8a26bfbb80 10 client.17775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:38:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.415+0000 7f8a26bfbb80 20 client.17775 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.415+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a769f0690 0x559a769f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.415+0000 7f8a26bfbb80 4 client.17775 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:38:23.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.415+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a76923ce0 con 0x559a769f0690 2023-12-14T18:38:23.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.429+0000 7f8a13fff700 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a769f0690 0x559a769f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.432+0000 7f8a13fff700 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a769f0690 0x559a769f2ac0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96bf73560b5ebf08 server_cookie=fdf8a3ec25c53e83 in_seq=0 out_seq=0 2023-12-14T18:38:23.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.433+0000 7f89f3fff700 10 client.17775 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:23.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.433+0000 7f89f3fff700 1 -- 172.21.15.31:0/2715599294 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a76923ce0 con 0x559a769f0690 2023-12-14T18:38:23.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.433+0000 7f89f3fff700 10 client.17775 handle_command_reply: tid=0 2023-12-14T18:38:23.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 client.17775 shutdown 2023-12-14T18:38:23.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 msgr2=0x559a769e5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 0x559a769e5650 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c940 tx=0x7f8a040c9ee0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 shutdown_connections 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a769f0690 0x559a769f2ac0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769e0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8380 0x559a769e5650 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2715599294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8c90 0x559a769e5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 >> 172.21.15.31:0/2715599294 conn(0x559a769ceca0 msgr2=0x559a769d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.434+0000 7f89f3fff700 0 client.17775 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:23.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.442+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 shutdown_connections 2023-12-14T18:38:23.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.442+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2715599294 wait complete. 2023-12-14T18:38:23.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.452+0000 7f8a26bfbb80 20 client.17775 trim_cache size 0 max 16384 2023-12-14T18:38:23.477 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.478 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:23 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:23 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 Processor -- start 2023-12-14T18:38:23.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 -- start start 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8380 0x559a769ed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 0x559a769f4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a769def90 con 0x559a769d8c90 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a769f4950 con 0x559a769d8380 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.491+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a769dc880 con 0x559a769f4b60 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a18809700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8380 0x559a769ed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a18809700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8380 0x559a769ed1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:52494/0 (socket says 172.21.15.31:52494) 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a18809700 1 -- 172.21.15.31:0/3163358253 learned_addr learned my addr 172.21.15.31:0/3163358253 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 -- 172.21.15.31:0/3163358253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8380 msgr2=0x559a769ed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8380 0x559a769ed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 -- 172.21.15.31:0/3163358253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 msgr2=0x559a769f4410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 0x559a769f4410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 -- 172.21.15.31:0/3163358253 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a768f0ec0 con 0x559a769f4b60 2023-12-14T18:38:23.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.492+0000 7f8a1900a700 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769ea2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c010c10 tx=0x7f8a0c010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a824985154a155a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.493+0000 7f89fb7fe700 1 -- 172.21.15.31:0/3163358253 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c010040 con 0x559a769f4b60 2023-12-14T18:38:23.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.493+0000 7f89fb7fe700 1 -- 172.21.15.31:0/3163358253 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a0c029040 con 0x559a769f4b60 2023-12-14T18:38:23.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.493+0000 7f89fb7fe700 1 -- 172.21.15.31:0/3163358253 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c009c60 con 0x559a769f4b60 2023-12-14T18:38:23.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.494+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/3163358253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 msgr2=0x559a769ea2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.494+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769ea2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c010c10 tx=0x7f8a0c010790 comp rx=0 tx=0).stop 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.496+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/3163358253 shutdown_connections 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.496+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769f4b60 0x559a769ea2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.496+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769d8380 0x559a769ed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.496+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/3163358253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769d8c90 0x559a769f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.496+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/3163358253 >> 172.21.15.31:0/3163358253 conn(0x559a769ceca0 msgr2=0x559a769eca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.498+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/3163358253 shutdown_connections 2023-12-14T18:38:23.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.498+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/3163358253 wait complete. 2023-12-14T18:38:23.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.498+0000 7f8a26bfbb80 1 -- start start 2023-12-14T18:38:23.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.498+0000 7f8a26bfbb80 1 Processor -- start 2023-12-14T18:38:23.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 0x559a769e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769e75e0 0x559a76a8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76a8ab80 con 0x559a769f4b60 2023-12-14T18:38:23.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76a8acf0 con 0x559a769e75e0 2023-12-14T18:38:23.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a76a8aeb0 con 0x559a769d8380 2023-12-14T18:38:23.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.499+0000 7f8a26bfbb80 10 client.0 _ll_register_callbacks cb 0x7f8a04095c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.501+0000 7f8a13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 0x559a769e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.501+0000 7f8a13fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 0x559a769e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56082/0 (socket says 172.21.15.31:56082) 2023-12-14T18:38:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.501+0000 7f8a13fff700 1 -- 172.21.15.31:0/2000466623 learned_addr learned my addr 172.21.15.31:0/2000466623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:23.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.501+0000 7f8a1900a700 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769e75e0 0x559a76a8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f8a13fff700 1 -- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 msgr2=0x559a769e6b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f8a13fff700 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769e6b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f8a13fff700 1 -- 172.21.15.31:0/2000466623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769e75e0 msgr2=0x559a76a8a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f8a13fff700 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769e75e0 0x559a76a8a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f8a13fff700 1 -- 172.21.15.31:0/2000466623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c01d040 con 0x559a769f4b60 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f8a13fff700 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 0x559a769e70a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8a1419df80 tx=0x7f8a0800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce748c7551ad81f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f89f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:38:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f89f3fff700 1 -- 172.21.15.31:0/2000466623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08016070 con 0x559a769f4b60 2023-12-14T18:38:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.502+0000 7f89f3fff700 1 -- 172.21.15.31:0/2000466623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f8a080161d0 con 0x559a769f4b60 2023-12-14T18:38:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.503+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a040a1c40 con 0x559a769f4b60 2023-12-14T18:38:23.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.503+0000 7f89f3fff700 1 -- 172.21.15.31:0/2000466623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a08016390 con 0x559a769f4b60 2023-12-14T18:38:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 10 client.17778 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:23.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559a768d79a0 con 0x559a769f4b60 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f89f3fff700 1 -- 172.21.15.31:0/2000466623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a040a1c40 con 0x559a769f4b60 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f89f3fff700 1 -- 172.21.15.31:0/2000466623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a08003af0 con 0x559a769f4b60 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 10 client.17778 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 10 client.17778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 20 client.17778 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a76a8c660 0x559a76a8ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 4 client.17778 mds_command: new command op to 24538 tid=0[{"prefix": "scrub status"}] 2023-12-14T18:38:23.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.504+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559a76923a00 con 0x559a76a8c660 2023-12-14T18:38:23.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.507+0000 7f8a18809700 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a76a8c660 0x559a76a8ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.512+0000 7f89f3fff700 10 client.17778 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.512+0000 7f8a18809700 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a76a8c660 0x559a76a8ea70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6c7fd130c63c74 server_cookie=2615fb46c1f0603e in_seq=0 out_seq=0 2023-12-14T18:38:23.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.512+0000 7f89f3fff700 1 -- 172.21.15.31:0/2000466623 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559a76923a00 con 0x559a76a8c660 2023-12-14T18:38:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.512+0000 7f89f3fff700 10 client.17778 handle_command_reply: tid=0 2023-12-14T18:38:23.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.512+0000 7f8a26bfbb80 1 client.17778 shutdown 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 msgr2=0x559a769e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 0x559a769e70a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f8a1419df80 tx=0x7f8a0800be70 comp rx=0 tx=0).stop 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 shutdown_connections 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x559a76a8c660 0x559a76a8ea70 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x559a769e75e0 0x559a76a8a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a769d8380 0x559a769e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 --2- 172.21.15.31:0/2000466623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559a769f4b60 0x559a769e70a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 >> 172.21.15.31:0/2000466623 conn(0x559a769ceca0 msgr2=0x559a769da230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f89f3fff700 0 client.17778 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 shutdown_connections 2023-12-14T18:38:23.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.521+0000 7f8a26bfbb80 1 -- 172.21.15.31:0/2000466623 wait complete. 2023-12-14T18:38:23.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.524+0000 7f8a26bfbb80 20 client.17778 trim_cache size 0 max 16384 2023-12-14T18:38:23.531 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:38:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "status": "no active scrubs running", 2023-12-14T18:38:23.531 INFO:teuthology.orchestra.run.smithi031.stdout: "scrubs": {} 2023-12-14T18:38:23.531 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:38:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.533+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89fc0a14d0 msgr2=0x7f89fc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.533+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89fc0a14d0 0x7f89fc0a3980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009430 tx=0x7f8a0c005040 comp rx=0 tx=0).stop 2023-12-14T18:38:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.533+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 msgr2=0x7f8a1419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.533+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 0x7f8a1419c970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c970 tx=0x7f8a0400cc80 comp rx=0 tx=0).stop 2023-12-14T18:38:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.540+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 shutdown_connections 2023-12-14T18:38:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.540+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a14107af0 0x7f8a141a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.541+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8a14071e00 0x7f8a1419c970 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.541+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f89fc0a14d0 0x7f89fc0a3980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.541+0000 7f8a1aa6d700 1 --2- 172.21.15.31:0/2964115856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a14072260 0x7f8a1419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.541+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 >> 172.21.15.31:0/2964115856 conn(0x7f8a1406b7e0 msgr2=0x7f8a1410ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.549+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 shutdown_connections 2023-12-14T18:38:23.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.550+0000 7f8a1aa6d700 1 -- 172.21.15.31:0/2964115856 wait complete. 2023-12-14T18:38:23.585 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-12-14T18:38:23.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:38:23.639 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.639 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.639 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.639 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.639 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.640 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.640 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.640 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:23 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:23.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.853+0000 7f6597015700 1 -- 172.21.15.31:0/1979560834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010d1d0 msgr2=0x7f6590106960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.853+0000 7f6597015700 1 --2- 172.21.15.31:0/1979560834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010d1d0 0x7f6590106960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f65840098f0 tx=0x7f6584009c00 comp rx=0 tx=0).stop 2023-12-14T18:38:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.857+0000 7f6597015700 1 -- 172.21.15.31:0/1979560834 shutdown_connections 2023-12-14T18:38:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.857+0000 7f6597015700 1 --2- 172.21.15.31:0/1979560834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6590107090 0x7f6590113680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.857+0000 7f6597015700 1 --2- 172.21.15.31:0/1979560834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f659010c8c0 0x7f659010cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.857+0000 7f6597015700 1 --2- 172.21.15.31:0/1979560834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010d1d0 0x7f6590106960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.857+0000 7f6597015700 1 -- 172.21.15.31:0/1979560834 >> 172.21.15.31:0/1979560834 conn(0x7f6590074870 msgr2=0x7f6590076c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:23.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.860+0000 7f6597015700 1 -- 172.21.15.31:0/1979560834 shutdown_connections 2023-12-14T18:38:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.860+0000 7f6597015700 1 -- 172.21.15.31:0/1979560834 wait complete. 2023-12-14T18:38:23.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 Processor -- start 2023-12-14T18:38:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 -- start start 2023-12-14T18:38:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6590107090 0x7f659007ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010c8c0 0x7f659007f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f659010d1d0 0x7f6590079d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659007a460 con 0x7f6590107090 2023-12-14T18:38:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659007a5d0 con 0x7f659010d1d0 2023-12-14T18:38:23.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.861+0000 7f6597015700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659007a790 con 0x7f659010c8c0 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f658ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010c8c0 0x7f659007f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f658ffff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010c8c0 0x7f659007f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:34926/0 (socket says 172.21.15.31:34926) 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f658ffff700 1 -- 172.21.15.31:0/2723069493 learned_addr learned my addr 172.21.15.31:0/2723069493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f65955b2700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f659010d1d0 0x7f6590079d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f65955b2700 1 -- 172.21.15.31:0/2723069493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010c8c0 msgr2=0x7f659007f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f65955b2700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010c8c0 0x7f659007f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f65955b2700 1 -- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6590107090 msgr2=0x7f659007ec00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:38:23.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f65955b2700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6590107090 0x7f659007ec00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.862+0000 7f65955b2700 1 -- 172.21.15.31:0/2723069493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6584009520 con 0x7f659010d1d0 2023-12-14T18:38:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.863+0000 7f65955b2700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f659010d1d0 0x7f6590079d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f658800efc0 tx=0x7f658800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.863+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6588018070 con 0x7f659010d1d0 2023-12-14T18:38:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.864+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f658800f040 con 0x7f659010d1d0 2023-12-14T18:38:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.864+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658800ce90 con 0x7f659010d1d0 2023-12-14T18:38:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.865+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6590116980 con 0x7f659010d1d0 2023-12-14T18:38:23.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.865+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65901167f0 con 0x7f659010d1d0 2023-12-14T18:38:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.870+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6590116b10 con 0x7f659010d1d0 2023-12-14T18:38:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.876+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6588003a00 con 0x7f659010d1d0 2023-12-14T18:38:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.877+0000 7f658dffb700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f65780a1490 0x7f65780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:23.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.877+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f65880ccb40 con 0x7f659010d1d0 2023-12-14T18:38:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.879+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6588095b40 con 0x7f659010d1d0 2023-12-14T18:38:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.879+0000 7f6594db1700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f65780a1490 0x7f65780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:23.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:23.881+0000 7f6594db1700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f65780a1490 0x7f65780a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6580009730 tx=0x7f6580006cb0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.330+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f659007b3b0 con 0x7f659010d1d0 2023-12-14T18:38:24.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.333+0000 7f658dffb700 1 -- 172.21.15.31:0/2723069493 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f6588099160 con 0x7f659010d1d0 2023-12-14T18:38:24.337 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:38:24.337 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:29.509045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:38:24.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.352+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f65780a1490 msgr2=0x7f65780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.352+0000 7f6597015700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f65780a1490 0x7f65780a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6580009730 tx=0x7f6580006cb0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.352+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f659010d1d0 msgr2=0x7f6590079d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.352+0000 7f6597015700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f659010d1d0 0x7f6590079d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f658800efc0 tx=0x7f658800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.353+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 shutdown_connections 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.353+0000 7f6597015700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f659010d1d0 0x7f6590079d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.353+0000 7f6597015700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6590107090 0x7f659007ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.353+0000 7f6597015700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f65780a1490 0x7f65780a3940 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6580009730 tx=0x7f6580006cb0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.353+0000 7f6597015700 1 --2- 172.21.15.31:0/2723069493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f659010c8c0 0x7f659007f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.353+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 >> 172.21.15.31:0/2723069493 conn(0x7f6590074870 msgr2=0x7f6590075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.358+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 shutdown_connections 2023-12-14T18:38:24.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.358+0000 7f6597015700 1 -- 172.21.15.31:0/2723069493 wait complete. 2023-12-14T18:38:24.371 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2023-12-14T18:38:24.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:24 smithi031 ceph-mon[42085]: pgmap v329: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-12-14T18:38:24.386 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:24 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi176 to 182.5M 2023-12-14T18:38:24.387 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:24 smithi031 ceph-mon[42085]: Unable to set osd_memory_target on smithi176 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2023-12-14T18:38:24.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-12-14T18:38:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:24 smithi176 ceph-mon[43506]: pgmap v329: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-12-14T18:38:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:24 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi176 to 182.5M 2023-12-14T18:38:24.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:24 smithi176 ceph-mon[43506]: Unable to set osd_memory_target on smithi176 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2023-12-14T18:38:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:24 smithi204 ceph-mon[43866]: pgmap v329: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-12-14T18:38:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:24 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi176 to 182.5M 2023-12-14T18:38:24.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:24 smithi204 ceph-mon[43866]: Unable to set osd_memory_target on smithi176 to 191379148: error parsing value: Value '191379148' is below minimum 939524096 2023-12-14T18:38:24.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.671+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3516577003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f4108c00 msgr2=0x7f38f4108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.671+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3516577003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f4108c00 0x7f38f4108fd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009990 tx=0x7f38e8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.682+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3516577003 shutdown_connections 2023-12-14T18:38:24.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.682+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3516577003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38f41121b0 0x7f38f4114590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.682+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3516577003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f4108c00 0x7f38f4108fd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.682+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3516577003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38f4109510 0x7f38f4111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.682+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3516577003 >> 172.21.15.31:0/3516577003 conn(0x7f38f406b7e0 msgr2=0x7f38f4071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.683+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3516577003 shutdown_connections 2023-12-14T18:38:24.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.690+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3516577003 wait complete. 2023-12-14T18:38:24.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.691+0000 7f38fa7f5700 1 Processor -- start 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.691+0000 7f38fa7f5700 1 -- start start 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.692+0000 7f38fa7f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38f4108c00 0x7f38f4111950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.692+0000 7f38fa7f5700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38f4109510 0x7f38f410e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.692+0000 7f38fa7f5700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 0x7f38f410eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.692+0000 7f38fa7f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f410f530 con 0x7f38f4108c00 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.692+0000 7f38fa7f5700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f410f6a0 con 0x7f38f4109510 2023-12-14T18:38:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.692+0000 7f38fa7f5700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f411b430 con 0x7f38f41121b0 2023-12-14T18:38:24.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 0x7f38f410eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 0x7f38f410eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53764/0 (socket says 172.21.15.31:53764) 2023-12-14T18:38:24.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 -- 172.21.15.31:0/3907144873 learned_addr learned my addr 172.21.15.31:0/3907144873 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 -- 172.21.15.31:0/3907144873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38f4109510 msgr2=0x7f38f410e960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:38:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38f4109510 0x7f38f410e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 -- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38f4108c00 msgr2=0x7f38f4111950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.693+0000 7f38f8d92700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38f4108c00 0x7f38f4111950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.694+0000 7f38f8d92700 1 -- 172.21.15.31:0/3907144873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80095f0 con 0x7f38f41121b0 2023-12-14T18:38:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.700+0000 7f38f8d92700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 0x7f38f410eea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38e400b4a0 tx=0x7f38e400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.701+0000 7f38f17fa700 1 -- 172.21.15.31:0/3907144873 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4011830 con 0x7f38f41121b0 2023-12-14T18:38:24.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.701+0000 7f38f17fa700 1 -- 172.21.15.31:0/3907144873 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e4011e50 con 0x7f38f41121b0 2023-12-14T18:38:24.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.701+0000 7f38f17fa700 1 -- 172.21.15.31:0/3907144873 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4010e70 con 0x7f38f41121b0 2023-12-14T18:38:24.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.702+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f4116880 con 0x7f38f41121b0 2023-12-14T18:38:24.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.702+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f41166f0 con 0x7f38f41121b0 2023-12-14T18:38:24.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.706+0000 7f3906983b80 1 Processor -- start 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 -- start start 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2bf380 0x55962d2bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2cab50 0x55962d2ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d1a95e0 con 0x55962d2bf380 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d218870 con 0x55962d2bfc90 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f3906983b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d211d00 con 0x55962d2cab50 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f38f37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2cab50 0x55962d2ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f38f37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2cab50 0x55962d2ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53772/0 (socket says 172.21.15.31:53772) 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f38f37fe700 1 -- 172.21.15.31:0/2737103217 learned_addr learned my addr 172.21.15.31:0/2737103217 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.707+0000 7f38f3fff700 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.708+0000 7f38f3fff700 1 -- 172.21.15.31:0/2737103217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2cab50 msgr2=0x55962d2ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.708+0000 7f38f3fff700 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2cab50 0x55962d2ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.708+0000 7f38f3fff700 1 -- 172.21.15.31:0/2737103217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2bf380 msgr2=0x55962d2bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.708+0000 7f38f3fff700 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2bf380 0x55962d2bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.708+0000 7f38f3fff700 1 -- 172.21.15.31:0/2737103217 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55962d0933b0 con 0x55962d2bfc90 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.708+0000 7f38f3fff700 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2ca610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009990 tx=0x7f38e8009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cb98d3b410b851 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.709+0000 7f38da7fc700 1 -- 172.21.15.31:0/2737103217 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e801d070 con 0x55962d2bfc90 2023-12-14T18:38:24.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.709+0000 7f38da7fc700 1 -- 172.21.15.31:0/2737103217 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e800be00 con 0x55962d2bfc90 2023-12-14T18:38:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.709+0000 7f38da7fc700 1 -- 172.21.15.31:0/2737103217 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80053b0 con 0x55962d2bfc90 2023-12-14T18:38:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.710+0000 7f3906983b80 1 -- 172.21.15.31:0/2737103217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 msgr2=0x55962d2ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.710+0000 7f3906983b80 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2ca610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009990 tx=0x7f38e8009f60 comp rx=0 tx=0).stop 2023-12-14T18:38:24.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.714+0000 7f3906983b80 1 -- 172.21.15.31:0/2737103217 shutdown_connections 2023-12-14T18:38:24.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.714+0000 7f3906983b80 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2cab50 0x55962d2ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.714+0000 7f3906983b80 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2bf380 0x55962d2bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.714+0000 7f3906983b80 1 --2- 172.21.15.31:0/2737103217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2ca610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.714+0000 7f3906983b80 1 -- 172.21.15.31:0/2737103217 >> 172.21.15.31:0/2737103217 conn(0x55962d2b5ca0 msgr2=0x55962d2bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.715+0000 7f38f17fa700 1 -- 172.21.15.31:0/3907144873 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f38e4010510 con 0x7f38f41121b0 2023-12-14T18:38:24.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.716+0000 7f38f17fa700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38dc0a14d0 0x7f38dc0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.716+0000 7f38f17fa700 1 -- 172.21.15.31:0/3907144873 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f38e40cca60 con 0x7f38f41121b0 2023-12-14T18:38:24.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.717+0000 7f38f3fff700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38dc0a14d0 0x7f38dc0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.718+0000 7f38f3fff700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38dc0a14d0 0x7f38dc0a3980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38e8003de0 tx=0x7f38e801a040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.723+0000 7f3906983b80 1 -- 172.21.15.31:0/2737103217 shutdown_connections 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.723+0000 7f3906983b80 1 -- 172.21.15.31:0/2737103217 wait complete. 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.724+0000 7f3906983b80 1 -- start start 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.724+0000 7f3906983b80 1 Processor -- start 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d2c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2ca6e0 0x55962d2c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 0x55962d2d6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d1adf90 con 0x55962d2c8560 2023-12-14T18:38:24.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d1adc30 con 0x55962d2ca6e0 2023-12-14T18:38:24.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d0956a0 con 0x55962d2bfc90 2023-12-14T18:38:24.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.725+0000 7f3906983b80 10 client.0 _ll_register_callbacks cb 0x55962d1d1910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:24.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.728+0000 7f38f8d92700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 0x55962d2d6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.728+0000 7f38f8d92700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 0x55962d2d6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42186/0 (socket says 172.21.15.31:42186) 2023-12-14T18:38:24.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.728+0000 7f38f8d92700 1 -- 172.21.15.31:0/1302005875 learned_addr learned my addr 172.21.15.31:0/1302005875 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:24.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f37fe700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2ca6e0 0x55962d2c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f3fff700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d2c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f8d92700 1 -- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 msgr2=0x55962d2c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f8d92700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d2c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f8d92700 1 -- 172.21.15.31:0/1302005875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2ca6e0 msgr2=0x55962d2c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f8d92700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2ca6e0 0x55962d2c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f8d92700 1 -- 172.21.15.31:0/1302005875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f4107e70 con 0x55962d2c8560 2023-12-14T18:38:24.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38f8d92700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 0x55962d2d6750 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f38e4096210 tx=0x7f38e4099100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76616f569425342d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.729+0000 7f38d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.730+0000 7f38d37fe700 1 -- 172.21.15.31:0/1302005875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4099c00 con 0x55962d2c8560 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.730+0000 7f38d37fe700 1 -- 172.21.15.31:0/1302005875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f38e40a1bf0 con 0x55962d2c8560 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.730+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55962d2b32b0 con 0x55962d2c8560 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.730+0000 7f38d37fe700 1 -- 172.21.15.31:0/1302005875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38e40b92c0 con 0x55962d2c8560 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.730+0000 7f3906983b80 10 client.17781 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:24.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.730+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55962d1d8170 con 0x55962d2c8560 2023-12-14T18:38:24.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f38d37fe700 1 -- 172.21.15.31:0/1302005875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55962d2b32b0 con 0x55962d2c8560 2023-12-14T18:38:24.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f38d37fe700 1 -- 172.21.15.31:0/1302005875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38e4095590 con 0x55962d2c8560 2023-12-14T18:38:24.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f3906983b80 10 client.17781 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:24.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f3906983b80 10 client.17781 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:38:24.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f3906983b80 20 client.17781 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f3906983b80 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d2daf50 0x55962d2dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f3906983b80 4 client.17781 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:38:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.731+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55962d20ace0 con 0x55962d2daf50 2023-12-14T18:38:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.732+0000 7f38f3fff700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d2daf50 0x55962d2dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.732+0000 7f38d37fe700 10 client.17781 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:38:24.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.732+0000 7f38f3fff700 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d2daf50 0x55962d2dd360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cabfac88b0cdf95 server_cookie=cbd9c3fc5c403e14 in_seq=0 out_seq=0 2023-12-14T18:38:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f38d37fe700 1 -- 172.21.15.31:0/1302005875 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55962d20ace0 con 0x55962d2daf50 2023-12-14T18:38:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f38d37fe700 10 client.17781 handle_command_reply: tid=0 2023-12-14T18:38:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 client.17781 shutdown 2023-12-14T18:38:24.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 msgr2=0x55962d2d6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 0x55962d2d6750 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f38e4096210 tx=0x7f38e4099100 comp rx=0 tx=0).stop 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 shutdown_connections 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d2daf50 0x55962d2dd360 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2c8560 0x55962d2d6750 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d2c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 --2- 172.21.15.31:0/1302005875 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2ca6e0 0x55962d2c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 >> 172.21.15.31:0/1302005875 conn(0x55962d2b5ca0 msgr2=0x55962d2bb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f38d37fe700 0 client.17781 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 shutdown_connections 2023-12-14T18:38:24.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.733+0000 7f3906983b80 1 -- 172.21.15.31:0/1302005875 wait complete. 2023-12-14T18:38:24.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.734+0000 7f3906983b80 20 client.17781 trim_cache size 0 max 16384 2023-12-14T18:38:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 Processor -- start 2023-12-14T18:38:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 -- start start 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 0x55962d2ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 0x55962d2c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d2c0330 con 0x55962d2ca6e0 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.760+0000 7f3906983b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d2c5f30 con 0x55962d2bfc90 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f3906983b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d2c3780 con 0x55962d2c8560 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 0x55962d2ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 0x55962d2ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53794/0 (socket says 172.21.15.31:53794) 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 -- 172.21.15.31:0/216308833 learned_addr learned my addr 172.21.15.31:0/216308833 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f3fff700 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 -- 172.21.15.31:0/216308833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 msgr2=0x55962d2d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 -- 172.21.15.31:0/216308833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 msgr2=0x55962d2c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 0x55962d2c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.761+0000 7f38f37fe700 1 -- 172.21.15.31:0/216308833 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55962d1d7ec0 con 0x55962d2c8560 2023-12-14T18:38:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.764+0000 7f38f37fe700 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 0x55962d2ba9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00bec0 tx=0x7f38ec006c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ff178862c8544f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.765+0000 7f38daffd700 1 -- 172.21.15.31:0/216308833 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec039620 con 0x55962d2c8560 2023-12-14T18:38:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.766+0000 7f38daffd700 1 -- 172.21.15.31:0/216308833 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38ec032050 con 0x55962d2c8560 2023-12-14T18:38:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.766+0000 7f38daffd700 1 -- 172.21.15.31:0/216308833 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec037950 con 0x55962d2c8560 2023-12-14T18:38:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 -- 172.21.15.31:0/216308833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 msgr2=0x55962d2ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 0x55962d2ba9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00bec0 tx=0x7f38ec006c70 comp rx=0 tx=0).stop 2023-12-14T18:38:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 -- 172.21.15.31:0/216308833 shutdown_connections 2023-12-14T18:38:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 0x55962d2c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2bfc90 0x55962d2d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 --2- 172.21.15.31:0/216308833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2c8560 0x55962d2ba9d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 -- 172.21.15.31:0/216308833 >> 172.21.15.31:0/216308833 conn(0x55962d2b5ca0 msgr2=0x55962d2d71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 -- 172.21.15.31:0/216308833 shutdown_connections 2023-12-14T18:38:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 -- 172.21.15.31:0/216308833 wait complete. 2023-12-14T18:38:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.767+0000 7f3906983b80 1 -- start start 2023-12-14T18:38:24.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 Processor -- start 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d322fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2c8560 0x55962d323510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 0x55962d327f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d3284c0 con 0x55962d2ca6e0 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d328630 con 0x55962d2c8560 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962d3287f0 con 0x55962d2bfc90 2023-12-14T18:38:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.768+0000 7f3906983b80 10 client.0 _ll_register_callbacks cb 0x7f38e4099570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.769+0000 7f38f3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d322fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.769+0000 7f38f3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d322fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53800/0 (socket says 172.21.15.31:53800) 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.769+0000 7f38f3fff700 1 -- 172.21.15.31:0/1973461681 learned_addr learned my addr 172.21.15.31:0/1973461681 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.770+0000 7f38f3fff700 1 -- 172.21.15.31:0/1973461681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2c8560 msgr2=0x55962d323510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.770+0000 7f38f3fff700 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2c8560 0x55962d323510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.770+0000 7f38f3fff700 1 -- 172.21.15.31:0/1973461681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 msgr2=0x55962d327f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.770+0000 7f38f3fff700 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 0x55962d327f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.770+0000 7f38f3fff700 1 -- 172.21.15.31:0/1973461681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec00bbc0 con 0x55962d2bfc90 2023-12-14T18:38:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.770+0000 7f38f3fff700 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d322fd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f38e8003eb0 tx=0x7f38e8003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb4748b19cd1ed57 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.772+0000 7f38d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:38:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.772+0000 7f38d37fe700 1 -- 172.21.15.31:0/1973461681 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8054040 con 0x55962d2bfc90 2023-12-14T18:38:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.772+0000 7f38d37fe700 1 -- 172.21.15.31:0/1973461681 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f38e8005690 con 0x55962d2bfc90 2023-12-14T18:38:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.772+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38e800be00 con 0x55962d2bfc90 2023-12-14T18:38:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.772+0000 7f38d37fe700 1 -- 172.21.15.31:0/1973461681 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38e801d040 con 0x55962d2bfc90 2023-12-14T18:38:24.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.775+0000 7f3906983b80 10 client.27559 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:24.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.775+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55962d1be9a0 con 0x55962d2bfc90 2023-12-14T18:38:24.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f38d37fe700 1 -- 172.21.15.31:0/1973461681 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e800be00 con 0x55962d2bfc90 2023-12-14T18:38:24.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f38d37fe700 1 -- 172.21.15.31:0/1973461681 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f38e800f750 con 0x55962d2bfc90 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f3906983b80 10 client.27559 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f3906983b80 10 client.27559 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f3906983b80 20 client.27559 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f3906983b80 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d32fa90 0x55962d331ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f3906983b80 4 client.27559 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 --> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55962d20aa00 con 0x55962d32fa90 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.777+0000 7f38f37fe700 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d32fa90 0x55962d331ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:24.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.781+0000 7f38f37fe700 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d32fa90 0x55962d331ec0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e64ffb3d80b2d6a8 server_cookie=22627ee357677d2f in_seq=0 out_seq=0 2023-12-14T18:38:24.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.785+0000 7f38d37fe700 10 client.27559 ms_handle_connect on v2:172.21.15.204:6836/1057123586 2023-12-14T18:38:24.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.785+0000 7f38d37fe700 1 -- 172.21.15.31:0/1973461681 <== mds.2 v2:172.21.15.204:6836/1057123586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55962d20aa00 con 0x55962d32fa90 2023-12-14T18:38:24.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.785+0000 7f38d37fe700 10 client.27559 handle_command_reply: tid=0 2023-12-14T18:38:24.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.785+0000 7f3906983b80 1 client.27559 shutdown 2023-12-14T18:38:24.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 msgr2=0x55962d322fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d322fd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f38e8003eb0 tx=0x7f38e8003ee0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 shutdown_connections 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:6836/1057123586,v1:172.21.15.204:6837/1057123586] conn(0x55962d32fa90 0x55962d331ec0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962d2ca6e0 0x55962d327f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55962d2bfc90 0x55962d322fd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 --2- 172.21.15.31:0/1973461681 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55962d2c8560 0x55962d323510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.786+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 >> 172.21.15.31:0/1973461681 conn(0x55962d2b5ca0 msgr2=0x55962d2ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.788+0000 7f38d37fe700 0 client.27559 ms_handle_reset on v2:172.21.15.204:6836/1057123586 2023-12-14T18:38:24.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.791+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 shutdown_connections 2023-12-14T18:38:24.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.791+0000 7f3906983b80 1 -- 172.21.15.31:0/1973461681 wait complete. 2023-12-14T18:38:24.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.792+0000 7f3906983b80 20 client.27559 trim_cache size 0 max 16384 2023-12-14T18:38:24.794 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:38:24.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.803+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38dc0a14d0 msgr2=0x7f38dc0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.803+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38dc0a14d0 0x7f38dc0a3980 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38e8003de0 tx=0x7f38e801a040 comp rx=0 tx=0).stop 2023-12-14T18:38:24.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.803+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 msgr2=0x7f38f410eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:24.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.803+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 0x7f38f410eea0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f38e400b4a0 tx=0x7f38e400b860 comp rx=0 tx=0).stop 2023-12-14T18:38:24.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.812+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 shutdown_connections 2023-12-14T18:38:24.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.812+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f38dc0a14d0 0x7f38dc0a3980 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.812+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38f41121b0 0x7f38f410eea0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.812+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38f4108c00 0x7f38f4111950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.812+0000 7f38fa7f5700 1 --2- 172.21.15.31:0/3907144873 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f38f4109510 0x7f38f410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:24.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.812+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 >> 172.21.15.31:0/3907144873 conn(0x7f38f406b7e0 msgr2=0x7f38f406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.813+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 shutdown_connections 2023-12-14T18:38:24.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:24.813+0000 7f38fa7f5700 1 -- 172.21.15.31:0/3907144873 wait complete. 2023-12-14T18:38:24.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-12-14T18:38:25.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 -- 172.21.15.31:0/1977181592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb78103060 msgr2=0x7fdb78107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/1977181592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb78103060 0x7fdb78107020 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009920 tx=0x7fdb68009c30 comp rx=0 tx=0).stop 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 -- 172.21.15.31:0/1977181592 shutdown_connections 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/1977181592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/1977181592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb781026c0 0x7fdb78102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/1977181592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb78103060 0x7fdb78107020 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 -- 172.21.15.31:0/1977181592 >> 172.21.15.31:0/1977181592 conn(0x7fdb780fdf90 msgr2=0x7fdb781003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.114+0000 7fdb7da4f700 1 -- 172.21.15.31:0/1977181592 shutdown_connections 2023-12-14T18:38:25.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.115+0000 7fdb7da4f700 1 -- 172.21.15.31:0/1977181592 wait complete. 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.115+0000 7fdb7da4f700 1 Processor -- start 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 -- start start 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb781026c0 0x7fdb7806fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb78103060 0x7fdb78070230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7806aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7806b520 con 0x7fdb78103060 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7806b690 con 0x7fdb78107700 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb7da4f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7806b850 con 0x7fdb781026c0 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.116+0000 7fdb777fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7806aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7806aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41688/0 (socket says 172.21.15.31:41688) 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 -- 172.21.15.31:0/3279570242 learned_addr learned my addr 172.21.15.31:0/3279570242 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb781026c0 0x7fdb7806fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 -- 172.21.15.31:0/3279570242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb781026c0 msgr2=0x7fdb7806fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb781026c0 0x7fdb7806fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 -- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb78103060 msgr2=0x7fdb78070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb78103060 0x7fdb78070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.117+0000 7fdb777fe700 1 -- 172.21.15.31:0/3279570242 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68009520 con 0x7fdb78107700 2023-12-14T18:38:25.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.119+0000 7fdb777fe700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7806aee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b970 tx=0x7fdb7000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.127+0000 7fdb7ca4d700 1 -- 172.21.15.31:0/3279570242 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000c9b0 con 0x7fdb78107700 2023-12-14T18:38:25.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.128+0000 7fdb7ca4d700 1 -- 172.21.15.31:0/3279570242 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb70014440 con 0x7fdb78107700 2023-12-14T18:38:25.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.128+0000 7fdb7ca4d700 1 -- 172.21.15.31:0/3279570242 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700075d0 con 0x7fdb78107700 2023-12-14T18:38:25.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.128+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb781125c0 con 0x7fdb78107700 2023-12-14T18:38:25.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.128+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb78112430 con 0x7fdb78107700 2023-12-14T18:38:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.134+0000 7fdb7ca4d700 1 -- 172.21.15.31:0/3279570242 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdb700145b0 con 0x7fdb78107700 2023-12-14T18:38:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.146+0000 7fdb7ca4d700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdb640a14d0 0x7fdb640a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.146+0000 7fdb7ca4d700 1 -- 172.21.15.31:0/3279570242 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fdb70006e40 con 0x7fdb78107700 2023-12-14T18:38:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.147+0000 7fdb89bddb80 1 Processor -- start 2023-12-14T18:38:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 -- start start 2023-12-14T18:38:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 0x555d788f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d78902e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78905740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d787d95e0 con 0x555d78903360 2023-12-14T18:38:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d78848870 con 0x555d788f7b90 2023-12-14T18:38:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.154+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d78841d00 con 0x555d788f84a0 2023-12-14T18:38:25.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d78902e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d78902e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53834/0 (socket says 172.21.15.31:53834) 2023-12-14T18:38:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 -- 172.21.15.31:0/2399204977 learned_addr learned my addr 172.21.15.31:0/2399204977 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 -- 172.21.15.31:0/2399204977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 msgr2=0x555d788f7f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 0x555d788f7f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 -- 172.21.15.31:0/2399204977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 msgr2=0x555d78905740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:25.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78905740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.156+0000 7fdb767fc700 1 -- 172.21.15.31:0/2399204977 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d786c33b0 con 0x555d788f84a0 2023-12-14T18:38:25.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.157+0000 7fdb767fc700 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d78902e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009920 tx=0x7fdb68009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6e94ab4c0b7007 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:25 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2723069493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:38:25.162 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:25 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:25.163 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:25 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.157+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdb640a14d0 0x7fdb640a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.158+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdb640a14d0 0x7fdb640a3980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005fd0 tx=0x7fdb60009310 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.162+0000 7fdb6dffb700 1 -- 172.21.15.31:0/2399204977 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6801c070 con 0x555d788f84a0 2023-12-14T18:38:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.162+0000 7fdb6dffb700 1 -- 172.21.15.31:0/2399204977 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb6800b560 con 0x555d788f84a0 2023-12-14T18:38:25.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.162+0000 7fdb6dffb700 1 -- 172.21.15.31:0/2399204977 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68005300 con 0x555d788f84a0 2023-12-14T18:38:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.165+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2399204977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 msgr2=0x555d78902e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.165+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d78902e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009920 tx=0x7fdb68009480 comp rx=0 tx=0).stop 2023-12-14T18:38:25.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.166+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2399204977 shutdown_connections 2023-12-14T18:38:25.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.166+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78905740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.166+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 0x555d788f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.166+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2399204977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d78902e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.166+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2399204977 >> 172.21.15.31:0/2399204977 conn(0x555d788e5ca0 msgr2=0x555d788eb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.167+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2399204977 shutdown_connections 2023-12-14T18:38:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.168+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2399204977 wait complete. 2023-12-14T18:38:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.168+0000 7fdb89bddb80 1 -- start start 2023-12-14T18:38:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 Processor -- start 2023-12-14T18:38:25.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 0x555d78902ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d788fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78900580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d787ddf90 con 0x555d78903360 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d787ddc30 con 0x555d788f7b90 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d786c56a0 con 0x555d788f84a0 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.169+0000 7fdb89bddb80 10 client.0 _ll_register_callbacks cb 0x555d78801910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.170+0000 7fdb777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78900580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.170+0000 7fdb777fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78900580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42230/0 (socket says 172.21.15.31:42230) 2023-12-14T18:38:25.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.170+0000 7fdb777fe700 1 -- 172.21.15.31:0/743788993 learned_addr learned my addr 172.21.15.31:0/743788993 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.170+0000 7fdb767fc700 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d788fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb777fe700 1 -- 172.21.15.31:0/743788993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 msgr2=0x555d788fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb777fe700 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d788fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb777fe700 1 -- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 msgr2=0x555d78902ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb777fe700 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 0x555d78902ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb777fe700 1 -- 172.21.15.31:0/743788993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7810fb10 con 0x555d78903360 2023-12-14T18:38:25.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb777fe700 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78900580 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000be50 tx=0x7fdb700c9f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d35ec57898cda1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.172+0000 7fdb56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:38:25.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.173+0000 7fdb56ffd700 1 -- 172.21.15.31:0/743788993 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70098950 con 0x555d78903360 2023-12-14T18:38:25.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.173+0000 7fdb56ffd700 1 -- 172.21.15.31:0/743788993 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fdb70098ab0 con 0x555d78903360 2023-12-14T18:38:25.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.173+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d788e32b0 con 0x555d78903360 2023-12-14T18:38:25.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.173+0000 7fdb56ffd700 1 -- 172.21.15.31:0/743788993 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb700a1420 con 0x555d78903360 2023-12-14T18:38:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.181+0000 7fdb89bddb80 10 client.17787 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:25.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.181+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555d78808170 con 0x555d78903360 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.187+0000 7fdb56ffd700 1 -- 172.21.15.31:0/743788993 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d788e32b0 con 0x555d78903360 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.187+0000 7fdb56ffd700 1 -- 172.21.15.31:0/743788993 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb70095250 con 0x555d78903360 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.188+0000 7fdb89bddb80 10 client.17787 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.188+0000 7fdb89bddb80 10 client.17787 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.188+0000 7fdb89bddb80 20 client.17787 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.188+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d788f0eb0 0x555d788f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.188+0000 7fdb89bddb80 4 client.17787 mds_command: new command op to 24536 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:38:25.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.188+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d7883ace0 con 0x555d788f0eb0 2023-12-14T18:38:25.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.194+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d788f0eb0 0x555d788f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.204+0000 7fdb56ffd700 10 client.17787 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:38:25.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.204+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d788f0eb0 0x555d788f32c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83f6e494fc58a4fa server_cookie=b94fcb0965faa0d in_seq=0 out_seq=0 2023-12-14T18:38:25.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.209+0000 7fdb56ffd700 1 -- 172.21.15.31:0/743788993 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d7883ace0 con 0x555d788f0eb0 2023-12-14T18:38:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.209+0000 7fdb56ffd700 10 client.17787 handle_command_reply: tid=0 2023-12-14T18:38:25.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.211+0000 7fdb89bddb80 1 client.17787 shutdown 2023-12-14T18:38:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 msgr2=0x555d78900580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78900580 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000be50 tx=0x7fdb700c9f30 comp rx=0 tx=0).stop 2023-12-14T18:38:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 shutdown_connections 2023-12-14T18:38:25.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d78903360 0x555d78900580 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f7b90 0x555d78902ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d788f0eb0 0x555d788f32c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/743788993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d788f84a0 0x555d788fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.220+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 >> 172.21.15.31:0/743788993 conn(0x555d788e5ca0 msgr2=0x555d788eb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.225+0000 7fdb56ffd700 0 client.17787 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:38:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.227+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 shutdown_connections 2023-12-14T18:38:25.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.227+0000 7fdb89bddb80 1 -- 172.21.15.31:0/743788993 wait complete. 2023-12-14T18:38:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.234+0000 7fdb89bddb80 20 client.17787 trim_cache size 0 max 16384 2023-12-14T18:38:25.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 Processor -- start 2023-12-14T18:38:25.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 -- start start 2023-12-14T18:38:25.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d788faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d788f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d788ee590 con 0x555d788f7b90 2023-12-14T18:38:25.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7890ebf0 con 0x555d788f84a0 2023-12-14T18:38:25.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.260+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d788fbe30 con 0x555d78903360 2023-12-14T18:38:25.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41702/0 (socket says 172.21.15.31:41702) 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 -- 172.21.15.31:0/407321113 learned_addr learned my addr 172.21.15.31:0/407321113 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb777fe700 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d788f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d788faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 -- 172.21.15.31:0/407321113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 msgr2=0x555d788f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d788f2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 -- 172.21.15.31:0/407321113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 msgr2=0x555d788faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d788faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.261+0000 7fdb767fc700 1 -- 172.21.15.31:0/407321113 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d788eaca0 con 0x555d788f84a0 2023-12-14T18:38:25.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.262+0000 7fdb767fc700 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb680098f0 tx=0x7fdb6800f750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d6cc68a2b0aa35f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.262+0000 7fdb6e7fc700 1 -- 172.21.15.31:0/407321113 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6801c050 con 0x555d788f84a0 2023-12-14T18:38:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.262+0000 7fdb6e7fc700 1 -- 172.21.15.31:0/407321113 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb6801d040 con 0x555d788f84a0 2023-12-14T18:38:25.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.262+0000 7fdb6e7fc700 1 -- 172.21.15.31:0/407321113 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68003680 con 0x555d788f84a0 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 -- 172.21.15.31:0/407321113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 msgr2=0x555d78909d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909d40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb680098f0 tx=0x7fdb6800f750 comp rx=0 tx=0).stop 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 -- 172.21.15.31:0/407321113 shutdown_connections 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d788f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d788faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/407321113 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909d40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 -- 172.21.15.31:0/407321113 >> 172.21.15.31:0/407321113 conn(0x555d788e5ca0 msgr2=0x555d7890e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 -- 172.21.15.31:0/407321113 shutdown_connections 2023-12-14T18:38:25.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.263+0000 7fdb89bddb80 1 -- 172.21.15.31:0/407321113 wait complete. 2023-12-14T18:38:25.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.264+0000 7fdb89bddb80 1 -- start start 2023-12-14T18:38:25.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.264+0000 7fdb89bddb80 1 Processor -- start 2023-12-14T18:38:25.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d78908ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d7896df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d78909ba0 con 0x555d788f7b90 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d78909d10 con 0x555d788f84a0 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d7896e4a0 con 0x555d78903360 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb89bddb80 10 client.0 _ll_register_callbacks cb 0x7fdb70098fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d78908ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb76ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d78908ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42246/0 (socket says 172.21.15.31:42246) 2023-12-14T18:38:25.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb76ffd700 1 -- 172.21.15.31:0/2725481697 learned_addr learned my addr 172.21.15.31:0/2725481697 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb777fe700 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d7896df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.265+0000 7fdb777fe700 1 -- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 msgr2=0x555d78909430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.266+0000 7fdb777fe700 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.266+0000 7fdb777fe700 1 -- 172.21.15.31:0/2725481697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 msgr2=0x555d78908ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.266+0000 7fdb777fe700 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d78908ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.266+0000 7fdb777fe700 1 -- 172.21.15.31:0/2725481697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d786c33b0 con 0x555d78903360 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.266+0000 7fdb777fe700 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d7896df60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb70014c10 tx=0x7fdb70004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852a98034a1c930 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.267+0000 7fdb56ffd700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.267+0000 7fdb56ffd700 1 -- 172.21.15.31:0/2725481697 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700a1620 con 0x555d78903360 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.267+0000 7fdb56ffd700 1 -- 172.21.15.31:0/2725481697 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fdb700b1e40 con 0x555d78903360 2023-12-14T18:38:25.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.267+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb6800b560 con 0x555d78903360 2023-12-14T18:38:25.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.268+0000 7fdb56ffd700 1 -- 172.21.15.31:0/2725481697 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb70098950 con 0x555d78903360 2023-12-14T18:38:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.274+0000 7fdb89bddb80 10 client.27577 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.274+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555d7866fb60 con 0x555d78903360 2023-12-14T18:38:25.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.278+0000 7fdb56ffd700 1 -- 172.21.15.31:0/2725481697 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb6800b560 con 0x555d78903360 2023-12-14T18:38:25.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.278+0000 7fdb56ffd700 1 -- 172.21.15.31:0/2725481697 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb700bb020 con 0x555d78903360 2023-12-14T18:38:25.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.280+0000 7fdb89bddb80 10 client.27577 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.280+0000 7fdb89bddb80 10 client.27577 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2023-12-14T18:38:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.280+0000 7fdb89bddb80 20 client.27577 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.280+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d7896f6c0 0x555d78971aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.280+0000 7fdb89bddb80 4 client.27577 mds_command: new command op to 24536 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:38:25.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.280+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 --> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d7883aa00 con 0x555d7896f6c0 2023-12-14T18:38:25.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.284+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d7896f6c0 0x555d78971aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb76ffd700 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d7896f6c0 0x555d78971aa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e0487453e886d81 server_cookie=ca577ebd4328b9ac in_seq=0 out_seq=0 2023-12-14T18:38:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb56ffd700 10 client.27577 ms_handle_connect on v2:172.21.15.176:6838/2954758257 2023-12-14T18:38:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb56ffd700 1 -- 172.21.15.31:0/2725481697 <== mds.1 v2:172.21.15.176:6838/2954758257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555d7883aa00 con 0x555d7896f6c0 2023-12-14T18:38:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb56ffd700 10 client.27577 handle_command_reply: tid=0 2023-12-14T18:38:25.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb89bddb80 1 client.27577 shutdown 2023-12-14T18:38:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 msgr2=0x555d7896df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.288+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d7896df60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb70014c10 tx=0x7fdb70004be0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 shutdown_connections 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:6838/2954758257,v1:172.21.15.176:6839/2954758257] conn(0x555d7896f6c0 0x555d78971aa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d78903360 0x555d7896df60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x555d788f7b90 0x555d78908ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb89bddb80 1 --2- 172.21.15.31:0/2725481697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x555d788f84a0 0x555d78909430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 >> 172.21.15.31:0/2725481697 conn(0x555d788e5ca0 msgr2=0x555d788f2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.293+0000 7fdb56ffd700 0 client.27577 ms_handle_reset on v2:172.21.15.176:6838/2954758257 2023-12-14T18:38:25.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.295+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 shutdown_connections 2023-12-14T18:38:25.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.295+0000 7fdb89bddb80 1 -- 172.21.15.31:0/2725481697 wait complete. 2023-12-14T18:38:25.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.299+0000 7fdb89bddb80 20 client.27577 trim_cache size 0 max 16384 2023-12-14T18:38:25.305 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:38:25.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.307+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdb640a14d0 msgr2=0x7fdb640a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.307+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdb640a14d0 0x7fdb640a3980 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdb60005fd0 tx=0x7fdb60009310 comp rx=0 tx=0).stop 2023-12-14T18:38:25.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.307+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 msgr2=0x7fdb7806aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.307+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7806aee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b970 tx=0x7fdb7000bd30 comp rx=0 tx=0).stop 2023-12-14T18:38:25.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.311+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 shutdown_connections 2023-12-14T18:38:25.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.311+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb78107700 0x7fdb7806aee0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.311+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fdb640a14d0 0x7fdb640a3980 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.311+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb781026c0 0x7fdb7806fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.311+0000 7fdb7da4f700 1 --2- 172.21.15.31:0/3279570242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb78103060 0x7fdb78070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.311+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 >> 172.21.15.31:0/3279570242 conn(0x7fdb780fdf90 msgr2=0x7fdb780feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.317+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 shutdown_connections 2023-12-14T18:38:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.319+0000 7fdb7da4f700 1 -- 172.21.15.31:0/3279570242 wait complete. 2023-12-14T18:38:25.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-12-14T18:38:25.491 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:25 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2723069493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:38:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:25 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:25.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:25 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.642+0000 7f7aae8ba700 1 -- 172.21.15.31:0/1825707781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa80736d0 msgr2=0x7f7aa8113630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.642+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/1825707781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa80736d0 0x7f7aa8113630 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000b0a0 tx=0x7f7aa000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.646+0000 7f7aae8ba700 1 -- 172.21.15.31:0/1825707781 shutdown_connections 2023-12-14T18:38:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.646+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/1825707781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa80736d0 0x7f7aa8113630 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.646+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/1825707781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7aa8074220 0x7f7aa80745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.646+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/1825707781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa8072d20 0x7f7aa8073190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.646+0000 7f7aae8ba700 1 -- 172.21.15.31:0/1825707781 >> 172.21.15.31:0/1825707781 conn(0x7f7aa8077570 msgr2=0x7f7aa8077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.657+0000 7f7aae8ba700 1 -- 172.21.15.31:0/1825707781 shutdown_connections 2023-12-14T18:38:25.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.662+0000 7f7aae8ba700 1 -- 172.21.15.31:0/1825707781 wait complete. 2023-12-14T18:38:25.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.663+0000 7f7aae8ba700 1 Processor -- start 2023-12-14T18:38:25.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 -- start start 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7aa8072d20 0x7f7aa8072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 0x7f7aa8072540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa8074220 0x7f7aa806bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa806c7b0 con 0x7f7aa80736d0 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa806c920 con 0x7f7aa8074220 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.664+0000 7f7aae8ba700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa806cae0 con 0x7f7aa8072d20 2023-12-14T18:38:25.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 0x7f7aa8072540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 0x7f7aa8072540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42254/0 (socket says 172.21.15.31:42254) 2023-12-14T18:38:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 -- 172.21.15.31:0/3929182540 learned_addr learned my addr 172.21.15.31:0/3929182540 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 -- 172.21.15.31:0/3929182540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7aa8072d20 msgr2=0x7f7aa8072000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:38:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7aa8072d20 0x7f7aa8072000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 -- 172.21.15.31:0/3929182540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa8074220 msgr2=0x7f7aa806bfe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:38:25.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa8074220 0x7f7aa806bfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.666+0000 7f7aa77fe700 1 -- 172.21.15.31:0/3929182540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa0009d00 con 0x7f7aa80736d0 2023-12-14T18:38:25.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.667+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 0x7f7aa8072540 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b550 tx=0x7f7a9800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.670+0000 7f7aa57fa700 1 -- 172.21.15.31:0/3929182540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98011830 con 0x7f7aa80736d0 2023-12-14T18:38:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.670+0000 7f7aa57fa700 1 -- 172.21.15.31:0/3929182540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a98011e50 con 0x7f7aa80736d0 2023-12-14T18:38:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.670+0000 7f7aa57fa700 1 -- 172.21.15.31:0/3929182540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800f2e0 con 0x7f7aa80736d0 2023-12-14T18:38:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.670+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa806ae20 con 0x7f7aa80736d0 2023-12-14T18:38:25.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.670+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa806ac90 con 0x7f7aa80736d0 2023-12-14T18:38:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:25 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2723069493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:38:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:25 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:25.673 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:25 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:25.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.679+0000 7f7abaa48b80 1 Processor -- start 2023-12-14T18:38:25.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 -- start start 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7aa57fa700 1 -- 172.21.15.31:0/3929182540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7a9800f440 con 0x7f7aa80736d0 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975b380 0x55d9b975b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975bc90 0x55d9b9766610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9766b50 0x55d9b9768f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b96455e0 con 0x55d9b975bc90 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b96b4870 con 0x55d9b9766b50 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b96add00 con 0x55d9b975b380 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7aace57700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975b380 0x55d9b975b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7aace57700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975b380 0x55d9b975b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53878/0 (socket says 172.21.15.31:53878) 2023-12-14T18:38:25.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7aace57700 1 -- 172.21.15.31:0/1505802358 learned_addr learned my addr 172.21.15.31:0/1505802358 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.680+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9766b50 0x55d9b9768f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa77fe700 1 -- 172.21.15.31:0/1505802358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975b380 msgr2=0x55d9b975b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975b380 0x55d9b975b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa77fe700 1 -- 172.21.15.31:0/1505802358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975bc90 msgr2=0x55d9b9766610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:25.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975bc90 0x55d9b9766610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa77fe700 1 -- 172.21.15.31:0/1505802358 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9b952f3b0 con 0x55d9b9766b50 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa57fa700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7a900a14d0 0x7f7a900a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aace57700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7a900a14d0 0x7f7a900a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa57fa700 1 -- 172.21.15.31:0/3929182540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7a980ccf70 con 0x7f7aa80736d0 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.681+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9766b50 0x55d9b9768f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a980ca330 tx=0x7f7a980d3e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a7d12c008d5c41 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.682+0000 7f7aace57700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7a900a14d0 0x7f7a900a3980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00060b0 tx=0x7f7aa0006020 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.682+0000 7f7a8e7fc700 1 -- 172.21.15.31:0/1505802358 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98096620 con 0x55d9b9766b50 2023-12-14T18:38:25.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.682+0000 7f7a8e7fc700 1 -- 172.21.15.31:0/1505802358 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a980d3790 con 0x55d9b9766b50 2023-12-14T18:38:25.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.682+0000 7f7a8e7fc700 1 -- 172.21.15.31:0/1505802358 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9809a170 con 0x55d9b9766b50 2023-12-14T18:38:25.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.683+0000 7f7abaa48b80 1 -- 172.21.15.31:0/1505802358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9766b50 msgr2=0x55d9b9768f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.683+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9766b50 0x55d9b9768f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7a980ca330 tx=0x7f7a980d3e00 comp rx=0 tx=0).stop 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 -- 172.21.15.31:0/1505802358 shutdown_connections 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9766b50 0x55d9b9768f30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975b380 0x55d9b975b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/1505802358 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975bc90 0x55d9b9766610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 -- 172.21.15.31:0/1505802358 >> 172.21.15.31:0/1505802358 conn(0x55d9b9751ca0 msgr2=0x55d9b97577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 -- 172.21.15.31:0/1505802358 shutdown_connections 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 -- 172.21.15.31:0/1505802358 wait complete. 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.684+0000 7f7abaa48b80 1 -- start start 2023-12-14T18:38:25.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 Processor -- start 2023-12-14T18:38:25.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b97664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975bc90 0x55d9b97637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 0x55d9b9763d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9649f90 con 0x55d9b975b380 2023-12-14T18:38:25.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9649c30 con 0x55d9b975bc90 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b95316a0 con 0x55d9b9766b50 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.685+0000 7f7abaa48b80 10 client.0 _ll_register_callbacks cb 0x55d9b966d910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.686+0000 7f7aace57700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 0x55d9b9763d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.686+0000 7f7aace57700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 0x55d9b9763d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53882/0 (socket says 172.21.15.31:53882) 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.686+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b97664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.686+0000 7f7aace57700 1 -- 172.21.15.31:0/3361853640 learned_addr learned my addr 172.21.15.31:0/3361853640 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.686+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975bc90 0x55d9b97637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.689+0000 7f7aace57700 1 -- 172.21.15.31:0/3361853640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975bc90 msgr2=0x55d9b97637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.689+0000 7f7aace57700 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975bc90 0x55d9b97637d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7aace57700 1 -- 172.21.15.31:0/3361853640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 msgr2=0x55d9b97664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7aace57700 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b97664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7aace57700 1 -- 172.21.15.31:0/3361853640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa806ac90 con 0x55d9b9766b50 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7aace57700 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 0x55d9b9763d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0007720 tx=0x7f7aa00099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b755750fb9a0e3b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7a877fe700 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7a877fe700 1 -- 172.21.15.31:0/3361853640 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0005450 con 0x55d9b9766b50 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.690+0000 7f7a877fe700 1 -- 172.21.15.31:0/3361853640 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7aa0023040 con 0x55d9b9766b50 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.691+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9b96a6ce0 con 0x55d9b9766b50 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.691+0000 7f7a877fe700 1 -- 172.21.15.31:0/3361853640 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa00055b0 con 0x55d9b9766b50 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.691+0000 7f7abaa48b80 10 client.27583 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:25.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.691+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d9b9674170 con 0x55d9b9766b50 2023-12-14T18:38:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.691+0000 7f7a877fe700 1 -- 172.21.15.31:0/3361853640 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aa0034030 con 0x55d9b9766b50 2023-12-14T18:38:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7a877fe700 1 -- 172.21.15.31:0/3361853640 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7aa001baa0 con 0x55d9b9766b50 2023-12-14T18:38:25.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7abaa48b80 10 client.27583 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7abaa48b80 10 client.27583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7abaa48b80 20 client.27583 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9776590 0x55d9b97789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7abaa48b80 4 client.27583 mds_command: new command op to 24538 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9b96a6a00 con 0x55d9b9776590 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.692+0000 7f7aa7fff700 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9776590 0x55d9b97789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.693+0000 7f7aa7fff700 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9776590 0x55d9b97789a0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb632bbcea0f2909 server_cookie=9a59ace801691cee in_seq=0 out_seq=0 2023-12-14T18:38:25.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.693+0000 7f7a877fe700 10 client.27583 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.698+0000 7f7a877fe700 1 -- 172.21.15.31:0/3361853640 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9b96a6a00 con 0x55d9b9776590 2023-12-14T18:38:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.698+0000 7f7a877fe700 10 client.27583 handle_command_reply: tid=0 2023-12-14T18:38:25.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.698+0000 7f7abaa48b80 1 client.27583 shutdown 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 msgr2=0x55d9b9763d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 0x55d9b9763d50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0007720 tx=0x7f7aa00099e0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 shutdown_connections 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9776590 0x55d9b97789a0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b9766b50 0x55d9b9763d50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b97664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/3361853640 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975bc90 0x55d9b97637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 >> 172.21.15.31:0/3361853640 conn(0x55d9b9751ca0 msgr2=0x55d9b9757370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.708+0000 7f7a877fe700 0 client.27583 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:25.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.709+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 shutdown_connections 2023-12-14T18:38:25.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.709+0000 7f7abaa48b80 1 -- 172.21.15.31:0/3361853640 wait complete. 2023-12-14T18:38:25.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.709+0000 7f7abaa48b80 20 client.27583 trim_cache size 0 max 16384 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 Processor -- start 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 -- start start 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9751ca0 0x55d9b9762f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b9765f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b97664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b975f6b0 con 0x55d9b975b380 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b975e5b0 con 0x55d9b9751ca0 2023-12-14T18:38:25.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.738+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b9778cb0 con 0x55d9b975bc90 2023-12-14T18:38:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.740+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b9765f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.740+0000 7f7aa77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b9765f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42274/0 (socket says 172.21.15.31:42274) 2023-12-14T18:38:25.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.740+0000 7f7aa77fe700 1 -- 172.21.15.31:0/2794543673 learned_addr learned my addr 172.21.15.31:0/2794543673 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.741+0000 7f7aace57700 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b97664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.742+0000 7f7aace57700 1 -- 172.21.15.31:0/2794543673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9751ca0 msgr2=0x55d9b9762f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:38:25.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.742+0000 7f7aace57700 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9751ca0 0x55d9b9762f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.742+0000 7f7aace57700 1 -- 172.21.15.31:0/2794543673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 msgr2=0x55d9b9765f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.742+0000 7f7aace57700 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b9765f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.742+0000 7f7aace57700 1 -- 172.21.15.31:0/2794543673 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9b9673ec0 con 0x55d9b975bc90 2023-12-14T18:38:25.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.742+0000 7f7aace57700 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b97664b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009610 tx=0x7f7aa000afe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae09f915695ac06d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.745+0000 7f7a8effd700 1 -- 172.21.15.31:0/2794543673 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0027560 con 0x55d9b975bc90 2023-12-14T18:38:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.745+0000 7f7a8effd700 1 -- 172.21.15.31:0/2794543673 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aa0034040 con 0x55d9b975bc90 2023-12-14T18:38:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.745+0000 7f7a8effd700 1 -- 172.21.15.31:0/2794543673 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa001b8d0 con 0x55d9b975bc90 2023-12-14T18:38:25.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2794543673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 msgr2=0x55d9b97664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b97664b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009610 tx=0x7f7aa000afe0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2794543673 shutdown_connections 2023-12-14T18:38:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b97664b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b9751ca0 0x55d9b9762f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2794543673 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b975b380 0x55d9b9765f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2794543673 >> 172.21.15.31:0/2794543673 conn(0x55d9b9769290 msgr2=0x55d9b97785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2794543673 shutdown_connections 2023-12-14T18:38:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.746+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2794543673 wait complete. 2023-12-14T18:38:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 -- start start 2023-12-14T18:38:25.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 Processor -- start 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 0x55d9b976af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975b380 0x55d9b976b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b976b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b97c94e0 con 0x55d9b9751ca0 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b97c9650 con 0x55d9b975b380 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.747+0000 7f7abaa48b80 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b97c9810 con 0x55d9b975bc90 2023-12-14T18:38:25.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.748+0000 7f7abaa48b80 10 client.0 _ll_register_callbacks cb 0x55d9b966d910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:38:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.750+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 0x55d9b976af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.750+0000 7f7aa7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 0x55d9b976af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42288/0 (socket says 172.21.15.31:42288) 2023-12-14T18:38:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.750+0000 7f7aa7fff700 1 -- 172.21.15.31:0/2391433580 learned_addr learned my addr 172.21.15.31:0/2391433580 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975b380 0x55d9b976b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aace57700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b976b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa7fff700 1 -- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 msgr2=0x55d9b976b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa7fff700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b976b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa7fff700 1 -- 172.21.15.31:0/2391433580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975b380 msgr2=0x55d9b976b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa7fff700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975b380 0x55d9b976b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa7fff700 1 -- 172.21.15.31:0/2391433580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9b9674170 con 0x55d9b9751ca0 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.751+0000 7f7aa7fff700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 0x55d9b976af00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c018d90 tx=0x7f7a9c018e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644a4346fde676a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.759+0000 7f7a877fe700 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.759+0000 7f7a877fe700 1 -- 172.21.15.31:0/2391433580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c009ed0 con 0x55d9b9751ca0 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.759+0000 7f7a877fe700 1 -- 172.21.15.31:0/2391433580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f7a9c005c10 con 0x55d9b9751ca0 2023-12-14T18:38:25.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.760+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9b974f2b0 con 0x55d9b9751ca0 2023-12-14T18:38:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.761+0000 7f7a877fe700 1 -- 172.21.15.31:0/2391433580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a9c005dd0 con 0x55d9b9751ca0 2023-12-14T18:38:25.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.762+0000 7f7aace57700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b976b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:38:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.768+0000 7f7abaa48b80 10 client.17805 fetch_fsmap learned FSMap version 34 2023-12-14T18:38:25.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.768+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d9b965a9a0 con 0x55d9b9751ca0 2023-12-14T18:38:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.769+0000 7f7a877fe700 1 -- 172.21.15.31:0/2391433580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9b974f2b0 con 0x55d9b9751ca0 2023-12-14T18:38:25.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.769+0000 7f7a877fe700 1 -- 172.21.15.31:0/2391433580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7a9c0067c0 con 0x55d9b9751ca0 2023-12-14T18:38:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.770+0000 7f7abaa48b80 10 client.17805 fetch_fsmap finished waiting for FSMap version 34 2023-12-14T18:38:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.770+0000 7f7abaa48b80 10 client.17805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.770+0000 7f7abaa48b80 20 client.17805 populate_metadata read hostname 'smithi031' 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.770+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9766b50 0x55d9b97ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.770+0000 7f7abaa48b80 4 client.17805 mds_command: new command op to 24538 tid=0[{"prefix": "damage ls"}] 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.770+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 --> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9b94fccf0 con 0x55d9b9766b50 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.779+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9766b50 0x55d9b97ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.779+0000 7f7aa77fe700 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9766b50 0x55d9b97ccf40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edba0d10ec7b3fe6 server_cookie=f5753f8a736c0b73 in_seq=0 out_seq=0 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.779+0000 7f7a877fe700 10 client.17805 ms_handle_connect on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.779+0000 7f7a877fe700 1 -- 172.21.15.31:0/2391433580 <== mds.0 v2:172.21.15.204:6838/1902825357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9b94fccf0 con 0x55d9b9766b50 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.779+0000 7f7a877fe700 10 client.17805 handle_command_reply: tid=0 2023-12-14T18:38:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.779+0000 7f7abaa48b80 1 client.17805 shutdown 2023-12-14T18:38:25.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.780+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 msgr2=0x55d9b976af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.780+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 0x55d9b976af00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c018d90 tx=0x7f7a9c018e70 comp rx=0 tx=0).stop 2023-12-14T18:38:25.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 shutdown_connections 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:6838/1902825357,v1:172.21.15.204:6839/1902825357] conn(0x55d9b9766b50 0x55d9b97ccf40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b975bc90 0x55d9b976b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d9b9751ca0 0x55d9b976af00 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7abaa48b80 1 --2- 172.21.15.31:0/2391433580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x55d9b975b380 0x55d9b976b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 >> 172.21.15.31:0/2391433580 conn(0x55d9b9769290 msgr2=0x55d9b975ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.792+0000 7f7a877fe700 0 client.17805 ms_handle_reset on v2:172.21.15.204:6838/1902825357 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.795+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 shutdown_connections 2023-12-14T18:38:25.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.795+0000 7f7abaa48b80 1 -- 172.21.15.31:0/2391433580 wait complete. 2023-12-14T18:38:25.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.798+0000 7f7abaa48b80 20 client.17805 trim_cache size 0 max 16384 2023-12-14T18:38:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-12-14T18:38:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.804+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7a900a14d0 msgr2=0x7f7a900a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.804+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7a900a14d0 0x7f7a900a3980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00060b0 tx=0x7f7aa0006020 comp rx=0 tx=0).stop 2023-12-14T18:38:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.804+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 msgr2=0x7f7aa8072540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:25.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.804+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 0x7f7aa8072540 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800b550 tx=0x7f7a9800b910 comp rx=0 tx=0).stop 2023-12-14T18:38:25.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.809+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 shutdown_connections 2023-12-14T18:38:25.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.809+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7aa8074220 0x7f7aa806bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.809+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f7a900a14d0 0x7f7a900a3980 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.809+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7aa8072d20 0x7f7aa8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.809+0000 7f7aae8ba700 1 --2- 172.21.15.31:0/3929182540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aa80736d0 0x7f7aa8072540 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:25.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.809+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 >> 172.21.15.31:0/3929182540 conn(0x7f7aa8077570 msgr2=0x7f7aa81127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:25.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.815+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 shutdown_connections 2023-12-14T18:38:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:25.821+0000 7f7aae8ba700 1 -- 172.21.15.31:0/3929182540 wait complete. 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: pgmap v330: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.167 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:38:26.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:38:26.168 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:26 smithi031 ceph-mon[42085]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: pgmap v330: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:38:26.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:38:26.493 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:26 smithi176 ceph-mon[43506]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: pgmap v330: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:38:26.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:38:26.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:38:26.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:26 smithi204 ceph-mon[43866]: from='mgr.15585 ' entity='mgr.x' 2023-12-14T18:38:26.866 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-12-14T18:38:26.867 INFO:tasks.fwd_scrub:done joining 2023-12-14T18:38:26.867 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-14T18:38:26.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:38:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 -- 172.21.15.31:0/1859523499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c104460 msgr2=0x7f350c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 --2- 172.21.15.31:0/1859523499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c104460 0x7f350c068db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f350400b0a0 tx=0x7f350400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 -- 172.21.15.31:0/1859523499 shutdown_connections 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 --2- 172.21.15.31:0/1859523499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c104460 0x7f350c068db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 --2- 172.21.15.31:0/1859523499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ff420 0x7f350c0ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 --2- 172.21.15.31:0/1859523499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c0ffdc0 0x7f350c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 -- 172.21.15.31:0/1859523499 >> 172.21.15.31:0/1859523499 conn(0x7f350c075fd0 msgr2=0x7f350c0763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 -- 172.21.15.31:0/1859523499 shutdown_connections 2023-12-14T18:38:27.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.113+0000 7f351460f700 1 -- 172.21.15.31:0/1859523499 wait complete. 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.114+0000 7f351460f700 1 Processor -- start 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.114+0000 7f351460f700 1 -- start start 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.114+0000 7f351460f700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 0x7f350c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.114+0000 7f351460f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ffdc0 0x7f350c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f351460f700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c104460 0x7f350c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f351460f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19d5c0 con 0x7f350c0ffdc0 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f351460f700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19d730 con 0x7f350c104460 2023-12-14T18:38:27.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f351460f700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19d8f0 con 0x7f350c0ff420 2023-12-14T18:38:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f35123ab700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 0x7f350c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3511baa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ffdc0 0x7f350c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f35123ab700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 0x7f350c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53914/0 (socket says 172.21.15.31:53914) 2023-12-14T18:38:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f35123ab700 1 -- 172.21.15.31:0/1298800264 learned_addr learned my addr 172.21.15.31:0/1298800264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:27.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3512bac700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c104460 0x7f350c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3511baa700 1 -- 172.21.15.31:0/1298800264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 msgr2=0x7f350c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3511baa700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 0x7f350c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3511baa700 1 -- 172.21.15.31:0/1298800264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c104460 msgr2=0x7f350c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3511baa700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c104460 0x7f350c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f3511baa700 1 -- 172.21.15.31:0/1298800264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504009d00 con 0x7f350c0ffdc0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.115+0000 7f35123ab700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 0x7f350c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.116+0000 7f3511baa700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ffdc0 0x7f350c198b20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f350c100190 tx=0x7f34fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.116+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00e880 con 0x7f350c0ffdc0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.116+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34fc018480 con 0x7f350c0ffdc0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.116+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00eea0 con 0x7f350c0ffdc0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.116+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c112650 con 0x7f350c0ffdc0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.116+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c10c030 con 0x7f350c0ffdc0 2023-12-14T18:38:27.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.117+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c1127e0 con 0x7f350c0ffdc0 2023-12-14T18:38:27.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.125+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f34fc00e9e0 con 0x7f350c0ffdc0 2023-12-14T18:38:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.127+0000 7f35037fe700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f34f80a1340 0x7f34f80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.127+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f34fc013070 con 0x7f350c0ffdc0 2023-12-14T18:38:27.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.127+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f34fc0cd790 con 0x7f350c0ffdc0 2023-12-14T18:38:27.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.127+0000 7f35123ab700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f34f80a1340 0x7f34f80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.128+0000 7f35123ab700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f34f80a1340 0x7f34f80a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3508009ea0 tx=0x7f3508009290 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:27.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:27 smithi031 ceph-mon[42085]: Adjusting osd_memory_target on smithi031 to 183.5M 2023-12-14T18:38:27.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:27 smithi031 ceph-mon[42085]: Unable to set osd_memory_target on smithi031 to 192481587: error parsing value: Value '192481587' is below minimum 939524096 2023-12-14T18:38:27.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:27 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:38:27.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:27 smithi031 ceph-mon[42085]: Service mds.all has no related daemon already placed. 2023-12-14T18:38:27.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:27 smithi031 ceph-mon[42085]: Service mgr has no related daemon already placed. 2023-12-14T18:38:27.216 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:27 smithi031 ceph-mon[42085]: Service mon has no related daemon already placed. 2023-12-14T18:38:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:27 smithi176 ceph-mon[43506]: Adjusting osd_memory_target on smithi031 to 183.5M 2023-12-14T18:38:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:27 smithi176 ceph-mon[43506]: Unable to set osd_memory_target on smithi031 to 192481587: error parsing value: Value '192481587' is below minimum 939524096 2023-12-14T18:38:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:27 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:38:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:27 smithi176 ceph-mon[43506]: Service mds.all has no related daemon already placed. 2023-12-14T18:38:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:27 smithi176 ceph-mon[43506]: Service mgr has no related daemon already placed. 2023-12-14T18:38:27.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:27 smithi176 ceph-mon[43506]: Service mon has no related daemon already placed. 2023-12-14T18:38:27.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.497+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f350c1a5710 con 0x7f350c0ffdc0 2023-12-14T18:38:27.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.506+0000 7f35037fe700 1 -- 172.21.15.31:0/1298800264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+25454 (secure 0 0 0) 0x7f34fc0964d0 con 0x7f350c0ffdc0 2023-12-14T18:38:27.510 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:38:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:38:09.534375+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2924181707":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2688246057":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2688246057":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2160431679":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/3108903497":"2023-12-15T18:27:52.189495+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/3199333831":"2023-12-15T18:27:52.189495+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/3645411014":"2023-12-15T18:27:52.189495+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:38:27.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.510+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f34f80a1340 msgr2=0x7f34f80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.510+0000 7f351460f700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f34f80a1340 0x7f34f80a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3508009ea0 tx=0x7f3508009290 comp rx=0 tx=0).stop 2023-12-14T18:38:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.510+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ffdc0 msgr2=0x7f350c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.510+0000 7f351460f700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ffdc0 0x7f350c198b20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f350c100190 tx=0x7f34fc00cc20 comp rx=0 tx=0).stop 2023-12-14T18:38:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.511+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 shutdown_connections 2023-12-14T18:38:27.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.511+0000 7f351460f700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f350c104460 0x7f350c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.511+0000 7f351460f700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f34f80a1340 0x7f34f80a37f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.511+0000 7f351460f700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f350c0ff420 0x7f350c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.511+0000 7f351460f700 1 --2- 172.21.15.31:0/1298800264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ffdc0 0x7f350c198b20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.511+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 >> 172.21.15.31:0/1298800264 conn(0x7f350c075fd0 msgr2=0x7f350c102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:27.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.515+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 shutdown_connections 2023-12-14T18:38:27.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.517+0000 7f351460f700 1 -- 172.21.15.31:0/1298800264 wait complete. 2023-12-14T18:38:27.553 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:38:27.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:38:27.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:27 smithi204 ceph-mon[43866]: Adjusting osd_memory_target on smithi031 to 183.5M 2023-12-14T18:38:27.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:27 smithi204 ceph-mon[43866]: Unable to set osd_memory_target on smithi031 to 192481587: error parsing value: Value '192481587' is below minimum 939524096 2023-12-14T18:38:27.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:27 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:38:27.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:27 smithi204 ceph-mon[43866]: Service mds.all has no related daemon already placed. 2023-12-14T18:38:27.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:27 smithi204 ceph-mon[43866]: Service mgr has no related daemon already placed. 2023-12-14T18:38:27.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:27 smithi204 ceph-mon[43866]: Service mon has no related daemon already placed. 2023-12-14T18:38:27.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.836+0000 7ff2aa868700 1 -- 172.21.15.31:0/2046822723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 msgr2=0x7ff2a4073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.836+0000 7ff2aa868700 1 --2- 172.21.15.31:0/2046822723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a4073190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff298009990 tx=0x7ff298009ca0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.845+0000 7ff2aa868700 1 -- 172.21.15.31:0/2046822723 shutdown_connections 2023-12-14T18:38:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.845+0000 7ff2aa868700 1 --2- 172.21.15.31:0/2046822723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a40736d0 0x7ff2a4113630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.845+0000 7ff2aa868700 1 --2- 172.21.15.31:0/2046822723 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a4074220 0x7ff2a40745f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.845+0000 7ff2aa868700 1 --2- 172.21.15.31:0/2046822723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a4073190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.845+0000 7ff2aa868700 1 -- 172.21.15.31:0/2046822723 >> 172.21.15.31:0/2046822723 conn(0x7ff2a4077570 msgr2=0x7ff2a4077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:27.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.848+0000 7ff2aa868700 1 -- 172.21.15.31:0/2046822723 shutdown_connections 2023-12-14T18:38:27.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.850+0000 7ff2aa868700 1 -- 172.21.15.31:0/2046822723 wait complete. 2023-12-14T18:38:27.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.851+0000 7ff2aa868700 1 Processor -- start 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.851+0000 7ff2aa868700 1 -- start start 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2aa868700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a41a2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2aa868700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a40736d0 0x7ff2a41a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2aa868700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4074220 0x7ff2a419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2aa868700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419d0f0 con 0x7ff2a4074220 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2aa868700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419d230 con 0x7ff2a40736d0 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2aa868700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419d3f0 con 0x7ff2a4072d20 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a41a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a41a2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53922/0 (socket says 172.21.15.31:53922) 2023-12-14T18:38:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 -- 172.21.15.31:0/877376476 learned_addr learned my addr 172.21.15.31:0/877376476 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a8e05700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4074220 0x7ff2a419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 -- 172.21.15.31:0/877376476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a40736d0 msgr2=0x7ff2a41a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a40736d0 0x7ff2a41a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 -- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4074220 msgr2=0x7ff2a419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4074220 0x7ff2a419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a3fff700 1 -- 172.21.15.31:0/877376476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980095f0 con 0x7ff2a4072d20 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.852+0000 7ff2a8e05700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4074220 0x7ff2a419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.853+0000 7ff2a3fff700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a41a2940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00ba50 tx=0x7ff29c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.853+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00cab0 con 0x7ff2a4072d20 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.853+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a4116990 con 0x7ff2a4072d20 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.853+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a4116800 con 0x7ff2a4072d20 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.853+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff29c00cc10 con 0x7ff2a4072d20 2023-12-14T18:38:27.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.853+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0075b0 con 0x7ff2a4072d20 2023-12-14T18:38:27.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.854+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff290005320 con 0x7ff2a4072d20 2023-12-14T18:38:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.856+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff29c014440 con 0x7ff2a4072d20 2023-12-14T18:38:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.857+0000 7ff2a17fa700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff28c0a13c0 0x7ff28c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:27.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.857+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7ff29c0cc7f0 con 0x7ff2a4072d20 2023-12-14T18:38:27.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.860+0000 7ff2a37fe700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff28c0a13c0 0x7ff28c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:27.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.861+0000 7ff2a37fe700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff28c0a13c0 0x7ff28c0a3870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2980098e0 tx=0x7ff298005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:27.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:27.871+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff29c04e730 con 0x7ff2a4072d20 2023-12-14T18:38:28.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.242+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff290005f70 con 0x7ff2a4072d20 2023-12-14T18:38:28.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.243+0000 7ff2a17fa700 1 -- 172.21.15.31:0/877376476 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff29c095870 con 0x7ff2a4072d20 2023-12-14T18:38:28.246 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:38:28.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.255+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff28c0a13c0 msgr2=0x7ff28c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:28.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.255+0000 7ff2aa868700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff28c0a13c0 0x7ff28c0a3870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2980098e0 tx=0x7ff298005fd0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.255+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 msgr2=0x7ff2a41a2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:28.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.255+0000 7ff2aa868700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a41a2940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00ba50 tx=0x7ff29c00be10 comp rx=0 tx=0).stop 2023-12-14T18:38:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.262+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 shutdown_connections 2023-12-14T18:38:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.262+0000 7ff2aa868700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2a4074220 0x7ff2a419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.262+0000 7ff2aa868700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7ff28c0a13c0 0x7ff28c0a3870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.262+0000 7ff2aa868700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2a4072d20 0x7ff2a41a2940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.262+0000 7ff2aa868700 1 --2- 172.21.15.31:0/877376476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2a40736d0 0x7ff2a41a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.262+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 >> 172.21.15.31:0/877376476 conn(0x7ff2a4077570 msgr2=0x7ff2a4112830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.269+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 shutdown_connections 2023-12-14T18:38:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.274+0000 7ff2aa868700 1 -- 172.21.15.31:0/877376476 wait complete. 2023-12-14T18:38:28.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:38:28.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:28 smithi031 ceph-mon[42085]: pgmap v331: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:38:28.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:28 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1298800264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:38:28.491 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:28 smithi176 ceph-mon[43506]: pgmap v331: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:38:28.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:28 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1298800264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 -- 172.21.15.31:0/4259971101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7941084a0 msgr2=0x7fe794079740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 --2- 172.21.15.31:0/4259971101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7941084a0 0x7fe794079740 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe788005ce0 tx=0x7fe788009a60 comp rx=0 tx=0).stop 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 -- 172.21.15.31:0/4259971101 shutdown_connections 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 --2- 172.21.15.31:0/4259971101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7941084a0 0x7fe794079740 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 --2- 172.21.15.31:0/4259971101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe79410c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 --2- 172.21.15.31:0/4259971101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe794107af0 0x7fe794107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.544+0000 7fe799973700 1 -- 172.21.15.31:0/4259971101 >> 172.21.15.31:0/4259971101 conn(0x7fe79406b7e0 msgr2=0x7fe794071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:28.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.546+0000 7fe799973700 1 -- 172.21.15.31:0/4259971101 shutdown_connections 2023-12-14T18:38:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.547+0000 7fe799973700 1 -- 172.21.15.31:0/4259971101 wait complete. 2023-12-14T18:38:28.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 Processor -- start 2023-12-14T18:38:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 -- start start 2023-12-14T18:38:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe794107af0 0x7fe794073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7941084a0 0x7fe7940740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe7941c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe794074a50 con 0x7fe7941084a0 2023-12-14T18:38:28.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79412eb50 con 0x7fe79410bef0 2023-12-14T18:38:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.548+0000 7fe799973700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79412ed10 con 0x7fe794107af0 2023-12-14T18:38:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.551+0000 7fe7937fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe7941c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.551+0000 7fe7937fe700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe7941c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41798/0 (socket says 172.21.15.31:41798) 2023-12-14T18:38:28.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.551+0000 7fe7937fe700 1 -- 172.21.15.31:0/4174575809 learned_addr learned my addr 172.21.15.31:0/4174575809 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7927fc700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7941084a0 0x7fe7940740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7937fe700 1 -- 172.21.15.31:0/4174575809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe794107af0 msgr2=0x7fe794073bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7937fe700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe794107af0 0x7fe794073bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7937fe700 1 -- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7941084a0 msgr2=0x7fe7940740f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7937fe700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7941084a0 0x7fe7940740f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7937fe700 1 -- 172.21.15.31:0/4174575809 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880095f0 con 0x7fe79410bef0 2023-12-14T18:38:28.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7927fc700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7941084a0 0x7fe7940740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:38:28.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.552+0000 7fe7937fe700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe7941c20a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe788005740 tx=0x7fe788009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.555+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788003ad0 con 0x7fe79410bef0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.555+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7880040f0 con 0x7fe79410bef0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.555+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79407c730 con 0x7fe79410bef0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.555+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79407c5a0 con 0x7fe79410bef0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.555+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78800ede0 con 0x7fe79410bef0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.556+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe780005320 con 0x7fe79410bef0 2023-12-14T18:38:28.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.560+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe788003c30 con 0x7fe79410bef0 2023-12-14T18:38:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.561+0000 7fe798971700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe77c0a14d0 0x7fe77c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.561+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fe788012070 con 0x7fe79410bef0 2023-12-14T18:38:28.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.561+0000 7fe792ffd700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe77c0a14d0 0x7fe77c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.562+0000 7fe792ffd700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe77c0a14d0 0x7fe77c0a3980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe784009af0 tx=0x7fe784009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:28.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.563+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe7880975c0 con 0x7fe79410bef0 2023-12-14T18:38:28.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:28 smithi204 ceph-mon[43866]: pgmap v331: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 544 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:38:28.590 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:28 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1298800264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:38:28.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.924+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe780005f70 con 0x7fe79410bef0 2023-12-14T18:38:28.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.928+0000 7fe798971700 1 -- 172.21.15.31:0/4174575809 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe78809abe0 con 0x7fe79410bef0 2023-12-14T18:38:28.930 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.939+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe77c0a14d0 msgr2=0x7fe77c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.939+0000 7fe799973700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe77c0a14d0 0x7fe77c0a3980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe784009af0 tx=0x7fe784009260 comp rx=0 tx=0).stop 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 msgr2=0x7fe7941c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe7941c20a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe788005740 tx=0x7fe788009b50 comp rx=0 tx=0).stop 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 shutdown_connections 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe79410bef0 0x7fe7941c20a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe77c0a14d0 0x7fe77c0a3980 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe784009af0 tx=0x7fe784009260 comp rx=0 tx=0).stop 2023-12-14T18:38:28.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe794107af0 0x7fe794073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 --2- 172.21.15.31:0/4174575809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7941084a0 0x7fe7940740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:28.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.940+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 >> 172.21.15.31:0/4174575809 conn(0x7fe79406b7e0 msgr2=0x7fe794108be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:28.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.941+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 shutdown_connections 2023-12-14T18:38:28.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:28.941+0000 7fe799973700 1 -- 172.21.15.31:0/4174575809 wait complete. 2023-12-14T18:38:28.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:38:29.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.233+0000 7fe5cd910700 1 -- 172.21.15.31:0/491004583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c80ff3b0 msgr2=0x7fe5c80ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:29.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.233+0000 7fe5cd910700 1 --2- 172.21.15.31:0/491004583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c80ff3b0 0x7fe5c80ff780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800dd50 tx=0x7fe5b800f330 comp rx=0 tx=0).stop 2023-12-14T18:38:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.237+0000 7fe5cd910700 1 -- 172.21.15.31:0/491004583 shutdown_connections 2023-12-14T18:38:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.237+0000 7fe5cd910700 1 --2- 172.21.15.31:0/491004583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5c81043f0 0x7fe5c810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.237+0000 7fe5cd910700 1 --2- 172.21.15.31:0/491004583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c80ff3b0 0x7fe5c80ff780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.237+0000 7fe5cd910700 1 --2- 172.21.15.31:0/491004583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c80ffd50 0x7fe5c8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.237+0000 7fe5cd910700 1 -- 172.21.15.31:0/491004583 >> 172.21.15.31:0/491004583 conn(0x7fe5c80763a0 msgr2=0x7fe5c80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:29.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.240+0000 7fe5cd910700 1 -- 172.21.15.31:0/491004583 shutdown_connections 2023-12-14T18:38:29.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.247+0000 7fe5cd910700 1 -- 172.21.15.31:0/491004583 wait complete. 2023-12-14T18:38:29.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.248+0000 7fe5cd910700 1 Processor -- start 2023-12-14T18:38:29.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 -- start start 2023-12-14T18:38:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5c80ff3b0 0x7fe5c8194260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c80ffd50 0x7fe5c81947a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 0x7fe5c81b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c81b1570 con 0x7fe5c80ffd50 2023-12-14T18:38:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806a540 con 0x7fe5c80ff3b0 2023-12-14T18:38:29.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.254+0000 7fe5cd910700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806a700 con 0x7fe5c81043f0 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 0x7fe5c81b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 0x7fe5c81b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53956/0 (socket says 172.21.15.31:53956) 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 -- 172.21.15.31:0/1247472635 learned_addr learned my addr 172.21.15.31:0/1247472635 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 -- 172.21.15.31:0/1247472635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5c80ff3b0 msgr2=0x7fe5c8194260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5c80ff3b0 0x7fe5c8194260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 -- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c80ffd50 msgr2=0x7fe5c81947a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:38:29.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c80ffd50 0x7fe5c81947a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 -- 172.21.15.31:0/1247472635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b800da00 con 0x7fe5c81043f0 2023-12-14T18:38:29.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.255+0000 7fe5c77fe700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 0x7fe5c81b0fa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00d6d0 tx=0x7fe5bc00da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:29.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.256+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc004a30 con 0x7fe5c81043f0 2023-12-14T18:38:29.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.256+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c810e1a0 con 0x7fe5c81043f0 2023-12-14T18:38:29.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.256+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5bc0054e0 con 0x7fe5c81043f0 2023-12-14T18:38:29.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.256+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc00fc80 con 0x7fe5c81043f0 2023-12-14T18:38:29.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.256+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c810e010 con 0x7fe5c81043f0 2023-12-14T18:38:29.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.259+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b4005320 con 0x7fe5c81043f0 2023-12-14T18:38:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.275+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe5bc004b90 con 0x7fe5c81043f0 2023-12-14T18:38:29.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.275+0000 7fe5cc90e700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5b00a1400 0x7fe5b00a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.275+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fe5bc0ccff0 con 0x7fe5c81043f0 2023-12-14T18:38:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.276+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe5bc0cd470 con 0x7fe5c81043f0 2023-12-14T18:38:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.276+0000 7fe5c6ffd700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5b00a1400 0x7fe5b00a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.277+0000 7fe5c6ffd700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5b00a1400 0x7fe5b00a38b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8019040 tx=0x7fe5b8005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:29.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/877376476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:38:29.465 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:29 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4174575809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:38:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/877376476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:38:29.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:29 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4174575809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:38:29.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/877376476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:38:29.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:29 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4174575809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:38:29.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.643+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5b4005f70 con 0x7fe5c81043f0 2023-12-14T18:38:29.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.654+0000 7fe5cc90e700 1 -- 172.21.15.31:0/1247472635 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5bc017030 con 0x7fe5c81043f0 2023-12-14T18:38:29.657 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:38:29.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.659+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5b00a1400 msgr2=0x7fe5b00a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:29.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.659+0000 7fe5cd910700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5b00a1400 0x7fe5b00a38b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8019040 tx=0x7fe5b8005fd0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.659+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 msgr2=0x7fe5c81b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:29.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.659+0000 7fe5cd910700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 0x7fe5c81b0fa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00d6d0 tx=0x7fe5bc00da90 comp rx=0 tx=0).stop 2023-12-14T18:38:29.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.662+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 shutdown_connections 2023-12-14T18:38:29.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.662+0000 7fe5cd910700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe5b00a1400 0x7fe5b00a38b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.662+0000 7fe5cd910700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5c81043f0 0x7fe5c81b0fa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.662+0000 7fe5cd910700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe5c80ff3b0 0x7fe5c8194260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.662+0000 7fe5cd910700 1 --2- 172.21.15.31:0/1247472635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5c80ffd50 0x7fe5c81947a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.662+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 >> 172.21.15.31:0/1247472635 conn(0x7fe5c80763a0 msgr2=0x7fe5c8102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:29.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.663+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 shutdown_connections 2023-12-14T18:38:29.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.663+0000 7fe5cd910700 1 -- 172.21.15.31:0/1247472635 wait complete. 2023-12-14T18:38:29.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:38:29.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.934+0000 7f1e7d2de700 1 -- 172.21.15.31:0/2406583761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e781043d0 msgr2=0x7f1e7810bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:29.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.934+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/2406583761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e781043d0 0x7f1e7810bfa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009f90 tx=0x7f1e7000b330 comp rx=0 tx=0).stop 2023-12-14T18:38:29.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 -- 172.21.15.31:0/2406583761 shutdown_connections 2023-12-14T18:38:29.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/2406583761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e781043d0 0x7f1e7810bfa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/2406583761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e780ff390 0x7f1e780ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/2406583761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780ffd30 0x7f1e78103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 -- 172.21.15.31:0/2406583761 >> 172.21.15.31:0/2406583761 conn(0x7f1e78075fd0 msgr2=0x7f1e780763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:29.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 -- 172.21.15.31:0/2406583761 shutdown_connections 2023-12-14T18:38:29.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.935+0000 7f1e7d2de700 1 -- 172.21.15.31:0/2406583761 wait complete. 2023-12-14T18:38:29.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 Processor -- start 2023-12-14T18:38:29.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 -- start start 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e780ff390 0x7f1e78194050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780ffd30 0x7f1e78194590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 0x7f1e78194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781a8cc0 con 0x7f1e780ffd30 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781983c0 con 0x7f1e780ff390 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.936+0000 7f1e7d2de700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78198580 con 0x7f1e781043d0 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 0x7f1e78194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:29.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 0x7f1e78194ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:53988/0 (socket says 172.21.15.31:53988) 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 -- 172.21.15.31:0/3357991454 learned_addr learned my addr 172.21.15.31:0/3357991454 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e76ffd700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e780ff390 0x7f1e78194050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 -- 172.21.15.31:0/3357991454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e780ff390 msgr2=0x7f1e78194050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e780ff390 0x7f1e78194050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 -- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780ffd30 msgr2=0x7f1e78194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780ffd30 0x7f1e78194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.937+0000 7f1e777fe700 1 -- 172.21.15.31:0/3357991454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70009c40 con 0x7f1e781043d0 2023-12-14T18:38:29.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.938+0000 7f1e777fe700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 0x7f1e78194ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e70004c80 tx=0x7f1e70004cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:29.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.938+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000e040 con 0x7f1e781043d0 2023-12-14T18:38:29.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.938+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e70007910 con 0x7f1e781043d0 2023-12-14T18:38:29.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.938+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7810e150 con 0x7f1e781043d0 2023-12-14T18:38:29.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.938+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7810dfc0 con 0x7f1e781043d0 2023-12-14T18:38:29.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.939+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e70005260 con 0x7f1e781043d0 2023-12-14T18:38:29.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.940+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7810e2e0 con 0x7f1e781043d0 2023-12-14T18:38:29.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.941+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1e7001d470 con 0x7f1e781043d0 2023-12-14T18:38:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.942+0000 7f1e5ffff700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1e600a12b0 0x7f1e600a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.942+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7f1e700ce960 con 0x7f1e781043d0 2023-12-14T18:38:29.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.942+0000 7f1e76ffd700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1e600a12b0 0x7f1e600a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:29.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.943+0000 7f1e76ffd700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1e600a12b0 0x7f1e600a3760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009a10 tx=0x7f1e68009190 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:29.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:29.954+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1e700979c0 con 0x7f1e781043d0 2023-12-14T18:38:30.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.283+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1e780679f0 con 0x7f1e781043d0 2023-12-14T18:38:30.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.284+0000 7f1e5ffff700 1 -- 172.21.15.31:0/3357991454 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1e7009afe0 con 0x7f1e781043d0 2023-12-14T18:38:30.288 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:38:30.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1e600a12b0 msgr2=0x7f1e600a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1e600a12b0 0x7f1e600a3760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009a10 tx=0x7f1e68009190 comp rx=0 tx=0).stop 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 msgr2=0x7f1e78194ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 0x7f1e78194ad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e70004c80 tx=0x7f1e70004cb0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 shutdown_connections 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f1e600a12b0 0x7f1e600a3760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e781043d0 0x7f1e78194ad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1e780ff390 0x7f1e78194050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 --2- 172.21.15.31:0/3357991454 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780ffd30 0x7f1e78194590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.287+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 >> 172.21.15.31:0/3357991454 conn(0x7f1e78075fd0 msgr2=0x7f1e78102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:30.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.288+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 shutdown_connections 2023-12-14T18:38:30.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.288+0000 7f1e7d2de700 1 -- 172.21.15.31:0/3357991454 wait complete. 2023-12-14T18:38:30.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-14T18:38:30.469 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:30 smithi031 ceph-mon[42085]: pgmap v332: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:38:30.469 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:30 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1247472635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:38:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:30 smithi176 ceph-mon[43506]: pgmap v332: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:38:30.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:30 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1247472635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:38:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.556+0000 7fcc65f3a700 1 -- 172.21.15.31:0/2068406304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 msgr2=0x7fcc6011be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.556+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/2068406304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc6011be20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcc5801c0d0 tx=0x7fcc5801c3e0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 -- 172.21.15.31:0/2068406304 shutdown_connections 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/2068406304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc6011be20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/2068406304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc60107af0 0x7fcc6010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/2068406304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc60107ec0 0x7fcc60108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 -- 172.21.15.31:0/2068406304 >> 172.21.15.31:0/2068406304 conn(0x7fcc6006c4b0 msgr2=0x7fcc60071670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 -- 172.21.15.31:0/2068406304 shutdown_connections 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.557+0000 7fcc65f3a700 1 -- 172.21.15.31:0/2068406304 wait complete. 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.558+0000 7fcc65f3a700 1 Processor -- start 2023-12-14T18:38:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.558+0000 7fcc65f3a700 1 -- start start 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc65f3a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc60107af0 0x7fcc60118cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc65f3a700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc60107ec0 0x7fcc60119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc65f3a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc60114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc65f3a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60115510 con 0x7fcc60107af0 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc65f3a700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60115680 con 0x7fcc60119a40 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc65f3a700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60115840 con 0x7fcc60107ec0 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc60114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:30.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc60114d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:41856/0 (socket says 172.21.15.31:41856) 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 -- 172.21.15.31:0/543199982 learned_addr learned my addr 172.21.15.31:0/543199982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5f7fe700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc60107af0 0x7fcc60118cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 -- 172.21.15.31:0/543199982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc60107ec0 msgr2=0x7fcc60119210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc60107ec0 0x7fcc60119210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 -- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc60107af0 msgr2=0x7fcc60118cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc60107af0 0x7fcc60118cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:30.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.559+0000 7fcc5ffff700 1 -- 172.21.15.31:0/543199982 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5801ad10 con 0x7fcc60119a40 2023-12-14T18:38:30.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.561+0000 7fcc5ffff700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc60114d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc5801c0a0 tx=0x7fcc58018940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:30.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.561+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5801f050 con 0x7fcc60119a40 2023-12-14T18:38:30.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.561+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc58005570 con 0x7fcc60119a40 2023-12-14T18:38:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.561+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5802c8c0 con 0x7fcc60119a40 2023-12-14T18:38:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.563+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc60111a30 con 0x7fcc60119a40 2023-12-14T18:38:30.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.563+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc601118a0 con 0x7fcc60119a40 2023-12-14T18:38:30.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.565+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc400052f0 con 0x7fcc60119a40 2023-12-14T18:38:30.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.567+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcc580056e0 con 0x7fcc60119a40 2023-12-14T18:38:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.568+0000 7fcc5d7fa700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc4c0a1490 0x7fcc4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:30.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.568+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9613+0+0 (secure 0 0 0) 0x7fcc580dff10 con 0x7fcc60119a40 2023-12-14T18:38:30.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.569+0000 7fcc5f7fe700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc4c0a1490 0x7fcc4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:30.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.570+0000 7fcc5f7fe700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc4c0a1490 0x7fcc4c0a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006e4a0 tx=0x7fcc50009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:30.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:30.574+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcc580a8f90 con 0x7fcc60119a40 2023-12-14T18:38:30.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:30 smithi204 ceph-mon[43866]: pgmap v332: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:38:30.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:30 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1247472635' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:38:31.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.039+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fcc40005160 con 0x7fcc60119a40 2023-12-14T18:38:31.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.054+0000 7fcc5d7fa700 1 -- 172.21.15.31:0/543199982 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) v1 ==== 81+0+733771 (secure 0 0 0) 0x7fcc58031090 con 0x7fcc60119a40 2023-12-14T18:38:31.071 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 24, 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 15585, 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:31.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1869071623 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-12-14T18:38:31.073 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1869071623 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.31:6801/1869071623", 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2023-12-14T18:27:52.189770+0000", 2023-12-14T18:38:31.074 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540138322906710015, 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 25438, 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "z", 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138322906710015, 2023-12-14T18:38:31.075 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.076 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.080 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.081 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.084 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.087 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-12-14T18:38:31.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.093 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.097 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-12-14T18:38:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.099 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-12-14T18:38:31.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.102 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-12-14T18:38:31.106 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.107 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-12-14T18:38:31.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-12-14T18:38:31.109 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-12-14T18:38:31.113 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.114 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-14T18:38:31.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.119 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.120 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.123 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-12-14T18:38:31.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-12-14T18:38:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-12-14T18:38:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-14T18:38:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-12-14T18:38:31.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.137 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-12-14T18:38:31.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "alertmanager_web_password": { 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alertmanager_web_password", 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Alertmanager web password", 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "alertmanager_web_user": { 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alertmanager_web_user", 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Alertmanager web user", 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-12-14T18:38:31.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-14T18:38:31.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-12-14T18:38:31.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-14T18:38:31.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-12-14T18:38:31.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-12-14T18:38:31.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.161 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.164 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.165 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.177 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.178 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.197 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-12-14T18:38:31.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-12-14T18:38:31.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-12-14T18:38:31.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-12-14T18:38:31.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_web_password": { 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_web_password", 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus web password", 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_web_user": { 2023-12-14T18:38:31.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_web_user", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus web user", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-12-14T18:38:31.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-14T18:38:31.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.219 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-14T18:38:31.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-12-14T18:38:31.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.225 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-12-14T18:38:31.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.228 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.229 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.233 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-14T18:38:31.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-12-14T18:38:31.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-14T18:38:31.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-14T18:38:31.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-14T18:38:31.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-14T18:38:31.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.259 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.261 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.263 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-14T18:38:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-14T18:38:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-14T18:38:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-14T18:38:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-12-14T18:38:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-12-14T18:38:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-12-14T18:38:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-12-14T18:38:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-12-14T18:38:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-12-14T18:38:31.332 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.336 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-14T18:38:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-14T18:38:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-12-14T18:38:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-14T18:38:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-12-14T18:38:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2023-12-14T18:38:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-14T18:38:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-12-14T18:38:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-12-14T18:38:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-12-14T18:38:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-12-14T18:38:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-12-14T18:38:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-12-14T18:38:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.463 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "noautoscale": { 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "noautoscale", 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "global autoscale flag", 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.478 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-14T18:38:31.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.481 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.482 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.486 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-12-14T18:38:31.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-12-14T18:38:31.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.496 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-12-14T18:38:31.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-12-14T18:38:31.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.517 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.518 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.524 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.525 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-12-14T18:38:31.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.536 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.543 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.545 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.546 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-12-14T18:38:31.549 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.555 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.556 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.564 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.566 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.570 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.573 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-12-14T18:38:31.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-12-14T18:38:31.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.587 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.589 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.590 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.592 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.593 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.595 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.598 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.601 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.602 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.604 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.605 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.608 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.610 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.612 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.613 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-14T18:38:31.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.626 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.627 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.630 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.631 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-12-14T18:38:31.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.637 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.638 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-12-14T18:38:31.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.647 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.652 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.656 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.658 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.659 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.660 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.661 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-12-14T18:38:31.662 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.664 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.665 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-12-14T18:38:31.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 25441, 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138322906710015, 2023-12-14T18:38:31.682 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.687 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.688 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.689 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.693 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-12-14T18:38:31.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-12-14T18:38:31.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-14T18:38:31.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-12-14T18:38:31.703 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-12-14T18:38:31.708 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-12-14T18:38:31.709 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.710 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-12-14T18:38:31.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-12-14T18:38:31.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:38:31.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.718 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.719 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-12-14T18:38:31.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-14T18:38:31.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-14T18:38:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-12-14T18:38:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-12-14T18:38:31.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-12-14T18:38:31.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "alertmanager_web_password": { 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alertmanager_web_password", 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Alertmanager web password", 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "alertmanager_web_user": { 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alertmanager_web_user", 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Alertmanager web user", 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.738 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-14T18:38:31.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-12-14T18:38:31.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.745 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-14T18:38:31.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-12-14T18:38:31.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-12-14T18:38:31.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-12-14T18:38:31.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-12-14T18:38:31.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-14T18:38:31.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-12-14T18:38:31.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-14T18:38:31.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-12-14T18:38:31.765 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-12-14T18:38:31.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-12-14T18:38:31.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.779 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.781 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-14T18:38:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-12-14T18:38:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_web_password": { 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_web_password", 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus web password", 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_web_user": { 2023-12-14T18:38:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_web_user", 2023-12-14T18:38:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus web user", 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-12-14T18:38:31.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.806 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-12-14T18:38:31.808 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.810 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.812 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.813 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.814 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-12-14T18:38:31.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-12-14T18:38:31.818 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.819 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.822 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.823 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-14T18:38:31.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-12-14T18:38:31.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-12-14T18:38:31.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.834 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-14T18:38:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.843 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-14T18:38:31.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-14T18:38:31.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.852 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-14T18:38:31.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.854 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.857 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-14T18:38:31.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-14T18:38:31.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.872 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.879 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:31.881 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-14T18:38:31.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-14T18:38:31.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-12-14T18:38:31.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.894 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.897 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.900 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.903 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.904 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:31.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-12-14T18:38:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.914 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.918 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.919 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.921 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-12-14T18:38:31.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-12-14T18:38:31.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-12-14T18:38:31.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.935 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.942 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.944 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.945 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-12-14T18:38:31.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-12-14T18:38:31.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.952 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.953 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-12-14T18:38:31.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.962 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-14T18:38:31.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.967 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.968 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.971 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.974 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.975 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.977 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.978 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.979 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.980 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2023-12-14T18:38:31.983 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.989 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.990 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.993 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.994 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.995 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:31.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:31.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:31.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-12-14T18:38:32.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.002 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-12-14T18:38:32.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-12-14T18:38:32.007 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.009 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.010 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.012 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.016 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.017 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.027 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.032 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.036 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-12-14T18:38:32.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-12-14T18:38:32.039 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-12-14T18:38:32.040 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.041 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.045 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.046 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.049 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.052 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.055 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.060 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.062 INFO:teuthology.orchestra.run.smithi031.stdout: "noautoscale": { 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "noautoscale", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "global autoscale flag", 2023-12-14T18:38:32.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.073 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.074 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-14T18:38:32.082 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.087 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.088 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.089 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.090 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.091 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-12-14T18:38:32.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-12-14T18:38:32.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-12-14T18:38:32.099 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.101 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.102 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.103 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.108 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.110 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.111 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-12-14T18:38:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.115 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.116 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.121 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:32.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.125 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.128 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.130 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.131 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.134 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.135 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.136 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.139 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-12-14T18:38:32.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.143 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.144 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.145 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.150 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.152 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.153 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.154 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-12-14T18:38:32.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.161 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-12-14T18:38:32.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-12-14T18:38:32.168 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.170 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.174 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.178 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.181 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.184 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.185 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.189 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.199 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-12-14T18:38:32.202 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.203 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.204 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.208 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-12-14T18:38:32.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-14T18:38:32.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-12-14T18:38:32.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.213 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-12-14T18:38:32.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-12-14T18:38:32.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.224 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-12-14T18:38:32.235 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.236 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.241 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.243 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.246 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-12-14T18:38:32.257 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-12-14T18:38:32.258 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-12-14T18:38:32.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.260 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.262 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.263 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.267 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-12-14T18:38:32.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2023-12-14T18:38:32.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard", 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "restful", 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_schedule" 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-12-14T18:38:32.276 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.278 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.279 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.283 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-12-14T18:38:32.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.290 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.291 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-12-14T18:38:32.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-12-14T18:38:32.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.295 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-12-14T18:38:32.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-12-14T18:38:32.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-12-14T18:38:32.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:38:32.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-12-14T18:38:32.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.305 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-14T18:38:32.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.308 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.309 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:32.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.313 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.317 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.329 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-12-14T18:38:32.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.332 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-12-14T18:38:32.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "alertmanager_web_password": { 2023-12-14T18:38:32.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alertmanager_web_password", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Alertmanager web password", 2023-12-14T18:38:32.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "alertmanager_web_user": { 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alertmanager_web_user", 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Alertmanager web user", 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-12-14T18:38:32.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-14T18:38:32.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.340 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.341 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.344 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-14T18:38:32.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-12-14T18:38:32.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.349 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-12-14T18:38:32.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.356 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-12-14T18:38:32.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-12-14T18:38:32.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-12-14T18:38:32.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.366 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.367 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.368 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-12-14T18:38:32.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-12-14T18:38:32.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-12-14T18:38:32.372 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-14T18:38:32.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.375 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-14T18:38:32.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.378 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.381 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.382 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.385 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.386 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.388 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-14T18:38:32.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-12-14T18:38:32.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-12-14T18:38:32.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-12-14T18:38:32.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-12-14T18:38:32.397 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_web_password": { 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_web_password", 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus web password", 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.401 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_web_user": { 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_web_user", 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus web user", 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-12-14T18:38:32.406 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-14T18:38:32.407 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.408 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.409 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-12-14T18:38:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-12-14T18:38:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.413 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-12-14T18:38:32.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-12-14T18:38:32.425 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.430 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.432 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.433 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.434 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.436 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.437 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.438 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.444 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.445 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.447 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-14T18:38:32.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-14T18:38:32.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.461 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.464 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.467 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.470 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-14T18:38:32.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-14T18:38:32.481 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-14T18:38:32.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.487 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.500 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-14T18:38:32.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-14T18:38:32.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-12-14T18:38:32.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.509 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.510 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.514 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.516 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.518 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.521 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.522 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:32.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-12-14T18:38:32.529 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-12-14T18:38:32.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-12-14T18:38:32.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.535 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.536 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.539 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.543 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.551 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.553 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-12-14T18:38:32.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.557 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.558 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.562 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.564 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.565 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.566 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.567 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.570 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.580 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.581 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-12-14T18:38:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.587 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:38:32.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.590 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-12-14T18:38:32.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "k8sevents", 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_event_retention_days": { 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ceph_event_retention_days", 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7", 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "config_check_secs": { 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_check_secs", 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "10", 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mds_autoscaler", 2023-12-14T18:38:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.674 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.677 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.679 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.684 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.685 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.686 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.689 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.692 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.693 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.697 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.699 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.700 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "noautoscale": { 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "noautoscale", 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "global autoscale flag", 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.705 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.710 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.712 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-12-14T18:38:32.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.723 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-14T18:38:32.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.725 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.726 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.731 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.734 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.736 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-12-14T18:38:32.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.744 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.749 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.750 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-12-14T18:38:32.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-12-14T18:38:32.756 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-12-14T18:38:32.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.761 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.762 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.764 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.767 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rgw", 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.772 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.777 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rook", 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "drive_group_interval": { 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "drive_group_interval", 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300.0", 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.780 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.781 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.782 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.785 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.786 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "storage_class": { 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "storage_class", 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "local", 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.788 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.791 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.794 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.796 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-12-14T18:38:32.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.798 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.799 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.801 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.802 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.803 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-12-14T18:38:32.804 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.806 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-12-14T18:38:32.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.808 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-12-14T18:38:32.809 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.810 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.811 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.813 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.816 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.817 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-12-14T18:38:32.818 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.818 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.821 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.822 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.824 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.825 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.826 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.828 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.829 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.832 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.833 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.834 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.836 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.837 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.838 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.840 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.841 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.842 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-12-14T18:38:32.843 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.844 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.847 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.849 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.851 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.852 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.853 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-12-14T18:38:32.854 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.856 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-12-14T18:38:32.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.858 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.859 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.860 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.861 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.862 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.863 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.864 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-12-14T18:38:32.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-12-14T18:38:32.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.877 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.881 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-12-14T18:38:32.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.884 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.885 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.889 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.892 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.893 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.897 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-12-14T18:38:32.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-12-14T18:38:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-12-14T18:38:32.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.908 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-12-14T18:38:32.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-12-14T18:38:32.911 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.913 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.915 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-14T18:38:32.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "services": { 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "dashboard": "https://172.21.15.31:8443/" 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-12-14T18:38:32.919 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-12-14T18:38:32.920 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2023-12-14T18:38:32.921 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-12-14T18:38:32.922 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-12-14T18:38:32.923 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 88, 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1319666998 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:32.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 174624949 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4041918398 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:32.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 6674827 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:32.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2912138382 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-12-14T18:38:32.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:0", 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 538765851 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-12-14T18:38:32.930 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:38:32.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:31 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3357991454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:38:32.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:31 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/543199982' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-14T18:38:32.932 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:32 smithi204 ceph-mon[43866]: pgmap v333: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:38:32.932 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:31 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3357991454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:38:32.932 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:31 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/543199982' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-14T18:38:32.932 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:32 smithi176 ceph-mon[43506]: pgmap v333: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc4c0a1490 msgr2=0x7fcc4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc4c0a1490 0x7fcc4c0a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006e4a0 tx=0x7fcc50009260 comp rx=0 tx=0).stop 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 msgr2=0x7fcc60114d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc60114d80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc5801c0a0 tx=0x7fcc58018940 comp rx=0 tx=0).stop 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 shutdown_connections 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc60119a40 0x7fcc60114d80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:32.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc60107af0 0x7fcc60118cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:32.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc4c0a1490 0x7fcc4c0a3940 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006e4a0 tx=0x7fcc50009260 comp rx=0 tx=0).stop 2023-12-14T18:38:32.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 --2- 172.21.15.31:0/543199982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc60107ec0 0x7fcc60119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:32.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.066+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 >> 172.21.15.31:0/543199982 conn(0x7fcc6006c4b0 msgr2=0x7fcc6006fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:32.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.069+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 shutdown_connections 2023-12-14T18:38:32.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:31.070+0000 7fcc65f3a700 1 -- 172.21.15.31:0/543199982 wait complete. 2023-12-14T18:38:32.939 DEBUG:tasks.check_counter:Getting stats from a 2023-12-14T18:38:32.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-14T18:38:32.940 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:31 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3357991454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:38:32.940 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:31 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/543199982' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-14T18:38:32.941 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:32 smithi031 ceph-mon[42085]: pgmap v333: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 6 op/s 2023-12-14T18:38:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:38:34.239 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:34 smithi031 ceph-mon[42085]: pgmap v334: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:34.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:34 smithi176 ceph-mon[43506]: pgmap v334: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:34.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:34 smithi204 ceph-mon[43866]: pgmap v334: 129 pgs: 16 active+clean+snaptrim, 55 active+clean+snaptrim_wait, 58 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:35.711 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.002890029, 2023-12-14T18:38:35.712 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000655885, 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:35.713 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 158, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:38:35.714 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 179536, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.013368921, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.008466269, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002571186, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 155, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.014029151, 2023-12-14T18:38:35.715 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000090510 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 179536 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.716 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 220, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 209, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1287584, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 118416, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.042806824, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.014024896, 2023-12-14T18:38:35.717 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.031219640, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.004950074, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 209, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.017428256, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000083388 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:35.718 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1287584, 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 118416 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-12-14T18:38:35.719 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:35.720 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:35.721 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:35.722 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 149011, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 32, 2023-12-14T18:38:35.723 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:35.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-12-14T18:38:35.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 218, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1254114, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 218, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-12-14T18:38:35.726 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 218, 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1254114, 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:35.727 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:38:36.124 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-12-14T18:38:36.124 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-12-14T18:38:36.124 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-12-14T18:38:36.125 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-12-14T18:38:36.125 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-12-14T18:38:36.125 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-12-14T18:38:36.125 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-12-14T18:38:36.125 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-12-14T18:38:36.125 DEBUG:tasks.check_counter:Getting stats from d 2023-12-14T18:38:36.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-14T18:38:36.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:36 smithi031 ceph-mon[42085]: pgmap v335: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:38:36.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:36 smithi204 ceph-mon[43866]: pgmap v335: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:38:36.661 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:38:36.741 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:36 smithi176 ceph-mon[43506]: pgmap v335: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:38:38.158 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:38:38.158 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:38.158 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 878, 2023-12-14T18:38:38.158 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 878, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 102402174, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 272326, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 8, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 4, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.148942520, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.024103842, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.090205725, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.025698770, 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:38.159 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 877, 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.061530123, 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000070159 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.160 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 1113, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1094, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 135831904, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 392211, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 6, 2023-12-14T18:38:38.161 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 5, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.223289653, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.036393893, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.149051570, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.037172018, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1093, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.073963885, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000067670 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1354016, 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 122880 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:38.163 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 919, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 1077, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 63254056, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 707715, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 8, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 6, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.155521925, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.048748302, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.063530576, 2023-12-14T18:38:38.164 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.030870755, 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 1074, 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.106716335, 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000099363 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.165 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 422528 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 1, 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-MDSRank": { 2023-12-14T18:38:38.166 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 4129, 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.123696984, 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000029958 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "finisher-PurgeQueue": { 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "queue_len": 0, 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "complete_latency": { 2023-12-14T18:38:38.167 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "request": 0, 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "reply": 0, 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "reply_latency": { 2023-12-14T18:38:38.168 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_reply": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "forward": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_complete": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_fetch_keys": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_commit": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_split": 0, 2023-12-14T18:38:38.169 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_merge": 0, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes": 72539, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_top": 729, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_bottom": 30346, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pin_tail": 41464, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_pinned": 71497, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_expired": 0, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "inodes_with_caps": 0, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": 0, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "subtrees": 1, 2023-12-14T18:38:38.170 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_hit": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_forward": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_discover": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_remote_ino": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "traverse_lock": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "load_cent": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "q": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "exported": 0, 2023-12-14T18:38:38.171 INFO:teuthology.orchestra.run.smithi031.stdout: "exported_inodes": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "imported": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "imported_inodes": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_dir_fetch": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "openino_peer_discover": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rfiles": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rbytes": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "root_rsnaps": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_set_tag": 0, 2023-12-14T18:38:38.172 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "scrub_file_inodes": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_inode_file_caps": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_revoke": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_grant": 0, 2023-12-14T18:38:38.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_caps_dirty": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_cap_release": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "process_request_cap_release": 0 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache": { 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update": 0, 2023-12-14T18:38:38.174 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_update_receipt": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_try_discover": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_send_discover": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "dir_handle_discover": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_delayed": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_created": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_enqueued": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_reintegrated": 0, 2023-12-14T18:38:38.175 INFO:teuthology.orchestra.run.smithi031.stdout: "strays_migrated": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_processing": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_started": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_completed": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_exportdir": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_flush": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragmentdir": 0, 2023-12-14T18:38:38.176 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_fragstats": 0, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "ireq_inodestats": 0 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log": { 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "evadd": 0, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "evex": 0, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "evtrm": 0, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "ev": 0, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "evexg": 0, 2023-12-14T18:38:38.177 INFO:teuthology.orchestra.run.smithi031.stdout: "evexd": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "segadd": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "segex": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "segtrm": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "seg": 133, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "segexg": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "segexd": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "expos": 14545227, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "wrpos": 0, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rdpos": 303471948, 2023-12-14T18:38:38.178 INFO:teuthology.orchestra.run.smithi031.stdout: "jlat": { 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "replayed": 88742 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mem": { 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ino": 33165, 2023-12-14T18:38:38.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ino+": 65532, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "ino-": 32367, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "dir": 1035, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "dir+": 1385, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "dir-": 350, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "dn": 72539, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "dn+": 87919, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "dn-": 15380, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "cap": 0, 2023-12-14T18:38:38.180 INFO:teuthology.orchestra.run.smithi031.stdout: "cap+": 0, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "cap-": 0, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rss": 224212, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "heap": 207156 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_server": { 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_client_request": 0, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "dispatch_server_request": 0, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_request": 0, 2023-12-14T18:38:38.181 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_client_session": 0, 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "handle_peer_request": 0, 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "req_create_latency": { 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getattr_latency": { 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.182 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getfilelock_latency": { 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "req_link_latency": { 2023-12-14T18:38:38.183 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookup_latency": { 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookuphash_latency": { 2023-12-14T18:38:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupino_latency": { 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupname_latency": { 2023-12-14T18:38:38.185 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupparent_latency": { 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lookupsnap_latency": { 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "req_lssnap_latency": { 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mkdir_latency": { 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mknod_latency": { 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.188 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "req_mksnap_latency": { 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "req_open_latency": { 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.189 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "req_readdir_latency": { 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rename_latency": { 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.190 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "req_renamesnap_latency": { 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "req_snapdiff_latency": { 2023-12-14T18:38:38.191 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmdir_latency": { 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmsnap_latency": { 2023-12-14T18:38:38.192 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "req_rmxattr_latency": { 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setattr_latency": { 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setdirlayout_latency": { 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setfilelock_latency": { 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setlayout_latency": { 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "req_setxattr_latency": { 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "req_symlink_latency": { 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "req_unlink_latency": { 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:38:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "req_getvxattr_latency": { 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessions": { 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "session_count": 1, 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "session_add": 1, 2023-12-14T18:38:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "session_remove": 0, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_open": 1, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "sessions_stale": 0, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "total_load": 0, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "average_load": 0, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "avg_session_uptime": 2043 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-12-14T18:38:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 1366813, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 33237, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-12-14T18:38:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 10168, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 213, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 127146897, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 1053148, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter": { 2023-12-14T18:38:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "op_active": 0, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "op_laggy": 0, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send": 2657, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "op_send_bytes": 21, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "op_resend": 2, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "op_reply": 2657, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "op_latency": { 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2657, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 5.526907312, 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.002080130 2023-12-14T18:38:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "op_inflight": 0, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "oplen_avg": { 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 2657, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 2658 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "op": 2657, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "op_r": 2657, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "op_w": 0, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "op_rmw": 0, 2023-12-14T18:38:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "op_pg": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_stat": 1711, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_create": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_read": 944, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_write": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writefull": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_writesame": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_append": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_zero": 0, 2023-12-14T18:38:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_truncate": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_delete": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_mapext": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_sparse_read": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_clonerange": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_getxattr": 1, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_setxattr": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_rmxattr": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:38:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_call": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_watch": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_notify": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "osdop_other": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_active": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_send": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_resend": 0, 2023-12-14T18:38:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "linger_ping": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_active": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_send": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "poolop_resend": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_active": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_send": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "poolstat_resend": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_active": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_send": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "statfs_resend": 0, 2023-12-14T18:38:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "command_active": 0, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "command_send": 0, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "command_resend": 0, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "map_epoch": 99, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "map_full": 0, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "map_inc": 14, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_sessions": 11, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_open": 11, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_session_close": 0, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_laggy": 0, 2023-12-14T18:38:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_wr": 0, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_rd": 2, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_del": 0 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "oft": { 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_objs": 0, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_updates": 0, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "omap_total_removes": 0 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "osd-slow-ops": { 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "slow_ops_count": 0 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "purge_queue": { 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops": 0, 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_ops_high_water": 0, 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing": 0, 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executing_high_water": 0, 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_executed": 0, 2023-12-14T18:38:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "pq_item_in_journal": 0 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 2906, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 301210986, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 2906, 2023-12-14T18:38:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2906, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 301210986, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_bytes": { 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2657, 2023-12-14T18:38:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 299392974, 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2657, 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 299392974, 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-objecter_ops": { 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 1024, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 2657, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 2657, 2023-12-14T18:38:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 2657, 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 2657, 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2023-12-14T18:38:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-write_buf_throttle-0x55eae68b2460": { 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 3758096384, 2023-12-14T18:38:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 0, 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:38.224 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:38:38.545 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:38 smithi031 ceph-mon[42085]: pgmap v336: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:38.546 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2023-12-14T18:38:38.546 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2023-12-14T18:38:38.546 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2023-12-14T18:38:38.546 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2023-12-14T18:38:38.547 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2023-12-14T18:38:38.547 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:38:38.547 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:38:38.547 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2023-12-14T18:38:38.547 DEBUG:tasks.check_counter:Getting stats from g 2023-12-14T18:38:38.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-14T18:38:38.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:38 smithi204 ceph-mon[43866]: pgmap v336: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:38.741 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:38 smithi176 ceph-mon[43506]: pgmap v336: 129 pgs: 12 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 91 active+clean; 544 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:38.904 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 0, 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 2, 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 0, 2023-12-14T18:38:40.140 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.001881802, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000491310, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:40.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:40.142 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 6, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 161, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 3312, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 180160, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014476818, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.009322548, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002628881, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:40.143 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 159, 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.016084017, 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000101157 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.144 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 180160 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 217, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 208, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1284864, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 118560, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2023-12-14T18:38:40.145 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.042969212, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.014248579, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.030498032, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.005158339, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 207, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.013674828, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000066061 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.146 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1284864, 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 118560 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2023-12-14T18:38:40.147 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:40.148 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:40.149 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:40.150 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:40.151 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148977, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2023-12-14T18:38:40.152 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:40.153 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-12-14T18:38:40.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:40.154 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:40.154 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:40.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 219, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1254244, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 219, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 219, 2023-12-14T18:38:40.155 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1254244, 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:40.156 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:38:40.427 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:40 smithi031 ceph-mon[42085]: pgmap v337: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:38:40.527 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-12-14T18:38:40.527 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-12-14T18:38:40.527 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-12-14T18:38:40.528 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-12-14T18:38:40.528 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-12-14T18:38:40.528 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-12-14T18:38:40.528 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-12-14T18:38:40.528 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-12-14T18:38:40.528 DEBUG:tasks.check_counter:Getting stats from j 2023-12-14T18:38:40.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-14T18:38:40.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:40 smithi204 ceph-mon[43866]: pgmap v337: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:38:40.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:40 smithi176 ceph-mon[43506]: pgmap v337: 129 pgs: 8 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 102 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-12-14T18:38:40.883 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:38:42.026 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-12-14T18:38:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 4, 2023-12-14T18:38:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 4, 2023-12-14T18:38:42.026 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 2992, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 448, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 4, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 0, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.002772647, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.000045389, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.000956245, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 3, 2023-12-14T18:38:42.027 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.001059797, 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000353265 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-14T18:38:42.028 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 448 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 221, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 212, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 1285952, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 120928, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 2, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 1, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.042237824, 2023-12-14T18:38:42.029 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.013824985, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.030157966, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.005205256, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 211, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.015815847, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000074956 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:42.030 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 1285952, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 120928 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_messages": 2, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages": 162, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_bytes": 180240, 2023-12-14T18:38:42.031 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_total_time": 0.014127398, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_send_time": 0.009602278, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_recv_time": 0.002180052, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 159, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.019155381, 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000120474 2023-12-14T18:38:42.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "msgr_send_encrypted_bytes": 180240 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "cct": { 2023-12-14T18:38:42.033 INFO:teuthology.orchestra.run.smithi031.stdout: "total_workers": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "unhealthy_workers": 0 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool": { 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:42.034 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:42.035 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:42.036 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_items": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_bytes": 148977, 2023-12-14T18:38:42.037 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_anon_items": 30, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bytes": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_items": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_bytes": 0, 2023-12-14T18:38:42.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_items": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "pgmap_items": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_bytes": 928, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_co_items": 2, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_1_items": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:42.039 INFO:teuthology.orchestra.run.smithi031.stdout: "unittest_2_items": 0 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "val": 0, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": 104857600, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get_started": 0, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get": 223, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get_sum": 1254764, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "get_or_fail_success": 223, 2023-12-14T18:38:42.040 INFO:teuthology.orchestra.run.smithi031.stdout: "take": 0, 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "take_sum": 0, 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "put": 223, 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "put_sum": 1254764, 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "wait": { 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "avgcount": 0, 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "sum": 0.000000000, 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: "avgtime": 0.000000000 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:42.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-12-14T18:38:42.042 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-12-14T18:38:42.360 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:42 smithi031 ceph-mon[42085]: pgmap v338: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-12-14T18:38:42.421 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-14T18:38:42.421 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-14T18:38:42.421 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-12-14T18:38:42.421 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-12-14T18:38:42.421 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-12-14T18:38:42.422 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-12-14T18:38:42.422 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-12-14T18:38:42.422 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-12-14T18:38:42.422 DEBUG:tasks.check_counter:Getting stats from b 2023-12-14T18:38:42.422 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-14T18:38:42.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:42 smithi204 ceph-mon[43866]: pgmap v338: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-12-14T18:38:42.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:42 smithi176 ceph-mon[43506]: pgmap v338: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-12-14T18:38:42.758 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:38:44.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:44 smithi176 ceph-mon[43506]: pgmap v339: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:44.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:44 smithi176 ceph-mon[43506]: osdmap e100: 12 total, 12 up, 12 in 2023-12-14T18:38:44.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:44 smithi204 ceph-mon[43866]: pgmap v339: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:44.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:44 smithi204 ceph-mon[43866]: osdmap e100: 12 total, 12 up, 12 in 2023-12-14T18:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2023-12-14T18:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 51745, 2023-12-14T18:38:44.607 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 52832, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 57795456, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 199324646, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 29, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 6, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 2.999865596, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 1.131477115, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 1.794019286, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.388350001, 2023-12-14T18:38:44.608 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 52832, 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 5.117654997, 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000096866 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 85335, 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.012669206, 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000148 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.609 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 10456, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 10566, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 4458400, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 98402134, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 11, 2023-12-14T18:38:44.610 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 5, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 1.112028831, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.443226589, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.190590620, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.441640044, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 10566, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 1.659482331, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000157058 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.611 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 8, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000001372, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000171 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1373392, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 447312 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 110991, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 182358, 2023-12-14T18:38:44.612 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 36866103, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 161035228, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 27, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 7, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 6.046085181, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 2.576879165, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 452.395023146, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.662178743, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 182353, 2023-12-14T18:38:44.613 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 27.997910395, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000153536 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 216772, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.033347062, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000153 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 193008 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.614 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1313, 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 60.750451844, 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.046268432 2023-12-14T18:38:44.615 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 31102, 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 9.747712755, 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000313411 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2023-12-14T18:38:44.616 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 58304, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 51420, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 51420, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 109.958218973, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002138432 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 6884, 2023-12-14T18:38:44.617 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 86, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 939, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 116, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 52802, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 1166, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 23609, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 28027, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 51136, 2023-12-14T18:38:44.618 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 4186, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 0, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 0, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 5, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 78059, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 71040, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 6613, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 8, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 59, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2023-12-14T18:38:44.619 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 0, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 1000, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 1, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 48, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 3899, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 133, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 0, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 11, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 7, 2023-12-14T18:38:44.620 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 246, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 6, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 11, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 4, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 4, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 7, 2023-12-14T18:38:44.621 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 4, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 39, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 32110, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 33240, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 33052, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 255, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 29929 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.622 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 1845, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 3275, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 1640, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 1770, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 1972, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 2, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 17974, 2023-12-14T18:38:44.623 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 17972, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 48, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2023-12-14T18:38:44.624 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 121, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 84467, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 84467, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2023-12-14T18:38:44.625 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 120, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 120, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 272364524, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 0, 2023-12-14T18:38:44.626 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 595, 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 56.570248022, 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.095076047 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 0 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 32836, 2023-12-14T18:38:44.627 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 54616, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 21780, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 1048, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 2861, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 1813, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 52802, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 74960, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 22158, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 0, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 35692, 2023-12-14T18:38:44.628 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 35692, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 255896, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 198964 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 59472, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 25, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 58304, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 31, 2023-12-14T18:38:44.629 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 53, 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 14027, 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 27.123456152, 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001933660 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4, 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.448203573, 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.112050893 2023-12-14T18:38:44.630 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.631 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 17974, 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 26.096754028, 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.001451916 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.632 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.633 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.634 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.635 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.636 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2023-12-14T18:38:44.637 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1441, 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 14.583828568, 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.010120630 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2023-12-14T18:38:44.638 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2023-12-14T18:38:44.639 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2, 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.065978922, 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.032989461 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2023-12-14T18:38:44.640 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2023-12-14T18:38:44.641 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.642 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.643 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 17972, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 41.639997730, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002316937 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:38:44.644 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 1, 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 3, 2023-12-14T18:38:44.645 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 2, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 1, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 18246, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 18246, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 602 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:44.646 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:44.647 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:44.648 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:44.649 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 1049431, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 21914, 2023-12-14T18:38:44.650 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 88, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 1, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 9928, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 207, 2023-12-14T18:38:44.651 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 111662447, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 931194, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:44.652 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 38239, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 281381632, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 27, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 38239, 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2023-12-14T18:38:44.653 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 38239, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 381.197982159, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.009968827 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 38239, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 39338 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 38239, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 269, 2023-12-14T18:38:44.654 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 37970, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 0, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 10, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 172, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 767, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 79, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2023-12-14T18:38:44.655 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 36135, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 11, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 20, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2023-12-14T18:38:44.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2023-12-14T18:38:44.657 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2023-12-14T18:38:44.658 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 100, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 15, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 12, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 12, 2023-12-14T18:38:44.659 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 1941, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 172, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 31 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 1, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2023-12-14T18:38:44.660 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 58498 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 16, 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 8, 2023-12-14T18:38:44.661 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 17972, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 173188, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 83515087, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 173188, 2023-12-14T18:38:44.662 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 173188, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 83515087, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.663 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 38239, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 283604124, 2023-12-14T18:38:44.664 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 38239, 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 283604124, 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:44.665 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 38239, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 38239, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 38239, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 38239, 2023-12-14T18:38:44.666 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2023-12-14T18:38:44.667 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 17972, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 2223868, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 17972, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 172, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 2223868, 2023-12-14T18:38:44.668 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x55d96f34c0a0": { 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:44.669 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 84467, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 268170220, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 84467, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 595, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 268170220, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:44.670 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:44.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.671 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:44.671 INFO:teuthology.orchestra.run.smithi176.stdout:} 2023-12-14T18:38:44.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:44 smithi031 ceph-mon[42085]: pgmap v339: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:38:44.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:44 smithi031 ceph-mon[42085]: osdmap e100: 12 total, 12 up, 12 in 2023-12-14T18:38:44.996 INFO:tasks.check_counter:Daemon mds.b mds.exported=48 2023-12-14T18:38:44.996 INFO:tasks.check_counter:Daemon mds.b mds.imported=133 2023-12-14T18:38:44.996 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1845 2023-12-14T18:38:44.996 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3275 2023-12-14T18:38:44.996 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-12-14T18:38:44.997 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:38:44.997 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:38:44.997 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=116 2023-12-14T18:38:44.997 DEBUG:tasks.check_counter:Getting stats from e 2023-12-14T18:38:44.997 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-14T18:38:45.307 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:38:46.299 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 0, 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 2, 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 0, 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 4, 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 0, 2023-12-14T18:38:46.300 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.001280537, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.000263519, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:46.301 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 2, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 162, 2023-12-14T18:38:46.302 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 180240, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.010882148, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.007002359, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.001473212, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 159, 2023-12-14T18:38:46.303 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.012187357, 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000076650 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 180240 2023-12-14T18:38:46.304 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 225, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 214, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 1288944, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 121376, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 2, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 1, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.040252933, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.012153474, 2023-12-14T18:38:46.305 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.028925767, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.005368380, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 214, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.026223730, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000122540 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:46.306 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1288944, 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 121376 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 0, 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:46.307 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:46.308 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:46.309 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:46.310 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 149057, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 32, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2023-12-14T18:38:46.311 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2023-12-14T18:38:46.312 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 928, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 2, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2023-12-14T18:38:46.313 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 223, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 1254764, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 223, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 223, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 1254764, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:46.314 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:46.315 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:46.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:46.315 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:46.315 INFO:teuthology.orchestra.run.smithi176.stdout:} 2023-12-14T18:38:46.449 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:46 smithi176 ceph-mon[43506]: pgmap v341: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:46.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:46 smithi204 ceph-mon[43866]: pgmap v341: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:46.654 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2023-12-14T18:38:46.654 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2023-12-14T18:38:46.654 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2023-12-14T18:38:46.654 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2023-12-14T18:38:46.654 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-12-14T18:38:46.654 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-12-14T18:38:46.655 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-12-14T18:38:46.655 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2023-12-14T18:38:46.655 DEBUG:tasks.check_counter:Getting stats from h 2023-12-14T18:38:46.655 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-14T18:38:46.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:46 smithi031 ceph-mon[42085]: pgmap v341: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:46.966 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:38:47.925 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2023-12-14T18:38:47.925 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:47.925 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 1476, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 1475, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 92760438, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 457936, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 8, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 4, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.172201188, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.041180017, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.077266199, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.041051585, 2023-12-14T18:38:47.926 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 1474, 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.075028267, 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000050901 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.927 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 448 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 570, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 729, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 79860813, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 602518, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 9, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 7, 2023-12-14T18:38:47.928 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.091713284, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.017598895, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.049028832, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.019140711, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 726, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.040602630, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000055926 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.929 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 425808 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 865, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 846, 2023-12-14T18:38:47.930 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 97744223, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 315073, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 4, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.130286742, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.025067862, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.086133165, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.021468181, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 845, 2023-12-14T18:38:47.931 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.043555890, 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000051545 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1355328, 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 123808 2023-12-14T18:38:47.932 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 1, 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-MDSRank": { 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 4176, 2023-12-14T18:38:47.933 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.108155682, 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000025899 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "finisher-PurgeQueue": { 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "queue_len": 0, 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "complete_latency": { 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.934 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "request": 0, 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "reply": 0, 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "reply_latency": { 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.935 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_reply": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "forward": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_complete": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_fetch_keys": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_commit": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_split": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_merge": 0, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes": 70827, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_top": 1203, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_bottom": 33170, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pin_tail": 36454, 2023-12-14T18:38:47.936 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_pinned": 69108, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_expired": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "inodes_with_caps": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "caps": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "subtrees": 3, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_hit": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_forward": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_discover": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:38:47.937 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_remote_ino": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "traverse_lock": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "load_cent": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "q": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "exported": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "exported_inodes": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "imported": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "imported_inodes": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_dir_fetch": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:38:47.938 INFO:teuthology.orchestra.run.smithi176.stdout: "openino_peer_discover": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rfiles": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rbytes": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "root_rsnaps": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_set_tag": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:38:47.939 INFO:teuthology.orchestra.run.smithi176.stdout: "scrub_file_inodes": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_inode_file_caps": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_revoke": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_grant": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_caps_dirty": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_cap_release": 0, 2023-12-14T18:38:47.940 INFO:teuthology.orchestra.run.smithi176.stdout: "process_request_cap_release": 0 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_cache": { 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_update_receipt": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_try_discover": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_send_discover": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "dir_handle_discover": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_delayed": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:38:47.941 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_created": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_enqueued": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_reintegrated": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "strays_migrated": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_processing": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_started": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "recovery_completed": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:38:47.942 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_exportdir": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_flush": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragmentdir": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_fragstats": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ireq_inodestats": 0 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_log": { 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "evadd": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "evex": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "evtrm": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "ev": 0, 2023-12-14T18:38:47.943 INFO:teuthology.orchestra.run.smithi176.stdout: "evexg": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "evexd": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "segadd": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "segex": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "segtrm": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "seg": 120, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "segexg": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "segexd": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "expos": 4194304, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "wrpos": 0, 2023-12-14T18:38:47.944 INFO:teuthology.orchestra.run.smithi176.stdout: "rdpos": 272363950, 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "jlat": { 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "replayed": 84467 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_mem": { 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "ino": 32856, 2023-12-14T18:38:47.945 INFO:teuthology.orchestra.run.smithi176.stdout: "ino+": 54018, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "ino-": 21162, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "dir": 1043, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "dir+": 1210, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "dir-": 167, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "dn": 70827, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "dn+": 74737, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "dn-": 3910, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "cap": 0, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "cap+": 0, 2023-12-14T18:38:47.946 INFO:teuthology.orchestra.run.smithi176.stdout: "cap-": 0, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "rss": 203548, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "heap": 207156 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_server": { 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_client_request": 0, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "dispatch_server_request": 0, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_request": 0, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_client_session": 0, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "handle_peer_request": 0, 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "req_create_latency": { 2023-12-14T18:38:47.947 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getattr_latency": { 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getfilelock_latency": { 2023-12-14T18:38:47.948 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "req_link_latency": { 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookup_latency": { 2023-12-14T18:38:47.949 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookuphash_latency": { 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupino_latency": { 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.950 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupname_latency": { 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupparent_latency": { 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.951 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lookupsnap_latency": { 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "req_lssnap_latency": { 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.952 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mkdir_latency": { 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mknod_latency": { 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.953 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "req_mksnap_latency": { 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "req_open_latency": { 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.954 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "req_readdir_latency": { 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rename_latency": { 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.955 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "req_renamesnap_latency": { 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "req_snapdiff_latency": { 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.956 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmdir_latency": { 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmsnap_latency": { 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.957 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "req_rmxattr_latency": { 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setattr_latency": { 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.958 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setdirlayout_latency": { 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setfilelock_latency": { 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.959 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setlayout_latency": { 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "req_setxattr_latency": { 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.960 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "req_symlink_latency": { 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "req_unlink_latency": { 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.961 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "req_getvxattr_latency": { 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_sessions": { 2023-12-14T18:38:47.962 INFO:teuthology.orchestra.run.smithi176.stdout: "session_count": 1, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "session_add": 3, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "session_remove": 2, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_open": 2, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "sessions_stale": 0, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "total_load": 0, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "average_load": 0, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "avg_session_uptime": 3428 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2023-12-14T18:38:47.963 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:47.964 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:47.965 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:47.966 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 1355438, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 32924, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2023-12-14T18:38:47.967 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 9928, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 207, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2023-12-14T18:38:47.968 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 125230946, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 1037218, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "objecter": { 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "op_active": 0, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "op_laggy": 0, 2023-12-14T18:38:47.969 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send": 2654, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "op_send_bytes": 21, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "op_resend": 3, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "op_reply": 2654, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "op_latency": { 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2654, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 5.337944770, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.002011282 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "op_inflight": 0, 2023-12-14T18:38:47.970 INFO:teuthology.orchestra.run.smithi176.stdout: "oplen_avg": { 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 2654, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 2655 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "op": 2654, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "op_r": 2654, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "op_w": 0, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "op_rmw": 0, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "op_pg": 0, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_stat": 1724, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_create": 0, 2023-12-14T18:38:47.971 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_read": 928, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_write": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writefull": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_writesame": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_append": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_zero": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_truncate": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_delete": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_mapext": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_sparse_read": 0, 2023-12-14T18:38:47.972 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_clonerange": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_getxattr": 1, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_setxattr": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_rmxattr": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_call": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_watch": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_notify": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:38:47.973 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "osdop_other": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_active": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_send": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_resend": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "linger_ping": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_active": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_send": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "poolop_resend": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_active": 0, 2023-12-14T18:38:47.974 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_send": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "poolstat_resend": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_active": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_send": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "statfs_resend": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "command_active": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "command_send": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "command_resend": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "map_epoch": 100, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "map_full": 0, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "map_inc": 15, 2023-12-14T18:38:47.975 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_sessions": 11, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_open": 11, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_session_close": 0, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_laggy": 0, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_wr": 0, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_rd": 2, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_del": 0 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "oft": { 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_objs": 0, 2023-12-14T18:38:47.976 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_updates": 0, 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "omap_total_removes": 0 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "osd-slow-ops": { 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "slow_ops_count": 0 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "purge_queue": { 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops": 0, 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_ops_high_water": 0, 2023-12-14T18:38:47.977 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing": 0, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executing_high_water": 0, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_executed": 0, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "pq_item_in_journal": 0 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 2907, 2023-12-14T18:38:47.978 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 270088086, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 2907, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 2907, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 270088086, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.979 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_bytes": { 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:47.980 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 2654, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 268267944, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 2654, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 268267944, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.981 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-objecter_ops": { 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 1024, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 2654, 2023-12-14T18:38:47.982 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 2654, 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 2654, 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 2654, 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:38:47.983 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 0, 2023-12-14T18:38:47.984 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-write_buf_throttle-0x55d66a824460": { 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 3758096384, 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:47.985 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:47.986 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:47.987 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:47.987 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.987 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:47.987 INFO:teuthology.orchestra.run.smithi176.stdout:} 2023-12-14T18:38:48.296 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-12-14T18:38:48.296 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-12-14T18:38:48.296 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-12-14T18:38:48.296 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-12-14T18:38:48.296 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2023-12-14T18:38:48.297 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:38:48.297 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:38:48.297 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-12-14T18:38:48.297 DEBUG:tasks.check_counter:Getting stats from k 2023-12-14T18:38:48.297 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-14T18:38:48.492 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:48 smithi176 ceph-mon[43506]: pgmap v342: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:48.589 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:48 smithi204 ceph-mon[43866]: pgmap v342: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:48.608 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:38:48.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:48 smithi031 ceph-mon[42085]: pgmap v342: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:49.541 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2023-12-14T18:38:49.541 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:49.541 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 0, 2023-12-14T18:38:49.541 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 2, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 0, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 4, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 0, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.001155314, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.000299133, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:49.542 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.543 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 2, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 164, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 180592, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.010281600, 2023-12-14T18:38:49.544 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.006278938, 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.001639670, 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 161, 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.021513922, 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000133626 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:49.545 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 180592 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_messages": 227, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages": 216, 2023-12-14T18:38:49.546 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_bytes": 1289488, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_bytes": 122560, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_created_connections": 2, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_active_connections": 1, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_total_time": 0.041512668, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_send_time": 0.012834242, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_recv_time": 0.029057628, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_running_fast_dispatch_time": 0.005772317, 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:49.547 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 216, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.014923147, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000069088 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_recv_encrypted_bytes": 1289488, 2023-12-14T18:38:49.548 INFO:teuthology.orchestra.run.smithi176.stdout: "msgr_send_encrypted_bytes": 122560 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "cct": { 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "total_workers": 0, 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "unhealthy_workers": 0 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "mempool": { 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:49.549 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:49.550 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:49.551 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_items": 0, 2023-12-14T18:38:49.552 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_bytes": 148977, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_anon_items": 30, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_bytes": 0, 2023-12-14T18:38:49.553 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_items": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_bytes": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_items": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:49.554 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "pgmap_items": 0, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_bytes": 928, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_co_items": 2, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_1_items": 0, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "unittest_2_items": 0 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:49.555 INFO:teuthology.orchestra.run.smithi176.stdout: "val": 0, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "max": 104857600, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "get_started": 0, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "get": 225, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "get_sum": 1255024, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "get_or_fail_success": 225, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "take": 0, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "take_sum": 0, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "put": 225, 2023-12-14T18:38:49.556 INFO:teuthology.orchestra.run.smithi176.stdout: "put_sum": 1255024, 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout: "wait": { 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout: "avgcount": 0, 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout: "sum": 0.000000000, 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout: "avgtime": 0.000000000 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout: } 2023-12-14T18:38:49.557 INFO:teuthology.orchestra.run.smithi176.stdout:} 2023-12-14T18:38:49.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:49 smithi176 ceph-mon[43506]: pgmap v343: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:49.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:49 smithi176 ceph-mon[43506]: osdmap e101: 12 total, 12 up, 12 in 2023-12-14T18:38:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:49 smithi204 ceph-mon[43866]: pgmap v343: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:49.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:49 smithi204 ceph-mon[43866]: osdmap e101: 12 total, 12 up, 12 in 2023-12-14T18:38:49.946 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-12-14T18:38:49.946 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-12-14T18:38:49.946 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-12-14T18:38:49.947 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-12-14T18:38:49.947 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-12-14T18:38:49.947 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-12-14T18:38:49.947 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-12-14T18:38:49.947 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-12-14T18:38:49.947 DEBUG:tasks.check_counter:Getting stats from c 2023-12-14T18:38:49.947 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-14T18:38:49.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:49 smithi031 ceph-mon[42085]: pgmap v343: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:49.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:49 smithi031 ceph-mon[42085]: osdmap e101: 12 total, 12 up, 12 in 2023-12-14T18:38:50.542 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:38:51.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:51 smithi204 ceph-mon[43866]: pgmap v345: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:38:51.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:51 smithi031 ceph-mon[42085]: pgmap v345: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:38:51.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:51 smithi176 ceph-mon[43506]: pgmap v345: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:38:53.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:53 smithi204 ceph-mon[43866]: pgmap v346: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2023-12-14T18:38:53.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:53 smithi031 ceph-mon[42085]: pgmap v346: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2023-12-14T18:38:53.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:53 smithi176 ceph-mon[43506]: pgmap v346: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2023-12-14T18:38:54.014 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2023-12-14T18:38:54.014 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 57338, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 57393, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 74265503, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 143626248, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 29, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 3.274226732, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 1.258528503, 2023-12-14T18:38:54.015 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 1.602210184, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.576909220, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 57392, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 8.521743359, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000148483 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 72636, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.011642398, 2023-12-14T18:38:54.016 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000160 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1367456, 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 133056 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 114099, 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 192321, 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 36587798, 2023-12-14T18:38:54.017 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 191711255, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 15, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 6.350563413, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 2.818114694, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 273.868675473, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.525098163, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 192319, 2023-12-14T18:38:54.018 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 29.538435101, 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000153590 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 224640, 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.047929374, 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000213 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-14T18:38:54.019 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 431056 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 47084, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 75943, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 35796557, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 151565332, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 20, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 3.666943757, 2023-12-14T18:38:54.020 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 1.576613585, 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 1.638682875, 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.571234890, 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 75941, 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 9.121223514, 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000120109 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:54.021 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 80208, 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.015412482, 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000192 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2023-12-14T18:38:54.022 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 7410, 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 89.278726215, 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.012048411 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.023 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 31512, 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 15.503374516, 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000491983 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 57176, 2023-12-14T18:38:54.024 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 50262, 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 50262, 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 219.519510271, 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.004367504 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 6914, 2023-12-14T18:38:54.025 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 72, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 755, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 93, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 53157, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 718, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 29990, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 22449, 2023-12-14T18:38:54.026 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 52129, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 14649, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 1, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 1, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 4, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 79565, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 70214, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 6892, 2023-12-14T18:38:54.027 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 64, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 362, 2023-12-14T18:38:54.028 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 15366, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 344, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 16578, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 7, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 246, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2023-12-14T18:38:54.029 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 16578, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 4, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 4, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 16574, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 8, 2023-12-14T18:38:54.030 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 2540, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 29270, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 253, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 80, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 36509, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 33968, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 958, 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 39840 2023-12-14T18:38:54.031 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 1246, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 2621, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 1313, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 1322, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 1262, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 2, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:38:54.032 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 19375, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 19373, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2023-12-14T18:38:54.033 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 403, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 104, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 88742, 2023-12-14T18:38:54.034 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 4021, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 4021, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 84721, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 1904, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 133, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 5, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 5, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 130, 2023-12-14T18:38:54.035 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 2, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 14545227, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 303476445, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1875, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 321.324461853, 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.171373046 2023-12-14T18:38:54.036 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 33158, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 65651, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 32493, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1047, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 2699, 2023-12-14T18:38:54.037 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 1652, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 53157, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 87179, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 34022, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 1, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 53319, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 53318, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 334624, 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 198964 2023-12-14T18:38:54.038 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 67119, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 13, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 57176, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 255, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 58, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 9476, 2023-12-14T18:38:54.039 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 54.334497950, 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005733906 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.427442043, 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.213721021 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2023-12-14T18:38:54.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.041 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 18633, 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 40.921815591, 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002196201 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.042 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1, 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000088963, 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000088963 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.043 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.044 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2023-12-14T18:38:54.045 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.046 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.047 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2775, 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 25.327447003, 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.009127007 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.048 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2023-12-14T18:38:54.049 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.091308184, 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.045654092 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2023-12-14T18:38:54.050 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.051 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.052 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.053 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.054 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 19373, 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 98.416910537, 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005080106 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2023-12-14T18:38:54.055 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 1, 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 0, 2023-12-14T18:38:54.056 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 23008, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 23008, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 655 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:54.057 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:54.058 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:54.059 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2023-12-14T18:38:54.060 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 1050935, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 21526, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 1760, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 20, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2023-12-14T18:38:54.061 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9808, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 204, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:54.062 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 112616257, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 947386, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2023-12-14T18:38:54.063 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 59228, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 310987896, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 59228, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 59228, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 445.539529184, 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.007522447 2023-12-14T18:38:54.064 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 59228, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 60220 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 59228, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 17029, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 42199, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2023-12-14T18:38:54.065 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 68, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 379, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 2254, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 156, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2023-12-14T18:38:54.066 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 38924, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 16578, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 107, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2023-12-14T18:38:54.067 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2023-12-14T18:38:54.068 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2023-12-14T18:38:54.069 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 101, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 16, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2023-12-14T18:38:54.070 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 1580, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 144, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 30 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:38:54.071 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 57699 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 16, 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2023-12-14T18:38:54.072 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 8, 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 19373, 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:54.073 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 218517, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 126965634, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 218517, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 218517, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 126965634, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.074 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:54.075 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 59228, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 313408529, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 59228, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 313408529, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.076 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:54.077 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 59228, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 59228, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 59228, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 59228, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.078 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 19373, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2421785, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:54.079 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 19373, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 379, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2421785, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.080 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55be4358e0a0": { 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 88742, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 299282141, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 88742, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:54.081 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1875, 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 299282141, 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:54.082 INFO:teuthology.orchestra.run.smithi204.stdout:} 2023-12-14T18:38:54.365 INFO:tasks.check_counter:Daemon mds.c mds.exported=362 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds.imported=344 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1246 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=2621 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:38:54.366 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=93 2023-12-14T18:38:54.366 DEBUG:tasks.check_counter:Getting stats from f 2023-12-14T18:38:54.367 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.f perf dump 2023-12-14T18:38:54.679 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:38:55.671 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1893, 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1874, 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 68193591, 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 635342, 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2023-12-14T18:38:55.672 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.223516229, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.055426098, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.106490658, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.054297355, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1873, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.110236675, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000058855 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.673 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1357184, 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 125920 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 301, 2023-12-14T18:38:55.674 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 463, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 100758748, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 524047, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 9, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 7, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.079059266, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.011114801, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.057319881, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.006936012, 2023-12-14T18:38:55.675 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 460, 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.030311411, 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000065894 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.676 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 430992 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 828, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 827, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 79906684, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 256408, 2023-12-14T18:38:55.677 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.105319083, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.023676199, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.049351185, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.025094456, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 826, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.044546605, 2023-12-14T18:38:55.678 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000053930 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.679 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4411, 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.753015914, 2023-12-14T18:38:55.680 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000170713 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.681 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.682 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 38464, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 26892, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 11560, 2023-12-14T18:38:55.683 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 12, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 46, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 613, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 3, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2023-12-14T18:38:55.684 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2023-12-14T18:38:55.685 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 7, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 246, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:38:55.686 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:38:55.687 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2023-12-14T18:38:55.688 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2023-12-14T18:38:55.689 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2023-12-14T18:38:55.690 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2023-12-14T18:38:55.691 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 250282649, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 0, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 250887014, 2023-12-14T18:38:55.692 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 84700 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 16286, 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 59035, 2023-12-14T18:38:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 42749, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 541, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1783, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 1242, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 38464, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 74482, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 36018, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2023-12-14T18:38:55.694 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 152976, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207156 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2023-12-14T18:38:55.695 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.696 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.697 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.698 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.699 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2023-12-14T18:38:55.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.699 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.699 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.699 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.699 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.700 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.701 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2023-12-14T18:38:55.702 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.703 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.704 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.706 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.707 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.708 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.709 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2023-12-14T18:38:55.710 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.711 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2023-12-14T18:38:55.712 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 6, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 4, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 5758 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2023-12-14T18:38:55.713 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:55.714 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:55.715 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:55.716 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 792787, 2023-12-14T18:38:55.717 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 16373, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:55.718 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9808, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 204, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 64531975, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 532183, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2023-12-14T18:38:55.719 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2760, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2760, 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2023-12-14T18:38:55.720 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2760, 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 6.096912273, 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002209026 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2760, 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2761 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.721 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2760, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2760, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1734, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 1023, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2023-12-14T18:38:55.722 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2023-12-14T18:38:55.723 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:38:55.724 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2023-12-14T18:38:55.725 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 101, 2023-12-14T18:38:55.726 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 16, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 11, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 11, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2023-12-14T18:38:55.727 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.728 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:55.729 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 3018, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 248571391, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 3018, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:55.730 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 3018, 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 248571391, 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:55.731 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2760, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 246733565, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2760, 2023-12-14T18:38:55.732 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 246733565, 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:55.733 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2760, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2760, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2760, 2023-12-14T18:38:55.734 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2760, 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2023-12-14T18:38:55.735 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:55.736 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x558109d40460": { 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:55.737 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:55.738 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.739 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:55.739 INFO:teuthology.orchestra.run.smithi204.stdout:} 2023-12-14T18:38:55.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:55 smithi204 ceph-mon[43866]: pgmap v347: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:55.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:55 smithi031 ceph-mon[42085]: pgmap v347: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:55.991 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:55 smithi176 ceph-mon[43506]: pgmap v347: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:56.023 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-12-14T18:38:56.023 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-12-14T18:38:56.023 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-12-14T18:38:56.023 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-12-14T18:38:56.023 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-12-14T18:38:56.024 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:38:56.024 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:38:56.024 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-12-14T18:38:56.024 DEBUG:tasks.check_counter:Getting stats from i 2023-12-14T18:38:56.024 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.i perf dump 2023-12-14T18:38:56.336 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:38:57.251 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 33521, 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 32249, 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 72565328, 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 128046878, 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 179, 2023-12-14T18:38:57.252 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 2.164398809, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.743041203, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.703587237, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.745937807, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 32251, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 38.806035339, 2023-12-14T18:38:57.253 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001203250 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 13276, 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.002293784, 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000172 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1433600, 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 156768 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.254 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 93396, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 64697, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 86863033, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 139422638, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 16, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 4.557279563, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 1.647947549, 2023-12-14T18:38:57.255 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 2.176565555, 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 1.216280919, 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 64696, 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 77.537218015, 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001198485 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 82515, 2023-12-14T18:38:57.256 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012176542, 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000147 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 445168 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 125551, 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 223823, 2023-12-14T18:38:57.257 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 43647648, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 175104929, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 22, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 8, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 7.353702235, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 2.954778068, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 451.491820354, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 1.105638349, 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:57.258 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 223817, 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 78.354389506, 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000350082 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 250668, 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.052682369, 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000210 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.259 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 441760, 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 134256 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2023-12-14T18:38:57.260 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 9014, 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 72.987927073, 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.008097174 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2023-12-14T18:38:57.261 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 21074, 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 7.983409629, 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000378827 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 62130, 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 54247, 2023-12-14T18:38:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 54247, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 173.714469744, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003202287 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 7883, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 434, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 1, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 8397, 2023-12-14T18:38:57.263 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 156, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 16411, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 11467, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 4933, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 11, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 30, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 41007, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 8, 2023-12-14T18:38:57.264 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 8, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 5, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 77109, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 68936, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 7267, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 24, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 6, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2023-12-14T18:38:57.265 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 445, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 35388, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 378, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 11923, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2023-12-14T18:38:57.266 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 7, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 246, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 11923, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 159, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 16, 2023-12-14T18:38:57.267 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 143, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 11764, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 2, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 705, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 39330, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 1, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 9, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 34195, 2023-12-14T18:38:57.268 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 33293, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 717, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 26611 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 905, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 2129, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 1076, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 1192, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 1050, 2023-12-14T18:38:57.269 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 9, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 12787, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 12778, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:38:57.270 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 16, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 489, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 188, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.271 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 84700, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 84518, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 84518, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 182, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 25, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 132, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 131, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 131, 2023-12-14T18:38:57.272 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 1, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 250282649, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 250887150, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1992, 2023-12-14T18:38:57.273 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 159.812468484, 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.080227142 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 16250, 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 62335, 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 46085, 2023-12-14T18:38:57.274 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 550, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 3201, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 2651, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 16411, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 80057, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 63646, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 8, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 43373, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 43365, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 248156, 2023-12-14T18:38:57.275 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 207156 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 62590, 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 28, 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 62130, 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 225, 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 86, 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2023-12-14T18:38:57.276 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 26520, 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 100.317027032, 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003782693 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 421, 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.157875507, 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000375001 2023-12-14T18:38:57.277 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.278 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 13359, 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 17.492360980, 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001309406 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2023-12-14T18:38:57.279 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1, 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000201038, 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000201038 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.280 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.281 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 11, 2023-12-14T18:38:57.282 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.003633846, 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000330349 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 129, 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.131700493, 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001020934 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2023-12-14T18:38:57.283 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 11, 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.580988247, 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.143726204 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2023-12-14T18:38:57.284 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 918, 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.701686255, 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000764364 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.285 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4, 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.006811101, 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001702775 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.286 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 116, 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.375219555, 2023-12-14T18:38:57.287 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003234651 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 5, 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.026910397, 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.005382079 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 21, 2023-12-14T18:38:57.288 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.027440264, 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001306679 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 5, 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.015257142, 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003051428 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2023-12-14T18:38:57.289 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 22, 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 10.016594942, 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.455299770 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2023-12-14T18:38:57.290 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 34, 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.094319422, 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.002774100 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.291 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 12670, 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 42.766443523, 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.003375409 2023-12-14T18:38:57.292 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.293 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 6, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 4, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 2, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 804, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 402, 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 665 2023-12-14T18:38:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 360, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 360, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:57.295 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:57.296 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:57.297 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:57.298 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 840222, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 14641, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 880, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 10, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:57.299 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 9808, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 204, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 48789586, 2023-12-14T18:38:57.300 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 433090, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2023-12-14T18:38:57.301 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 91061, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 313971299, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 37, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 91061, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 91061, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 35058.218211139, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.384997070 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.302 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 91061, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 165994 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 91061, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 12608, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 78453, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2023-12-14T18:38:57.303 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 7113, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 41283, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 234, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 2210, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 207, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 5, 2023-12-14T18:38:57.304 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 26228, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 11949, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 62305, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2023-12-14T18:38:57.305 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 1, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2023-12-14T18:38:57.306 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2023-12-14T18:38:57.307 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 101, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 21, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 12, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 12, 2023-12-14T18:38:57.308 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 12984, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 870, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 605 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:38:57.309 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 62876 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 16, 2023-12-14T18:38:57.310 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 11, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 12778, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 252464, 2023-12-14T18:38:57.311 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 180283934, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 252464, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 252464, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 180283934, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.312 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:57.313 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 91061, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 315543945, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 91061, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 315543945, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.314 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2023-12-14T18:38:57.315 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 91061, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 91061, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 91061, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 91061, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:38:57.316 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 12778, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1579594, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 12778, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 218, 2023-12-14T18:38:57.317 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1579594, 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x556ac8d560a0": { 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:57.318 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 84700, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 246692846, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 84700, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 1992, 2023-12-14T18:38:57.319 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 246692846, 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:57.320 INFO:teuthology.orchestra.run.smithi204.stdout:} 2023-12-14T18:38:57.615 INFO:tasks.check_counter:Daemon mds.i mds.exported=445 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds.imported=378 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=905 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=2129 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=5 2023-12-14T18:38:57.616 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=156 2023-12-14T18:38:57.616 DEBUG:tasks.check_counter:Getting stats from l 2023-12-14T18:38:57.617 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 37e25396-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.l perf dump 2023-12-14T18:38:57.839 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:57 smithi204 ceph-mon[43866]: pgmap v348: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:57.927 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:38:57.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:57 smithi031 ceph-mon[42085]: pgmap v348: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:57.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:57 smithi176 ceph-mon[43506]: pgmap v348: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:58.918 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 4, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 4, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 2992, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 448, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 0, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.001647716, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.000030923, 2023-12-14T18:38:58.919 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.000784443, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000893578, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000297859 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:58.920 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 2, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 167, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 181120, 2023-12-14T18:38:58.921 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.010040812, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.006408866, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.001421383, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 164, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.015440555, 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000094149 2023-12-14T18:38:58.922 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 181120 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 226, 2023-12-14T18:38:58.923 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 217, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1287312, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 123888, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 2, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.041043613, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.012829954, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.028888170, 2023-12-14T18:38:58.924 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.005534587, 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 216, 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.014088862, 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000065226 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:58.925 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1287312, 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 123888 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 0, 2023-12-14T18:38:58.926 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:38:58.927 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:38:58.928 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:38:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2023-12-14T18:38:58.930 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 149007, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 30, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2023-12-14T18:38:58.931 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2023-12-14T18:38:58.932 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 928, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 2, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:38:58.933 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 228, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1255414, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 228, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2023-12-14T18:38:58.934 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 228, 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1255414, 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout: } 2023-12-14T18:38:58.935 INFO:teuthology.orchestra.run.smithi204.stdout:} 2023-12-14T18:38:59.282 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-12-14T18:38:59.282 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-12-14T18:38:59.282 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-12-14T18:38:59.282 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-12-14T18:38:59.282 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-12-14T18:38:59.283 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-12-14T18:38:59.283 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-12-14T18:38:59.283 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-12-14T18:38:59.283 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-14T18:38:59.297 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-14T18:38:59.297 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:38:59.297 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:38:59.342 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-12-14T18:38:59.342 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:38:59.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:38:59.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 -- 172.21.15.31:0/904976621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 msgr2=0x7fa50010b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 --2- 172.21.15.31:0/904976621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa50010b710 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800b0d0 tx=0x7fa4f800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 -- 172.21.15.31:0/904976621 shutdown_connections 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 --2- 172.21.15.31:0/904976621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa50010b710 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 --2- 172.21.15.31:0/904976621 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa500104930 0x7fa500104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 --2- 172.21.15.31:0/904976621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 0x7fa5000feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 -- 172.21.15.31:0/904976621 >> 172.21.15.31:0/904976621 conn(0x7fa500074840 msgr2=0x7fa500074c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 -- 172.21.15.31:0/904976621 shutdown_connections 2023-12-14T18:38:59.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.610+0000 7fa508311700 1 -- 172.21.15.31:0/904976621 wait complete. 2023-12-14T18:38:59.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 Processor -- start 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 -- start start 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 0x7fa500196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa5001969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa500104930 0x7fa50019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50019b460 con 0x7fa5000ff410 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50019b5d0 con 0x7fa500104930 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa508311700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50019b790 con 0x7fa5000fea60 2023-12-14T18:38:59.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa5058ac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa5001969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa5060ad700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 0x7fa500196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa5058ac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa5001969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37708/0 (socket says 172.21.15.31:37708) 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.611+0000 7fa5058ac700 1 -- 172.21.15.31:0/1623595447 learned_addr learned my addr 172.21.15.31:0/1623595447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5068ae700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa500104930 0x7fa50019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5060ad700 1 -- 172.21.15.31:0/1623595447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa500104930 msgr2=0x7fa50019ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5060ad700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa500104930 0x7fa50019ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5060ad700 1 -- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 msgr2=0x7fa5001969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5060ad700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa5001969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5060ad700 1 -- 172.21.15.31:0/1623595447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009d00 con 0x7fa5000fea60 2023-12-14T18:38:59.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5058ac700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa5001969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa5060ad700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 0x7fa500196480 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00e9f0 tx=0x7fa4fc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00c9c0 con 0x7fa5000fea60 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4fc010430 con 0x7fa5000fea60 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50010e180 con 0x7fa5000fea60 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc018680 con 0x7fa5000fea60 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.612+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50010dff0 con 0x7fa5000fea60 2023-12-14T18:38:59.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.613+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50010e310 con 0x7fa5000fea60 2023-12-14T18:38:59.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.614+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa4fc010860 con 0x7fa5000fea60 2023-12-14T18:38:59.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.615+0000 7fa4f77fe700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa4ec0a12c0 0x7fa4ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.615+0000 7fa5058ac700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa4ec0a12c0 0x7fa4ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.615+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7fa4fc014070 con 0x7fa5000fea60 2023-12-14T18:38:59.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.615+0000 7fa5058ac700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa4ec0a12c0 0x7fa4ec0a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa500197a90 tx=0x7fa4f0010040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:59.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.619+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa4fc051730 con 0x7fa5000fea60 2023-12-14T18:38:59.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.829+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa500067e40 con 0x7fa5000fea60 2023-12-14T18:38:59.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.830+0000 7fa4f77fe700 1 -- 172.21.15.31:0/1623595447 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+25252 (secure 0 0 0) 0x7fa4fc096230 con 0x7fa5000fea60 2023-12-14T18:38:59.832 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:38:59.832 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:38:48.564122+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2924181707":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2688246057":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2688246057":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2160431679":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/3108903497":"2023-12-15T18:27:52.189495+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/3199333831":"2023-12-15T18:27:52.189495+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/3645411014":"2023-12-15T18:27:52.189495+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa4ec0a12c0 msgr2=0x7fa4ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa4ec0a12c0 0x7fa4ec0a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa500197a90 tx=0x7fa4f0010040 comp rx=0 tx=0).stop 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 msgr2=0x7fa500196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 0x7fa500196480 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00e9f0 tx=0x7fa4fc00ed00 comp rx=0 tx=0).stop 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 shutdown_connections 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa500104930 0x7fa50019ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fa4ec0a12c0 0x7fa4ec0a3770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5000fea60 0x7fa500196480 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 --2- 172.21.15.31:0/1623595447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5000ff410 0x7fa5001969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 >> 172.21.15.31:0/1623595447 conn(0x7fa500074840 msgr2=0x7fa5001056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 shutdown_connections 2023-12-14T18:38:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.833+0000 7fa508311700 1 -- 172.21.15.31:0/1623595447 wait complete. 2023-12-14T18:38:59.846 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:38:59 smithi031 ceph-mon[42085]: pgmap v349: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:59.848 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:22:37.958695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:25:53.279345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:25:54.669485+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:25:57.764892+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:38:59.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:38:59.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 -- 172.21.15.31:0/432363244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 msgr2=0x7f5c740ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 --2- 172.21.15.31:0/432363244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c740ff760 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5c640099c0 tx=0x7f5c64009cd0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 -- 172.21.15.31:0/432363244 shutdown_connections 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 --2- 172.21.15.31:0/432363244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c741043d0 0x7f5c7410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 --2- 172.21.15.31:0/432363244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c740ff760 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 --2- 172.21.15.31:0/432363244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c740ffd30 0x7f5c74103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 -- 172.21.15.31:0/432363244 >> 172.21.15.31:0/432363244 conn(0x7f5c74075fd0 msgr2=0x7f5c740763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 -- 172.21.15.31:0/432363244 shutdown_connections 2023-12-14T18:38:59.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.988+0000 7f5c790a7700 1 -- 172.21.15.31:0/432363244 wait complete. 2023-12-14T18:38:59.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 Processor -- start 2023-12-14T18:38:59.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 -- start start 2023-12-14T18:38:59.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c74196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c740ffd30 0x7f5c741969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741043d0 0x7f5c7419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419b440 con 0x7f5c740ff390 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419b5b0 con 0x7f5c740ffd30 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.989+0000 7f5c790a7700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7419b770 con 0x7f5c741043d0 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c74196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c74196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37716/0 (socket says 172.21.15.31:37716) 2023-12-14T18:38:59.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 -- 172.21.15.31:0/998667043 learned_addr learned my addr 172.21.15.31:0/998667043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c7359e700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741043d0 0x7f5c7419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c7259c700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c740ffd30 0x7f5c741969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 -- 172.21.15.31:0/998667043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741043d0 msgr2=0x7f5c7419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741043d0 0x7f5c7419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 -- 172.21.15.31:0/998667043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c740ffd30 msgr2=0x7f5c741969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c740ffd30 0x7f5c741969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 -- 172.21.15.31:0/998667043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640095f0 con 0x7f5c740ff390 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c7259c700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c740ffd30 0x7f5c741969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:38:59.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c72d9d700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c74196460 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400b580 tx=0x7f5c6400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6401d070 con 0x7f5c740ff390 2023-12-14T18:38:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c640054f0 con 0x7f5c740ff390 2023-12-14T18:38:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c74110300 con 0x7f5c740ff390 2023-12-14T18:38:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004b30 con 0x7f5c740ff390 2023-12-14T18:38:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.990+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c74110170 con 0x7f5c740ff390 2023-12-14T18:38:59.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.991+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c74110490 con 0x7f5c740ff390 2023-12-14T18:38:59.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:38:59 smithi176 ceph-mon[43506]: pgmap v349: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:38:59.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.993+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5c6400fb80 con 0x7f5c740ff390 2023-12-14T18:38:59.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.993+0000 7f5c5bfff700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5c5c0a12c0 0x7f5c5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:38:59.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.993+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7f5c640ce300 con 0x7f5c740ff390 2023-12-14T18:38:59.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.993+0000 7f5c7259c700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5c5c0a12c0 0x7f5c5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:38:59.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.994+0000 7f5c7259c700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5c5c0a12c0 0x7f5c5c0a3770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009690 tx=0x7f5c68009040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:38:59.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:38:59.997+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5c640973e0 con 0x7f5c740ff390 2023-12-14T18:39:00.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:38:59 smithi204 ceph-mon[43866]: pgmap v349: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:39:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.206+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c741978c0 con 0x7f5c740ff390 2023-12-14T18:39:00.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.206+0000 7f5c5bfff700 1 -- 172.21.15.31:0/998667043 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c6409aa00 con 0x7f5c740ff390 2023-12-14T18:39:00.207 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-12-14T18:39:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5c5c0a12c0 msgr2=0x7f5c5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5c5c0a12c0 0x7f5c5c0a3770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009690 tx=0x7f5c68009040 comp rx=0 tx=0).stop 2023-12-14T18:39:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 msgr2=0x7f5c74196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c74196460 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400b580 tx=0x7f5c6400f690 comp rx=0 tx=0).stop 2023-12-14T18:39:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 shutdown_connections 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f5c5c0a12c0 0x7f5c5c0a3770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c741043d0 0x7f5c7419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c740ff390 0x7f5c74196460 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 --2- 172.21.15.31:0/998667043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c740ffd30 0x7f5c741969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 >> 172.21.15.31:0/998667043 conn(0x7f5c74075fd0 msgr2=0x7f5c74102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 shutdown_connections 2023-12-14T18:39:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.209+0000 7f5c790a7700 1 -- 172.21.15.31:0/998667043 wait complete. 2023-12-14T18:39:00.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:39:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 -- 172.21.15.31:0/1453843376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9140680c0 msgr2=0x7fe914068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 --2- 172.21.15.31:0/1453843376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9140680c0 0x7fe914068490 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fe9040099c0 tx=0x7fe904009cd0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 -- 172.21.15.31:0/1453843376 shutdown_connections 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 --2- 172.21.15.31:0/1453843376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9141054d0 0x7fe91410d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 --2- 172.21.15.31:0/1453843376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9140680c0 0x7fe914068490 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 --2- 172.21.15.31:0/1453843376 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe914068a60 0x7fe914104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 -- 172.21.15.31:0/1453843376 >> 172.21.15.31:0/1453843376 conn(0x7fe914075fd0 msgr2=0x7fe9140763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 -- 172.21.15.31:0/1453843376 shutdown_connections 2023-12-14T18:39:00.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.362+0000 7fe919945700 1 -- 172.21.15.31:0/1453843376 wait complete. 2023-12-14T18:39:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.363+0000 7fe919945700 1 Processor -- start 2023-12-14T18:39:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.363+0000 7fe919945700 1 -- start start 2023-12-14T18:39:00.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9140680c0 0x7fe914196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe914068a60 0x7fe9141968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 0x7fe91419ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419b340 con 0x7fe9141054d0 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419b4b0 con 0x7fe914068a60 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91419b670 con 0x7fe9140680c0 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 0x7fe91419ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 0x7fe91419ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37736/0 (socket says 172.21.15.31:37736) 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 -- 172.21.15.31:0/2645563050 learned_addr learned my addr 172.21.15.31:0/2645563050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:00.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe912ffd700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9140680c0 0x7fe914196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9127fc700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe914068a60 0x7fe9141968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 -- 172.21.15.31:0/2645563050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9140680c0 msgr2=0x7fe914196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9140680c0 0x7fe914196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 -- 172.21.15.31:0/2645563050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe914068a60 msgr2=0x7fe9141968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe914068a60 0x7fe9141968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 -- 172.21.15.31:0/2645563050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040095f0 con 0x7fe9141054d0 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe912ffd700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9140680c0 0x7fe914196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe9137fe700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 0x7fe91419ad00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00ef30 tx=0x7fe90c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:00.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c018070 con 0x7fe9141054d0 2023-12-14T18:39:00.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe914110290 con 0x7fe9141054d0 2023-12-14T18:39:00.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe90c00f040 con 0x7fe9141054d0 2023-12-14T18:39:00.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.364+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c003dc0 con 0x7fe9141054d0 2023-12-14T18:39:00.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.365+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe914110100 con 0x7fe9141054d0 2023-12-14T18:39:00.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.366+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914110420 con 0x7fe9141054d0 2023-12-14T18:39:00.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.367+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe90c00caa0 con 0x7fe9141054d0 2023-12-14T18:39:00.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.368+0000 7fe918943700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe8fc0a12c0 0x7fe8fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.368+0000 7fe912ffd700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe8fc0a12c0 0x7fe8fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.368+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7fe90c0cd6e0 con 0x7fe9141054d0 2023-12-14T18:39:00.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.368+0000 7fe912ffd700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe8fc0a12c0 0x7fe8fc0a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe904009990 tx=0x7fe90401a040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:00.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.371+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe90c0967c0 con 0x7fe9141054d0 2023-12-14T18:39:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.577+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9140679f0 con 0x7fe9141054d0 2023-12-14T18:39:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.577+0000 7fe918943700 1 -- 172.21.15.31:0/2645563050 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe90c099de0 con 0x7fe9141054d0 2023-12-14T18:39:00.578 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:39:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe8fc0a12c0 msgr2=0x7fe8fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe8fc0a12c0 0x7fe8fc0a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe904009990 tx=0x7fe90401a040 comp rx=0 tx=0).stop 2023-12-14T18:39:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 msgr2=0x7fe91419ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 0x7fe91419ad00 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fe90c00ef30 tx=0x7fe90c00c450 comp rx=0 tx=0).stop 2023-12-14T18:39:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 shutdown_connections 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9141054d0 0x7fe91419ad00 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fe8fc0a12c0 0x7fe8fc0a3770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9140680c0 0x7fe914196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 --2- 172.21.15.31:0/2645563050 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe914068a60 0x7fe9141968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 >> 172.21.15.31:0/2645563050 conn(0x7fe914075fd0 msgr2=0x7fe9140fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 shutdown_connections 2023-12-14T18:39:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.580+0000 7fe919945700 1 -- 172.21.15.31:0/2645563050 wait complete. 2023-12-14T18:39:00.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:39:00.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 -- 172.21.15.31:0/2158020423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc40680c0 msgr2=0x7fefc4068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/2158020423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc40680c0 0x7fefc4068490 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fefc00099c0 tx=0x7fefc0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 -- 172.21.15.31:0/2158020423 shutdown_connections 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/2158020423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc41054d0 0x7fefc410d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/2158020423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc40680c0 0x7fefc4068490 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/2158020423 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefc4068a60 0x7fefc4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 -- 172.21.15.31:0/2158020423 >> 172.21.15.31:0/2158020423 conn(0x7fefc4075fd0 msgr2=0x7fefc40763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 -- 172.21.15.31:0/2158020423 shutdown_connections 2023-12-14T18:39:00.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.734+0000 7fefcb9c6700 1 -- 172.21.15.31:0/2158020423 wait complete. 2023-12-14T18:39:00.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 Processor -- start 2023-12-14T18:39:00.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 -- start start 2023-12-14T18:39:00.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc40680c0 0x7fefc4196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefc4068a60 0x7fefc41968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 0x7fefc419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc419b350 con 0x7fefc41054d0 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc419b4c0 con 0x7fefc4068a60 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.735+0000 7fefcb9c6700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc419b680 con 0x7fefc40680c0 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 0x7fefc419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 0x7fefc419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37752/0 (socket says 172.21.15.31:37752) 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 -- 172.21.15.31:0/3153201077 learned_addr learned my addr 172.21.15.31:0/3153201077 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:00.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9762700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc40680c0 0x7fefc4196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc8f61700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefc4068a60 0x7fefc41968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 -- 172.21.15.31:0/3153201077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc40680c0 msgr2=0x7fefc4196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc40680c0 0x7fefc4196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 -- 172.21.15.31:0/3153201077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefc4068a60 msgr2=0x7fefc41968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefc4068a60 0x7fefc41968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 -- 172.21.15.31:0/3153201077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00095f0 con 0x7fefc41054d0 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefc9f63700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 0x7fefc419ad10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fefbc007ee0 tx=0x7fefbc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00ddd0 con 0x7fefc41054d0 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc4110290 con 0x7fefc41054d0 2023-12-14T18:39:00.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefbc00f040 con 0x7fefc41054d0 2023-12-14T18:39:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0048e0 con 0x7fefc41054d0 2023-12-14T18:39:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.736+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc4110100 con 0x7fefc41054d0 2023-12-14T18:39:00.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.737+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc4110420 con 0x7fefc41054d0 2023-12-14T18:39:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.739+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fefbc003a00 con 0x7fefc41054d0 2023-12-14T18:39:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.739+0000 7fefba7fc700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fefb00a12c0 0x7fefb00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.740+0000 7fefc9762700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fefb00a12c0 0x7fefb00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.740+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7fefbc0cd1b0 con 0x7fefc41054d0 2023-12-14T18:39:00.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.740+0000 7fefc9762700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fefb00a12c0 0x7fefb00a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefc0000c00 tx=0x7fefc0003680 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:00.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.743+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fefbc096200 con 0x7fefc41054d0 2023-12-14T18:39:00.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/1623595447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:39:00.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/998667043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:39:00.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:00 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2645563050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:39:00.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.973+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fefc40679f0 con 0x7fefc41054d0 2023-12-14T18:39:00.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.973+0000 7fefba7fc700 1 -- 172.21.15.31:0/3153201077 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7fefbc0230a0 con 0x7fefc41054d0 2023-12-14T18:39:00.974 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-12-14T18:39:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fefb00a12c0 msgr2=0x7fefb00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fefb00a12c0 0x7fefb00a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefc0000c00 tx=0x7fefc0003680 comp rx=0 tx=0).stop 2023-12-14T18:39:00.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 msgr2=0x7fefc419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 0x7fefc419ad10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fefbc007ee0 tx=0x7fefbc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 shutdown_connections 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefc41054d0 0x7fefc419ad10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fefb00a12c0 0x7fefb00a3770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fefc40680c0 0x7fefc4196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 --2- 172.21.15.31:0/3153201077 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefc4068a60 0x7fefc41968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 >> 172.21.15.31:0/3153201077 conn(0x7fefc4075fd0 msgr2=0x7fefc40fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:00.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 shutdown_connections 2023-12-14T18:39:00.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:00.975+0000 7fefcb9c6700 1 -- 172.21.15.31:0/3153201077 wait complete. 2023-12-14T18:39:00.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:39:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/1623595447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:39:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/998667043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:39:00.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:00 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2645563050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:39:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/1623595447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:39:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/998667043' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:39:01.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:00 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2645563050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:39:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 -- 172.21.15.31:0/3420708996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101470 msgr2=0x7fb2a4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 --2- 172.21.15.31:0/3420708996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101470 0x7fb2a4101840 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009990 tx=0x7fb2a0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 -- 172.21.15.31:0/3420708996 shutdown_connections 2023-12-14T18:39:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 --2- 172.21.15.31:0/3420708996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a410a7b0 0x7fb2a410cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 --2- 172.21.15.31:0/3420708996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101470 0x7fb2a4101840 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 --2- 172.21.15.31:0/3420708996 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2a4101d80 0x7fb2a410a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 -- 172.21.15.31:0/3420708996 >> 172.21.15.31:0/3420708996 conn(0x7fb2a40763a0 msgr2=0x7fb2a40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 -- 172.21.15.31:0/3420708996 shutdown_connections 2023-12-14T18:39:01.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.131+0000 7fb2ab588700 1 -- 172.21.15.31:0/3420708996 wait complete. 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 Processor -- start 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 -- start start 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a4101470 0x7fb2a4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 0x7fb2a41968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2a410a7b0 0x7fb2a419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419b360 con 0x7fb2a4101d80 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419b4d0 con 0x7fb2a410a7b0 2023-12-14T18:39:01.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2ab588700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419b690 con 0x7fb2a4101470 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2a8b23700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 0x7fb2a41968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2a8b23700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 0x7fb2a41968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37780/0 (socket says 172.21.15.31:37780) 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2a8b23700 1 -- 172.21.15.31:0/2223949834 learned_addr learned my addr 172.21.15.31:0/2223949834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2a9b25700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2a410a7b0 0x7fb2a419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.132+0000 7fb2a9324700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a4101470 0x7fb2a4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2a8b23700 1 -- 172.21.15.31:0/2223949834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a4101470 msgr2=0x7fb2a4196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2a8b23700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a4101470 0x7fb2a4196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2a8b23700 1 -- 172.21.15.31:0/2223949834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2a410a7b0 msgr2=0x7fb2a419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2a8b23700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2a410a7b0 0x7fb2a419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2a8b23700 1 -- 172.21.15.31:0/2223949834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00095f0 con 0x7fb2a4101d80 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2a8b23700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 0x7fb2a41968c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb29400d980 tx=0x7fb29400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294004220 con 0x7fb2a4101d80 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a4110260 con 0x7fb2a4101d80 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb294007640 con 0x7fb2a4101d80 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294005690 con 0x7fb2a4101d80 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.133+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a41100d0 con 0x7fb2a4101d80 2023-12-14T18:39:01.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.134+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a41050d0 con 0x7fb2a4101d80 2023-12-14T18:39:01.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.135+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb294004750 con 0x7fb2a4101d80 2023-12-14T18:39:01.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.136+0000 7fb29a7fc700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2900a12c0 0x7fb2900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.136+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7fb2940cd340 con 0x7fb2a4101d80 2023-12-14T18:39:01.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.136+0000 7fb2a9324700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2900a12c0 0x7fb2900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.136+0000 7fb2a9324700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2900a12c0 0x7fb2900a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b2e0 tx=0x7fb2a0005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:01.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.139+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb294096390 con 0x7fb2a4101d80 2023-12-14T18:39:01.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.347+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2a4068830 con 0x7fb2a4101d80 2023-12-14T18:39:01.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.348+0000 7fb29a7fc700 1 -- 172.21.15.31:0/2223949834 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v101) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb294016020 con 0x7fb2a4101d80 2023-12-14T18:39:01.349 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-12-14T18:39:01.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2900a12c0 msgr2=0x7fb2900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2900a12c0 0x7fb2900a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000b2e0 tx=0x7fb2a0005fd0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 msgr2=0x7fb2a41968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 0x7fb2a41968c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb29400d980 tx=0x7fb29400dc90 comp rx=0 tx=0).stop 2023-12-14T18:39:01.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 shutdown_connections 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2a410a7b0 0x7fb2a419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb2900a12c0 0x7fb2900a3770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a4101470 0x7fb2a4196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 --2- 172.21.15.31:0/2223949834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2a4101d80 0x7fb2a41968c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 >> 172.21.15.31:0/2223949834 conn(0x7fb2a40763a0 msgr2=0x7fb2a4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 shutdown_connections 2023-12-14T18:39:01.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.350+0000 7fb2ab588700 1 -- 172.21.15.31:0/2223949834 wait complete. 2023-12-14T18:39:01.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-14T18:39:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 -- 172.21.15.31:0/3408110690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c1054f0 msgr2=0x7f098c10d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 --2- 172.21.15.31:0/3408110690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c1054f0 0x7f098c10d0c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f098400b0d0 tx=0x7f098400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 -- 172.21.15.31:0/3408110690 shutdown_connections 2023-12-14T18:39:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 --2- 172.21.15.31:0/3408110690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c1054f0 0x7f098c10d0c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 --2- 172.21.15.31:0/3408110690 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c0694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 --2- 172.21.15.31:0/3408110690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c069a90 0x7f098c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 -- 172.21.15.31:0/3408110690 >> 172.21.15.31:0/3408110690 conn(0x7f098c075fa0 msgr2=0x7f098c0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 -- 172.21.15.31:0/3408110690 shutdown_connections 2023-12-14T18:39:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.503+0000 7f099377e700 1 -- 172.21.15.31:0/3408110690 wait complete. 2023-12-14T18:39:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 Processor -- start 2023-12-14T18:39:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 -- start start 2023-12-14T18:39:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 0x7f098c1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1054f0 0x7f098c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c19b490 con 0x7f098c069a90 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c19b600 con 0x7f098c0690f0 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099377e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c19b7c0 con 0x7f098c1054f0 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 0x7f098c1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099151a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 0x7f098c1969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37812/0 (socket says 172.21.15.31:37812) 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f099151a700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.31:54494/0 (socket says 172.21.15.31:54494) 2023-12-14T18:39:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 -- 172.21.15.31:0/2085992004 learned_addr learned my addr 172.21.15.31:0/2085992004 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0991d1b700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1054f0 0x7f098c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 -- 172.21.15.31:0/2085992004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1054f0 msgr2=0x7f098c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1054f0 0x7f098c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 -- 172.21.15.31:0/2085992004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 msgr2=0x7f098c1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.504+0000 7f0990d19700 1 -- 172.21.15.31:0/2085992004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984009d00 con 0x7f098c069a90 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f099151a700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f0990d19700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 0x7f098c1969f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f097c00d950 tx=0x7f097c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c004220 con 0x7f098c069a90 2023-12-14T18:39:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f097c004cf0 con 0x7f098c069a90 2023-12-14T18:39:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c1102c0 con 0x7f098c069a90 2023-12-14T18:39:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c005620 con 0x7f098c069a90 2023-12-14T18:39:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.505+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c110130 con 0x7f098c069a90 2023-12-14T18:39:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.506+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c110450 con 0x7f098c069a90 2023-12-14T18:39:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.507+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f097c004750 con 0x7f098c069a90 2023-12-14T18:39:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.508+0000 7f09827fc700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f09780a1340 0x7f09780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.508+0000 7f099151a700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f09780a1340 0x7f09780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.508+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7f097c0cd390 con 0x7f098c069a90 2023-12-14T18:39:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.508+0000 7f099151a700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f09780a1340 0x7f09780a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0988011490 tx=0x7f0988009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:01.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.511+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f097c096380 con 0x7f098c069a90 2023-12-14T18:39:01.713 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:01 smithi031 ceph-mon[42085]: pgmap v350: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:39:01.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:01 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3153201077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:39:01.714 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:01 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2223949834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:39:01.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.738+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f098c1a3700 con 0x7f098c069a90 2023-12-14T18:39:01.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.738+0000 7f09827fc700 1 -- 172.21.15.31:0/2085992004 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v101) v1 ==== 81+0+1001 (secure 0 0 0) 0x7f097c016070 con 0x7f098c069a90 2023-12-14T18:39:01.739 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2924181707 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.739 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2117792145 2023-12-15T18:20:25.264468+0000 2023-12-14T18:39:01.739 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/463659266 2023-12-15T18:20:47.403710+0000 2023-12-14T18:39:01.739 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2688246057 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2892448986 2023-12-15T18:20:25.264468+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2094677585 2023-12-15T18:20:25.264468+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2688246057 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2160431679 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3108903497 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2094677585 2023-12-15T18:20:25.264468+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/384230437 2023-12-15T18:20:47.403710+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3199333831 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2330625991 2023-12-15T18:20:47.403710+0000 2023-12-14T18:39:01.740 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2330625991 2023-12-15T18:20:47.403710+0000 2023-12-14T18:39:01.741 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/132110935 2023-12-15T18:20:25.264468+0000 2023-12-14T18:39:01.741 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1680573410 2023-12-15T18:20:47.403710+0000 2023-12-14T18:39:01.741 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3645411014 2023-12-15T18:27:52.189495+0000 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f09780a1340 msgr2=0x7f09780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f09780a1340 0x7f09780a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0988011490 tx=0x7f0988009260 comp rx=0 tx=0).stop 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 msgr2=0x7f098c1969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 0x7f098c1969f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f097c00d950 tx=0x7f097c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 shutdown_connections 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f09780a1340 0x7f09780a37f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f098c1054f0 0x7f098c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f098c0690f0 0x7f098c1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 --2- 172.21.15.31:0/2085992004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f098c069a90 0x7f098c1969f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:01.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 >> 172.21.15.31:0/2085992004 conn(0x7f098c075fa0 msgr2=0x7f098c0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:01.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 shutdown_connections 2023-12-14T18:39:01.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:01.741+0000 7f099377e700 1 -- 172.21.15.31:0/2085992004 wait complete. 2023-12-14T18:39:01.743 INFO:teuthology.orchestra.run.smithi031.stderr:listed 17 entries 2023-12-14T18:39:01.755 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-14T18:39:01.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:39:01.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-14T18:39:01.905 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:39:01.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:39:01.906 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-14T18:39:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:01 smithi176 ceph-mon[43506]: pgmap v350: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:39:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:01 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3153201077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:39:01.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:01 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2223949834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:39:02.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:39:02.001 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-12-14T18:39:02.092 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:01 smithi204 ceph-mon[43866]: pgmap v350: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:39:02.092 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:01 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3153201077' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:39:02.092 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:01 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2223949834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:39:02.184 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-14T18:39:02.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:39:02.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-14T18:39:02.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:39:02.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-12-14T18:39:02.305 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-14T18:39:02.317 INFO:tasks.cephadm:Teardown begin 2023-12-14T18:39:02.317 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-14T18:39:02.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:39:02.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 -- 172.21.15.31:0/1768619700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 msgr2=0x7f8840100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 --2- 172.21.15.31:0/1768619700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f8840100e50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8834009990 tx=0x7f8834009ca0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 -- 172.21.15.31:0/1768619700 shutdown_connections 2023-12-14T18:39:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 --2- 172.21.15.31:0/1768619700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8840101390 0x7f884010d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 --2- 172.21.15.31:0/1768619700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f88401051b0 0x7f8840105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 --2- 172.21.15.31:0/1768619700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f8840100e50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 -- 172.21.15.31:0/1768619700 >> 172.21.15.31:0/1768619700 conn(0x7f8840077520 msgr2=0x7f8840077920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:02.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 -- 172.21.15.31:0/1768619700 shutdown_connections 2023-12-14T18:39:02.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.466+0000 7f8845938700 1 -- 172.21.15.31:0/1768619700 wait complete. 2023-12-14T18:39:02.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.467+0000 7f8845938700 1 Processor -- start 2023-12-14T18:39:02.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.467+0000 7f8845938700 1 -- start start 2023-12-14T18:39:02.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8845938700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f88401963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8845938700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101390 0x7f88401968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8845938700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88401051b0 0x7f884019ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8845938700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019b390 con 0x7f88401009e0 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8845938700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019b500 con 0x7f8840101390 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8845938700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884019b6c0 con 0x7f88401051b0 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f88401963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f88401963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37818/0 (socket says 172.21.15.31:37818) 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 -- 172.21.15.31:0/7382614 learned_addr learned my addr 172.21.15.31:0/7382614 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:02.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883e7fc700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101390 0x7f88401968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883f7fe700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88401051b0 0x7f884019ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 -- 172.21.15.31:0/7382614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88401051b0 msgr2=0x7f884019ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88401051b0 0x7f884019ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 -- 172.21.15.31:0/7382614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101390 msgr2=0x7f88401968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101390 0x7f88401968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 -- 172.21.15.31:0/7382614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340095f0 con 0x7f88401009e0 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883e7fc700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101390 0x7f88401968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:39:02.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f883effd700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f88401963b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9a0 tx=0x7f883000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.468+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830009790 con 0x7f88401009e0 2023-12-14T18:39:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.469+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8840110320 con 0x7f88401009e0 2023-12-14T18:39:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.469+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8830010430 con 0x7f88401009e0 2023-12-14T18:39:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.469+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830018680 con 0x7f88401009e0 2023-12-14T18:39:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.469+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8840110190 con 0x7f88401009e0 2023-12-14T18:39:02.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.470+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88401104b0 con 0x7f88401009e0 2023-12-14T18:39:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.471+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f88300105a0 con 0x7f88401009e0 2023-12-14T18:39:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.472+0000 7f8844936700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88280a12c0 0x7f88280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.472+0000 7f883e7fc700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88280a12c0 0x7f88280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.472+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7f8830014070 con 0x7f88401009e0 2023-12-14T18:39:02.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.473+0000 7f883e7fc700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88280a12c0 0x7f88280a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88401979c0 tx=0x7f8834005fd0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:02.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.476+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f88300961f0 con 0x7f88401009e0 2023-12-14T18:39:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.733+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88401977a0 con 0x7f88401009e0 2023-12-14T18:39:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.733+0000 7f8844936700 1 -- 172.21.15.31:0/7382614 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f8830099810 con 0x7f88401009e0 2023-12-14T18:39:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:39:02.735 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:29.509045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:39:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88280a12c0 msgr2=0x7f88280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88280a12c0 0x7f88280a3770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88401979c0 tx=0x7f8834005fd0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 msgr2=0x7f88401963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f88401963b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9a0 tx=0x7f883000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 shutdown_connections 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f88280a12c0 0x7f88280a3770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88401051b0 0x7f884019ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88401009e0 0x7f88401963b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 --2- 172.21.15.31:0/7382614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8840101390 0x7f88401968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 >> 172.21.15.31:0/7382614 conn(0x7f8840077520 msgr2=0x7f88400ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:02.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 shutdown_connections 2023-12-14T18:39:02.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.736+0000 7f8845938700 1 -- 172.21.15.31:0/7382614 wait complete. 2023-12-14T18:39:02.739 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2023-12-14T18:39:02.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:39:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 -- 172.21.15.31:0/928688217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 msgr2=0x7f1904100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 --2- 172.21.15.31:0/928688217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f1904100920 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009990 tx=0x7f18f4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 -- 172.21.15.31:0/928688217 shutdown_connections 2023-12-14T18:39:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 --2- 172.21.15.31:0/928688217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904109b00 0x7f190410bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 --2- 172.21.15.31:0/928688217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f1904100920 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 --2- 172.21.15.31:0/928688217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1904100e60 0x7f19041095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.888+0000 7f19098b3700 1 -- 172.21.15.31:0/928688217 >> 172.21.15.31:0/928688217 conn(0x7f19040749f0 msgr2=0x7f1904074df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.889+0000 7f19098b3700 1 -- 172.21.15.31:0/928688217 shutdown_connections 2023-12-14T18:39:02.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.889+0000 7f19098b3700 1 -- 172.21.15.31:0/928688217 wait complete. 2023-12-14T18:39:02.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.889+0000 7f19098b3700 1 Processor -- start 2023-12-14T18:39:02.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 -- start start 2023-12-14T18:39:02.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f19041971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904100e60 0x7f19041976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1904109b00 0x7f19041a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1904198000 con 0x7f1904100550 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041a2770 con 0x7f1904109b00 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19098b3700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19041a28e0 con 0x7f1904100e60 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f1902ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f19041971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f1902ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f19041971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37838/0 (socket says 172.21.15.31:37838) 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f1902ffd700 1 -- 172.21.15.31:0/4109865567 learned_addr learned my addr 172.21.15.31:0/4109865567 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19037fe700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1904109b00 0x7f19041a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.890+0000 7f19027fc700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904100e60 0x7f19041976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f1902ffd700 1 -- 172.21.15.31:0/4109865567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904100e60 msgr2=0x7f19041976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f1902ffd700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904100e60 0x7f19041976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f1902ffd700 1 -- 172.21.15.31:0/4109865567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1904109b00 msgr2=0x7f19041a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f1902ffd700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1904109b00 0x7f19041a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f1902ffd700 1 -- 172.21.15.31:0/4109865567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40095f0 con 0x7f1904100550 2023-12-14T18:39:02.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f19027fc700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904100e60 0x7f19041976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f1902ffd700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f19041971b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000c00 tx=0x7f18f4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f401d070 con 0x7f1904100550 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18f4021410 con 0x7f1904100550 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190410f230 con 0x7f1904100550 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190410f0a0 con 0x7f1904100550 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.891+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40175f0 con 0x7f1904100550 2023-12-14T18:39:02.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.892+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190410f3c0 con 0x7f1904100550 2023-12-14T18:39:02.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.893+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f18f4021580 con 0x7f1904100550 2023-12-14T18:39:02.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.894+0000 7f19088b1700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18ec0a12c0 0x7f18ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:02.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.894+0000 7f19027fc700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18ec0a12c0 0x7f18ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:02.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.894+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7f18f40ce5e0 con 0x7f1904100550 2023-12-14T18:39:02.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.895+0000 7f19027fc700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18ec0a12c0 0x7f18ec0a3770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19040ffcf0 tx=0x7f18f8009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:02.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:02.897+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f18f40976c0 con 0x7f1904100550 2023-12-14T18:39:02.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:02 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/2085992004' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:39:02.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:02 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/2085992004' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:39:03.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:02 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/2085992004' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:39:03.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.153+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1904067e40 con 0x7f1904100550 2023-12-14T18:39:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.154+0000 7f19088b1700 1 -- 172.21.15.31:0/4109865567 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10813 (secure 0 0 0) 0x7f18f409ace0 con 0x7f1904100550 2023-12-14T18:39:03.155 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:39:03.155 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/28036797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":28036797},{"type":"v1","addr":"172.21.15.31:6835","nonce":28036797}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14589,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/2398776190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6838","nonce":2398776190},{"type":"v1","addr":"172.21.15.31:6839","nonce":2398776190}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6841/3456971189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6840","nonce":3456971189},{"type":"v1","addr":"172.21.15.31:6841","nonce":3456971189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6833/2002803706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6832","nonce":2002803706},{"type":"v1","addr":"172.21.15.176:6833","nonce":2002803706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/1411039043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1411039043},{"type":"v1","addr":"172.21.15.204:6833","nonce":1411039043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24476,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6835/1697669779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6834","nonce":1697669779},{"type":"v1","addr":"172.21.15.176:6835","nonce":1697669779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:25:55.286836+0000","modified":"2023-12-14T18:36:29.509045+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24538,"mds_1":24536,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.31:6837/3783760943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":3783760943},{"type":"v1","addr":"172.21.15.31:6837","nonce":3783760943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6837/2569700606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6836","nonce":2569700606},{"type":"v1","addr":"172.21.15.176:6837","nonce":2569700606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/3595337412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":3595337412},{"type":"v1","addr":"172.21.15.204:6835","nonce":3595337412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.204:6837/1057123586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1057123586},{"type":"v1","addr":"172.21.15.204:6837","nonce":1057123586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24536":{"gid":24536,"name":"b","rank":1,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.176:6839/2954758257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6838","nonce":2954758257},{"type":"v1","addr":"172.21.15.176:6839","nonce":2954758257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.204:6839/1902825357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1902825357},{"type":"v1","addr":"172.21.15.204:6839","nonce":1902825357}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:39:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18ec0a12c0 msgr2=0x7f18ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18ec0a12c0 0x7f18ec0a3770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19040ffcf0 tx=0x7f18f8009260 comp rx=0 tx=0).stop 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 msgr2=0x7f19041971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f19041971b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f18f4000c00 tx=0x7f18f4004790 comp rx=0 tx=0).stop 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 shutdown_connections 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1904109b00 0x7f19041a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1904100550 0x7f19041971b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f18ec0a12c0 0x7f18ec0a3770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 --2- 172.21.15.31:0/4109865567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1904100e60 0x7f19041976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 >> 172.21.15.31:0/4109865567 conn(0x7f19040749f0 msgr2=0x7f19040fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 shutdown_connections 2023-12-14T18:39:03.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.156+0000 7f19098b3700 1 -- 172.21.15.31:0/4109865567 wait complete. 2023-12-14T18:39:03.159 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2023-12-14T18:39:03.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:39:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 -- 172.21.15.31:0/236917909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ff3f0 msgr2=0x7f03740ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 --2- 172.21.15.31:0/236917909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ff3f0 0x7f03740ff7c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f036400b0d0 tx=0x7f036400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 -- 172.21.15.31:0/236917909 shutdown_connections 2023-12-14T18:39:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 --2- 172.21.15.31:0/236917909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0374104430 0x7f037410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 --2- 172.21.15.31:0/236917909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ff3f0 0x7f03740ff7c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 --2- 172.21.15.31:0/236917909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03740ffd90 0x7f0374103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 -- 172.21.15.31:0/236917909 >> 172.21.15.31:0/236917909 conn(0x7f0374075fd0 msgr2=0x7f03740763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.308+0000 7f0378ebb700 1 -- 172.21.15.31:0/236917909 shutdown_connections 2023-12-14T18:39:03.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.309+0000 7f0378ebb700 1 -- 172.21.15.31:0/236917909 wait complete. 2023-12-14T18:39:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.309+0000 7f0378ebb700 1 Processor -- start 2023-12-14T18:39:03.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.309+0000 7f0378ebb700 1 -- start start 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03740ff3f0 0x7f0374100df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 0x7f0374101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0374104430 0x7f03741a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0374101b60 con 0x7f03740ffd90 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03741a4bd0 con 0x7f03740ff3f0 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03741a4d10 con 0x7f0374104430 2023-12-14T18:39:03.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 0x7f0374101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 0x7f0374101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37856/0 (socket says 172.21.15.31:37856) 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 -- 172.21.15.31:0/3119237012 learned_addr learned my addr 172.21.15.31:0/3119237012 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f037259c700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03740ff3f0 0x7f0374100df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0372d9d700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0374104430 0x7f03741a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 -- 172.21.15.31:0/3119237012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0374104430 msgr2=0x7f03741a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0374104430 0x7f03741a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 -- 172.21.15.31:0/3119237012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03740ff3f0 msgr2=0x7f0374100df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03740ff3f0 0x7f0374100df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 -- 172.21.15.31:0/3119237012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364009d00 con 0x7f03740ffd90 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0372d9d700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0374104430 0x7f03741a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0371d9b700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 0x7f0374101330 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f036800c9b0 tx=0x7f036800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368007aa0 con 0x7f03740ffd90 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037410f1f0 con 0x7f03740ffd90 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0368007c00 con 0x7f03740ffd90 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368018660 con 0x7f03740ffd90 2023-12-14T18:39:03.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.310+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037410f060 con 0x7f03740ffd90 2023-12-14T18:39:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.311+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037410f380 con 0x7f03740ffd90 2023-12-14T18:39:03.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.314+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f036800f450 con 0x7f03740ffd90 2023-12-14T18:39:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.314+0000 7f03637fe700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f035c0a12c0 0x7f035c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.314+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7f03680cd4f0 con 0x7f03740ffd90 2023-12-14T18:39:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.314+0000 7f037259c700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f035c0a12c0 0x7f035c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.315+0000 7f037259c700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f035c0a12c0 0x7f035c0a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0364009fd0 tx=0x7f036400b040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:03.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.318+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0368096540 con 0x7f03740ffd90 2023-12-14T18:39:03.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.534+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0374067160 con 0x7f03740ffd90 2023-12-14T18:39:03.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.535+0000 7f03637fe700 1 -- 172.21.15.31:0/3119237012 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+25252 (secure 0 0 0) 0x7f0368099b60 con 0x7f03740ffd90 2023-12-14T18:39:03.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-12-14T18:39:03.537 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":101,"fsid":"37e25396-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:19:54.751625+0000","modified":"2023-12-14T18:38:48.564122+0000","last_up_change":"2023-12-14T18:25:20.611899+0000","last_in_change":"2023-12-14T18:25:04.199055+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:22:37.958695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:25:53.279345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:25:54.669485+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:25:57.764892+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"60ee717b-c58b-4f49-b38f-1ba95d414f5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6803","nonce":4133067944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6805","nonce":4133067944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6809","nonce":4133067944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4133067944},{"type":"v1","addr":"172.21.15.31:6807","nonce":4133067944}]},"public_addr":"172.21.15.31:6803/4133067944","cluster_addr":"172.21.15.31:6805/4133067944","heartbeat_back_addr":"172.21.15.31:6809/4133067944","heartbeat_front_addr":"172.21.15.31:6807/4133067944","state":["exists","up"]},{"osd":1,"uuid":"79738a41-e749-4359-85db-3cc4494bf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6801","nonce":3594574093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6803","nonce":3594574093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6807","nonce":3594574093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":3594574093},{"type":"v1","addr":"172.21.15.176:6805","nonce":3594574093}]},"public_addr":"172.21.15.176:6801/3594574093","cluster_addr":"172.21.15.176:6803/3594574093","heartbeat_back_addr":"172.21.15.176:6807/3594574093","heartbeat_front_addr":"172.21.15.176:6805/3594574093","state":["exists","up"]},{"osd":2,"uuid":"5119d0ba-0044-4206-b4e1-898df98d5f0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6801","nonce":4107722229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6803","nonce":4107722229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6807","nonce":4107722229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":4107722229},{"type":"v1","addr":"172.21.15.204:6805","nonce":4107722229}]},"public_addr":"172.21.15.204:6801/4107722229","cluster_addr":"172.21.15.204:6803/4107722229","heartbeat_back_addr":"172.21.15.204:6807/4107722229","heartbeat_front_addr":"172.21.15.204:6805/4107722229","state":["exists","up"]},{"osd":3,"uuid":"bfb73492-97c9-4615-9eed-75737af46d2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6811","nonce":3446506620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6813","nonce":3446506620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6817","nonce":3446506620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3446506620},{"type":"v1","addr":"172.21.15.31:6815","nonce":3446506620}]},"public_addr":"172.21.15.31:6811/3446506620","cluster_addr":"172.21.15.31:6813/3446506620","heartbeat_back_addr":"172.21.15.31:6817/3446506620","heartbeat_front_addr":"172.21.15.31:6815/3446506620","state":["exists","up"]},{"osd":4,"uuid":"a1c7deda-98af-495d-9680-d1b95ceb100b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6809","nonce":2531574099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6811","nonce":2531574099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6815","nonce":2531574099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2531574099},{"type":"v1","addr":"172.21.15.176:6813","nonce":2531574099}]},"public_addr":"172.21.15.176:6809/2531574099","cluster_addr":"172.21.15.176:6811/2531574099","heartbeat_back_addr":"172.21.15.176:6815/2531574099","heartbeat_front_addr":"172.21.15.176:6813/2531574099","state":["exists","up"]},{"osd":5,"uuid":"e58c9b5f-bcfc-48af-a8bf-e1744c662b84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6809","nonce":1402310939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6811","nonce":1402310939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6815","nonce":1402310939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1402310939},{"type":"v1","addr":"172.21.15.204:6813","nonce":1402310939}]},"public_addr":"172.21.15.204:6809/1402310939","cluster_addr":"172.21.15.204:6811/1402310939","heartbeat_back_addr":"172.21.15.204:6815/1402310939","heartbeat_front_addr":"172.21.15.204:6813/1402310939","state":["exists","up"]},{"osd":6,"uuid":"78200a1a-a3f7-48fb-aeb7-c335cc8b9e86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6819","nonce":2676550981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6821","nonce":2676550981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6825","nonce":2676550981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2676550981},{"type":"v1","addr":"172.21.15.31:6823","nonce":2676550981}]},"public_addr":"172.21.15.31:6819/2676550981","cluster_addr":"172.21.15.31:6821/2676550981","heartbeat_back_addr":"172.21.15.31:6825/2676550981","heartbeat_front_addr":"172.21.15.31:6823/2676550981","state":["exists","up"]},{"osd":7,"uuid":"2d0a6115-2273-425b-84a6-086924f6da23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6817","nonce":800429312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6819","nonce":800429312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6823","nonce":800429312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":800429312},{"type":"v1","addr":"172.21.15.176:6821","nonce":800429312}]},"public_addr":"172.21.15.176:6817/800429312","cluster_addr":"172.21.15.176:6819/800429312","heartbeat_back_addr":"172.21.15.176:6823/800429312","heartbeat_front_addr":"172.21.15.176:6821/800429312","state":["exists","up"]},{"osd":8,"uuid":"bb114398-c403-4e55-83ea-cdab039ce2e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6817","nonce":2332045081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6819","nonce":2332045081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6823","nonce":2332045081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":2332045081},{"type":"v1","addr":"172.21.15.204:6821","nonce":2332045081}]},"public_addr":"172.21.15.204:6817/2332045081","cluster_addr":"172.21.15.204:6819/2332045081","heartbeat_back_addr":"172.21.15.204:6823/2332045081","heartbeat_front_addr":"172.21.15.204:6821/2332045081","state":["exists","up"]},{"osd":9,"uuid":"4d9d2a00-4cc2-40e3-8587-405fa776464d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6827","nonce":1064380347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6829","nonce":1064380347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6833","nonce":1064380347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":1064380347},{"type":"v1","addr":"172.21.15.31:6831","nonce":1064380347}]},"public_addr":"172.21.15.31:6827/1064380347","cluster_addr":"172.21.15.31:6829/1064380347","heartbeat_back_addr":"172.21.15.31:6833/1064380347","heartbeat_front_addr":"172.21.15.31:6831/1064380347","state":["exists","up"]},{"osd":10,"uuid":"caf93ab4-d4cc-4d56-a291-abbc32df979b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6825","nonce":3741647642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6827","nonce":3741647642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6830","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6831","nonce":3741647642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":3741647642},{"type":"v1","addr":"172.21.15.176:6829","nonce":3741647642}]},"public_addr":"172.21.15.176:6825/3741647642","cluster_addr":"172.21.15.176:6827/3741647642","heartbeat_back_addr":"172.21.15.176:6831/3741647642","heartbeat_front_addr":"172.21.15.176:6829/3741647642","state":["exists","up"]},{"osd":11,"uuid":"a3cf56d3-54b6-43d9-91cd-1b869bcf5146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6825","nonce":2063130933}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6827","nonce":2063130933}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6831","nonce":2063130933}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2063130933},{"type":"v1","addr":"172.21.15.204:6829","nonce":2063130933}]},"public_addr":"172.21.15.204:6825/2063130933","cluster_addr":"172.21.15.204:6827/2063130933","heartbeat_back_addr":"172.21.15.204:6831/2063130933","heartbeat_front_addr":"172.21.15.204:6829/2063130933","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:00.279650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:17.801198+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:35.262613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:54.171941+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:12.053466+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:29.695759+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:48.647994+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:06.582749+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:24.307214+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:43.292396+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:01.285842+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:25:18.801572+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":10}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/2924181707":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2117792145":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/463659266":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6800/2688246057":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2892448986":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:6801/2688246057":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/2160431679":"2023-12-15T18:27:52.189495+0000","172.21.15.31:0/3108903497":"2023-12-15T18:27:52.189495+0000","172.21.15.31:6800/2094677585":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/384230437":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/3199333831":"2023-12-15T18:27:52.189495+0000","172.21.15.31:6800/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:6801/2330625991":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/132110935":"2023-12-15T18:20:25.264468+0000","172.21.15.31:0/1680573410":"2023-12-15T18:20:47.403710+0000","172.21.15.31:0/3645411014":"2023-12-15T18:27:52.189495+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:39:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f035c0a12c0 msgr2=0x7f035c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f035c0a12c0 0x7f035c0a3770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0364009fd0 tx=0x7f036400b040 comp rx=0 tx=0).stop 2023-12-14T18:39:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 msgr2=0x7f0374101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 0x7f0374101330 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f036800c9b0 tx=0x7f036800cd70 comp rx=0 tx=0).stop 2023-12-14T18:39:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 shutdown_connections 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f035c0a12c0 0x7f035c0a3770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0374104430 0x7f03741a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f03740ff3f0 0x7f0374100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 --2- 172.21.15.31:0/3119237012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03740ffd90 0x7f0374101330 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 >> 172.21.15.31:0/3119237012 conn(0x7f0374075fd0 msgr2=0x7f0374103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 shutdown_connections 2023-12-14T18:39:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.538+0000 7f0378ebb700 1 -- 172.21.15.31:0/3119237012 wait complete. 2023-12-14T18:39:03.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-14T18:39:03.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 -- 172.21.15.31:0/535473714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 msgr2=0x7fcc8010cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 --2- 172.21.15.31:0/535473714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc8010cc60 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800b0a0 tx=0x7fcc7800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 -- 172.21.15.31:0/535473714 shutdown_connections 2023-12-14T18:39:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 --2- 172.21.15.31:0/535473714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc8010cc60 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 --2- 172.21.15.31:0/535473714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc80101340 0x7fcc80101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 --2- 172.21.15.31:0/535473714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80101c50 0x7fcc8010a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.693+0000 7fcc85b70700 1 -- 172.21.15.31:0/535473714 >> 172.21.15.31:0/535473714 conn(0x7fcc80075fc0 msgr2=0x7fcc800763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:03.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.694+0000 7fcc85b70700 1 -- 172.21.15.31:0/535473714 shutdown_connections 2023-12-14T18:39:03.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.694+0000 7fcc85b70700 1 -- 172.21.15.31:0/535473714 wait complete. 2023-12-14T18:39:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.694+0000 7fcc85b70700 1 Processor -- start 2023-12-14T18:39:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.694+0000 7fcc85b70700 1 -- start start 2023-12-14T18:39:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc85b70700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc80101340 0x7fcc800ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc85b70700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80101c50 0x7fcc800ff6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc85b70700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc801a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc85b70700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc800ffee0 con 0x7fcc8010a860 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc85b70700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801a4b80 con 0x7fcc80101340 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc85b70700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc801a4cc0 con 0x7fcc80101c50 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc801a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc801a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37874/0 (socket says 172.21.15.31:37874) 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 -- 172.21.15.31:0/3040653167 learned_addr learned my addr 172.21.15.31:0/3040653167 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7f7fe700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc80101340 0x7fcc800ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7effd700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80101c50 0x7fcc800ff6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 -- 172.21.15.31:0/3040653167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80101c50 msgr2=0x7fcc800ff6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80101c50 0x7fcc800ff6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 -- 172.21.15.31:0/3040653167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc80101340 msgr2=0x7fcc800ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc80101340 0x7fcc800ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 -- 172.21.15.31:0/3040653167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78009d00 con 0x7fcc8010a860 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.695+0000 7fcc7ffff700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc801a4640 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcc78007990 tx=0x7fcc780079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.696+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800e040 con 0x7fcc8010a860 2023-12-14T18:39:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.696+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc80110290 con 0x7fcc8010a860 2023-12-14T18:39:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.696+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc78004c50 con 0x7fcc8010a860 2023-12-14T18:39:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.696+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc780052a0 con 0x7fcc8010a860 2023-12-14T18:39:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.696+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc80110100 con 0x7fcc8010a860 2023-12-14T18:39:03.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.697+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc80110420 con 0x7fcc8010a860 2023-12-14T18:39:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.698+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcc7801d470 con 0x7fcc8010a860 2023-12-14T18:39:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.699+0000 7fcc7cff9700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc680a12c0 0x7fcc680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.699+0000 7fcc7f7fe700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc680a12c0 0x7fcc680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.699+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9481+0+0 (secure 0 0 0) 0x7fcc780ceed0 con 0x7fcc8010a860 2023-12-14T18:39:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.699+0000 7fcc7f7fe700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc680a12c0 0x7fcc680a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009c60 tx=0x7fcc70009260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.703+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcc780980d0 con 0x7fcc8010a860 2023-12-14T18:39:03.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:03 smithi031 ceph-mon[42085]: pgmap v351: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:39:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:03 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/7382614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:39:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:03 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4109865567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:39:03.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:03 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3119237012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:39:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:03.964+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcc801a4e00 con 0x7fcc8010a860 2023-12-14T18:39:03.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:03 smithi176 ceph-mon[43506]: pgmap v351: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:39:03.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:03 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/7382614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:39:03.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:03 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4109865567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:39:03.996 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:03 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3119237012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:39:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:03 smithi204 ceph-mon[43866]: pgmap v351: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:39:04.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:03 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/7382614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:39:04.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:03 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4109865567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:39:04.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:03 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3119237012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:39:04.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.638+0000 7fcc7cff9700 1 -- 172.21.15.31:0/3040653167 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcc801a4e00 con 0x7fcc8010a860 2023-12-14T18:39:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.648+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc680a12c0 msgr2=0x7fcc680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.648+0000 7fcc85b70700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc680a12c0 0x7fcc680a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009c60 tx=0x7fcc70009260 comp rx=0 tx=0).stop 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.648+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 msgr2=0x7fcc801a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.648+0000 7fcc85b70700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc801a4640 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcc78007990 tx=0x7fcc780079c0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.649+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 shutdown_connections 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.649+0000 7fcc85b70700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc8010a860 0x7fcc801a4640 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.649+0000 7fcc85b70700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc80101340 0x7fcc800ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.649+0000 7fcc85b70700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fcc680a12c0 0x7fcc680a3770 secure :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009c60 tx=0x7fcc70009260 comp rx=0 tx=0).stop 2023-12-14T18:39:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.649+0000 7fcc85b70700 1 --2- 172.21.15.31:0/3040653167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc80101c50 0x7fcc800ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.649+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 >> 172.21.15.31:0/3040653167 conn(0x7fcc80075fc0 msgr2=0x7fcc800fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.655+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 shutdown_connections 2023-12-14T18:39:04.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.657+0000 7fcc85b70700 1 -- 172.21.15.31:0/3040653167 wait complete. 2023-12-14T18:39:04.672 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-14T18:39:04.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-14T18:39:04.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 -- 172.21.15.31:0/520057242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334119a40 msgr2=0x7fb33411be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:04.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 --2- 172.21.15.31:0/520057242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334119a40 0x7fb33411be20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb32401a9a0 tx=0x7fb32401acb0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 -- 172.21.15.31:0/520057242 shutdown_connections 2023-12-14T18:39:04.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 --2- 172.21.15.31:0/520057242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb334119a40 0x7fb33411be20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 --2- 172.21.15.31:0/520057242 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb334107af0 0x7fb33410eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 --2- 172.21.15.31:0/520057242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb334107ec0 0x7fb334108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.952+0000 7fb339336700 1 -- 172.21.15.31:0/520057242 >> 172.21.15.31:0/520057242 conn(0x7fb33406b7e0 msgr2=0x7fb334071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:04.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.958+0000 7fb339336700 1 -- 172.21.15.31:0/520057242 shutdown_connections 2023-12-14T18:39:04.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.961+0000 7fb339336700 1 -- 172.21.15.31:0/520057242 wait complete. 2023-12-14T18:39:04.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.961+0000 7fb339336700 1 Processor -- start 2023-12-14T18:39:04.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a[42062]: 2023-12-14T18:39:04.603+0000 7f706e4fd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:39:04.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:04 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3040653167' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-14T18:39:04.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:04 smithi031 ceph-mon[42085]: osdmap e102: 12 total, 12 up, 12 in 2023-12-14T18:39:04.964 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:39:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d[75661]: ignoring --setuser ceph since I am not root 2023-12-14T18:39:04.964 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:39:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d[75661]: ignoring --setgroup ceph since I am not root 2023-12-14T18:39:04.964 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:39:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d[75661]: starting mds.d at 2023-12-14T18:39:04.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 -- start start 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb334107af0 0x7fb334115140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb334119a40 0x7fb334115680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 0x7fb3341c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341c4100 con 0x7fb3341c3620 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341c4270 con 0x7fb334119a40 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.965+0000 7fb339336700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341c4430 con 0x7fb334107af0 2023-12-14T18:39:04.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.966+0000 7fb3337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 0x7fb3341c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.966+0000 7fb3337fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 0x7fb3341c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45196/0 (socket says 172.21.15.31:45196) 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.966+0000 7fb3337fe700 1 -- 172.21.15.31:0/264663962 learned_addr learned my addr 172.21.15.31:0/264663962 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.966+0000 7fb332ffd700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb334107af0 0x7fb334115140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.967+0000 7fb3337fe700 1 -- 172.21.15.31:0/264663962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb334107af0 msgr2=0x7fb334115140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.967+0000 7fb3337fe700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb334107af0 0x7fb334115140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.967+0000 7fb3337fe700 1 -- 172.21.15.31:0/264663962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb334119a40 msgr2=0x7fb334115680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.967+0000 7fb3337fe700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb334119a40 0x7fb334115680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:04.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.967+0000 7fb3337fe700 1 -- 172.21.15.31:0/264663962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32401a600 con 0x7fb3341c3620 2023-12-14T18:39:04.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.967+0000 7fb3337fe700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 0x7fb3341c3a90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb32401cc90 tx=0x7fb32401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:04.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.968+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324004080 con 0x7fb3341c3620 2023-12-14T18:39:04.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.968+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3240041e0 con 0x7fb3341c3620 2023-12-14T18:39:04.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.968+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb334111ad0 con 0x7fb3341c3620 2023-12-14T18:39:04.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.968+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32401f650 con 0x7fb3341c3620 2023-12-14T18:39:04.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.968+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb334111940 con 0x7fb3341c3620 2023-12-14T18:39:04.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.971+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb334111c60 con 0x7fb3341c3620 2023-12-14T18:39:04.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.973+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb32400e430 con 0x7fb3341c3620 2023-12-14T18:39:04.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.974+0000 7fb31bfff700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb31c0a14d0 0x7fb31c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:04.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.974+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9871+0+0 (secure 0 0 0) 0x7fb3240ce660 con 0x7fb3341c3620 2023-12-14T18:39:04.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.975+0000 7fb332ffd700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb31c0a14d0 0x7fb31c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:04.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.980+0000 7fb332ffd700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb31c0a14d0 0x7fb31c0a3980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb328006fd0 tx=0x7fb328009190 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:04.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:04.983+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb3240975f0 con 0x7fb3341c3620 2023-12-14T18:39:04.993 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h[68329]: ignoring --setuser ceph since I am not root 2023-12-14T18:39:04.993 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h[68329]: ignoring --setgroup ceph since I am not root 2023-12-14T18:39:04.993 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h[68329]: starting mds.h at 2023-12-14T18:39:04.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3040653167' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-14T18:39:04.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-mon[43506]: osdmap e102: 12 total, 12 up, 12 in 2023-12-14T18:39:04.995 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b[68675]: ignoring --setuser ceph since I am not root 2023-12-14T18:39:04.996 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b[68675]: ignoring --setgroup ceph since I am not root 2023-12-14T18:39:04.996 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-mds[68698]: -- 172.21.15.176:0/2870755462 <== mon.2 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598158a5520 con 0x55981587ac00 2023-12-14T18:39:04.996 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:39:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b[68675]: starting mds.b at 2023-12-14T18:39:05.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3040653167' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-14T18:39:05.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-mon[43866]: osdmap e102: 12 total, 12 up, 12 in 2023-12-14T18:39:05.091 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f[69178]: ignoring --setuser ceph since I am not root 2023-12-14T18:39:05.091 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f[69178]: ignoring --setgroup ceph since I am not root 2023-12-14T18:39:05.091 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f[69178]: starting mds.f at 2023-12-14T18:39:05.092 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: ignoring --setuser ceph since I am not root 2023-12-14T18:39:05.092 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: ignoring --setgroup ceph since I am not root 2023-12-14T18:39:05.092 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: starting mds.i at 2023-12-14T18:39:05.092 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c[69520]: ignoring --setuser ceph since I am not root 2023-12-14T18:39:05.092 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c[69520]: ignoring --setgroup ceph since I am not root 2023-12-14T18:39:05.092 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:39:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c[69520]: starting mds.c at 2023-12-14T18:39:05.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.441+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb3341cbe60 con 0x7fb3341c3620 2023-12-14T18:39:05.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.650+0000 7fb31bfff700 1 -- 172.21.15.31:0/264663962 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb32409ac10 con 0x7fb3341c3620 2023-12-14T18:39:05.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.666+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb31c0a14d0 msgr2=0x7fb31c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:05.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.666+0000 7fb339336700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb31c0a14d0 0x7fb31c0a3980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb328006fd0 tx=0x7fb328009190 comp rx=0 tx=0).stop 2023-12-14T18:39:05.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.666+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 msgr2=0x7fb3341c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:05.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.666+0000 7fb339336700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 0x7fb3341c3a90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb32401cc90 tx=0x7fb32401ccc0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.671+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 shutdown_connections 2023-12-14T18:39:05.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.671+0000 7fb339336700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3341c3620 0x7fb3341c3a90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.671+0000 7fb339336700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7fb31c0a14d0 0x7fb31c0a3980 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.671+0000 7fb339336700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb334107af0 0x7fb334115140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.671+0000 7fb339336700 1 --2- 172.21.15.31:0/264663962 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb334119a40 0x7fb334115680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.671+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 >> 172.21.15.31:0/264663962 conn(0x7fb33406b7e0 msgr2=0x7fb33411bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.673+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 shutdown_connections 2023-12-14T18:39:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.673+0000 7fb339336700 1 -- 172.21.15.31:0/264663962 wait complete. 2023-12-14T18:39:05.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-14T18:39:05.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:05 smithi031 ceph-mon[42085]: pgmap v353: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-12-14T18:39:05.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:05 smithi031 ceph-mon[42085]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-14T18:39:05.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:05 smithi031 ceph-mon[42085]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:39:05.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:05 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3040653167' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-14T18:39:05.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:05 smithi031 ceph-mon[42085]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-14T18:39:05.968 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:05 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/264663962' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:39:05.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.980+0000 7faec897e700 1 -- 172.21.15.31:0/1192306410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 msgr2=0x7faec4107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:05.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.980+0000 7faec897e700 1 --2- 172.21.15.31:0/1192306410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 0x7faec4107f60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009990 tx=0x7faeb8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.981+0000 7faec897e700 1 -- 172.21.15.31:0/1192306410 shutdown_connections 2023-12-14T18:39:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.981+0000 7faec897e700 1 --2- 172.21.15.31:0/1192306410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 0x7faec4079740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.981+0000 7faec897e700 1 --2- 172.21.15.31:0/1192306410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 0x7faec410c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.981+0000 7faec897e700 1 --2- 172.21.15.31:0/1192306410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 0x7faec4107f60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.981+0000 7faec897e700 1 -- 172.21.15.31:0/1192306410 >> 172.21.15.31:0/1192306410 conn(0x7faec406b7e0 msgr2=0x7faec4071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:05.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.986+0000 7faec897e700 1 -- 172.21.15.31:0/1192306410 shutdown_connections 2023-12-14T18:39:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:05 smithi176 ceph-mon[43506]: pgmap v353: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-12-14T18:39:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:05 smithi176 ceph-mon[43506]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-14T18:39:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:05 smithi176 ceph-mon[43506]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:39:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:05 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3040653167' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-14T18:39:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:05 smithi176 ceph-mon[43506]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-14T18:39:05.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:05 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/264663962' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:39:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.993+0000 7faec897e700 1 -- 172.21.15.31:0/1192306410 wait complete. 2023-12-14T18:39:05.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 Processor -- start 2023-12-14T18:39:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 -- start start 2023-12-14T18:39:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 0x7faec4073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:05.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 0x7faec4074100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 0x7faec41b9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4074a60 con 0x7faec41084a0 2023-12-14T18:39:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41b9dd0 con 0x7faec410bef0 2023-12-14T18:39:05.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.995+0000 7faec897e700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41b9f90 con 0x7faec4107af0 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.996+0000 7faec2d9d700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 0x7faec41b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.996+0000 7faec1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 0x7faec4074100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.996+0000 7faec1d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 0x7faec4074100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45208/0 (socket says 172.21.15.31:45208) 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.996+0000 7faec1d9b700 1 -- 172.21.15.31:0/3900769660 learned_addr learned my addr 172.21.15.31:0/3900769660 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.997+0000 7faec2d9d700 1 -- 172.21.15.31:0/3900769660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 msgr2=0x7faec4073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.997+0000 7faec2d9d700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 0x7faec4073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.997+0000 7faec2d9d700 1 -- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 msgr2=0x7faec4074100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.997+0000 7faec2d9d700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 0x7faec4074100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:05.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.997+0000 7faec2d9d700 1 -- 172.21.15.31:0/3900769660 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80095f0 con 0x7faec410bef0 2023-12-14T18:39:06.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:05.999+0000 7faec2d9d700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 0x7faec41b9890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faebc007f10 tx=0x7faebc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.000+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc00ddd0 con 0x7faec410bef0 2023-12-14T18:39:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.000+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faebc00f040 con 0x7faec410bef0 2023-12-14T18:39:06.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.000+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc005a90 con 0x7faec410bef0 2023-12-14T18:39:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.000+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec407cbd0 con 0x7faec410bef0 2023-12-14T18:39:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.000+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec407ca40 con 0x7faec410bef0 2023-12-14T18:39:06.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.002+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec407cd60 con 0x7faec410bef0 2023-12-14T18:39:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.009+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faebc015400 con 0x7faec410bef0 2023-12-14T18:39:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.010+0000 7faeb37fe700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faeac0a13f0 0x7faeac0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.010+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9871+0+0 (secure 0 0 0) 0x7faebc0cd2a0 con 0x7faec410bef0 2023-12-14T18:39:06.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.010+0000 7faec259c700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faeac0a13f0 0x7faeac0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.011+0000 7faec259c700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faeac0a13f0 0x7faeac0a38a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faeb4005fd0 tx=0x7faeb4005ee0 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.016+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faebc0970d0 con 0x7faec410bef0 2023-12-14T18:39:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:05 smithi204 ceph-mon[43866]: pgmap v353: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-12-14T18:39:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:05 smithi204 ceph-mon[43866]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-14T18:39:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:05 smithi204 ceph-mon[43866]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:39:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:05 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3040653167' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-14T18:39:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:05 smithi204 ceph-mon[43866]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-14T18:39:06.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:05 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/264663962' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:39:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.369+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faec40679f0 con 0x7faec410bef0 2023-12-14T18:39:06.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.685+0000 7faeb37fe700 1 -- 172.21.15.31:0/3900769660 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) v1 ==== 181+0+0 (secure 0 0 0) 0x7faebc023030 con 0x7faec410bef0 2023-12-14T18:39:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.693+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faeac0a13f0 msgr2=0x7faeac0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.693+0000 7faec897e700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faeac0a13f0 0x7faeac0a38a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7faeb4005fd0 tx=0x7faeb4005ee0 comp rx=0 tx=0).stop 2023-12-14T18:39:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.693+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 msgr2=0x7faec41b9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:06.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.693+0000 7faec897e700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 0x7faec41b9890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faebc007f10 tx=0x7faebc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:39:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.699+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 shutdown_connections 2023-12-14T18:39:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.699+0000 7faec897e700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faec410bef0 0x7faec41b9890 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.699+0000 7faec897e700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faeac0a13f0 0x7faeac0a38a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.699+0000 7faec897e700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faec4107af0 0x7faec4073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.699+0000 7faec897e700 1 --2- 172.21.15.31:0/3900769660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec41084a0 0x7faec4074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:06.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.699+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 >> 172.21.15.31:0/3900769660 conn(0x7faec406b7e0 msgr2=0x7faec407e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.703+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 shutdown_connections 2023-12-14T18:39:06.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:06.703+0000 7faec897e700 1 -- 172.21.15.31:0/3900769660 wait complete. 2023-12-14T18:39:06.732 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-12-14T18:39:06.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-14T18:39:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-14T18:39:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-14T18:39:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: Cluster is now healthy 2023-12-14T18:39:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6836/588473942,v1:172.21.15.176:6837/588473942] up:boot 2023-12-14T18:39:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6834/3194459448,v1:172.21.15.204:6835/3194459448] up:boot 2023-12-14T18:39:06.965 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.176:6838/2586460262,v1:172.21.15.176:6839/2586460262] up:boot 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6836/551995848,v1:172.21.15.204:6837/551995848] up:boot 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.204:6838/3176742602,v1:172.21.15.204:6839/3176742602] up:boot 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: mds.? [v2:172.21.15.31:6836/1116202557,v1:172.21.15.31:6837/1116202557] up:boot 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/264663962' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: fsmap 12 up:standby 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:39:06.966 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:39:06.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:39:06.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/3900769660' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:06.967 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:06 smithi031 ceph-mon[42085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: Cluster is now healthy 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6836/588473942,v1:172.21.15.176:6837/588473942] up:boot 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6834/3194459448,v1:172.21.15.204:6835/3194459448] up:boot 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.176:6838/2586460262,v1:172.21.15.176:6839/2586460262] up:boot 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6836/551995848,v1:172.21.15.204:6837/551995848] up:boot 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.204:6838/3176742602,v1:172.21.15.204:6839/3176742602] up:boot 2023-12-14T18:39:06.994 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: mds.? [v2:172.21.15.31:6836/1116202557,v1:172.21.15.31:6837/1116202557] up:boot 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/264663962' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: fsmap 12 up:standby 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/3900769660' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:06.995 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:06 smithi176 ceph-mon[43506]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 -- 172.21.15.31:0/3910141004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8108400 msgr2=0x7fabb8112e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 --2- 172.21.15.31:0/3910141004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8108400 0x7fabb8112e70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 -- 172.21.15.31:0/3910141004 shutdown_connections 2023-12-14T18:39:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 --2- 172.21.15.31:0/3910141004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 0x7fabb8115790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 --2- 172.21.15.31:0/3910141004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8107af0 0x7fabb8107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 --2- 172.21.15.31:0/3910141004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8108400 0x7fabb8112e70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.025+0000 7fabc03dc700 1 -- 172.21.15.31:0/3910141004 >> 172.21.15.31:0/3910141004 conn(0x7fabb806b7e0 msgr2=0x7fabb8071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.032+0000 7fabc03dc700 1 -- 172.21.15.31:0/3910141004 shutdown_connections 2023-12-14T18:39:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.035+0000 7fabc03dc700 1 -- 172.21.15.31:0/3910141004 wait complete. 2023-12-14T18:39:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.035+0000 7fabc03dc700 1 Processor -- start 2023-12-14T18:39:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 -- start start 2023-12-14T18:39:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8107af0 0x7fabb8112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:07.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8108400 0x7fabb810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 0x7fabb8110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81105d0 con 0x7fabb81133b0 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8110740 con 0x7fabb8108400 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.036+0000 7fabc03dc700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb8110900 con 0x7fabb8107af0 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.039+0000 7fabbe979700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 0x7fabb8110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.039+0000 7fabbe178700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8107af0 0x7fabb8112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.039+0000 7fabbe979700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 0x7fabb8110000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45234/0 (socket says 172.21.15.31:45234) 2023-12-14T18:39:07.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.039+0000 7fabbe979700 1 -- 172.21.15.31:0/4007619922 learned_addr learned my addr 172.21.15.31:0/4007619922 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.041+0000 7fabbd977700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8108400 0x7fabb810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.043+0000 7fabbd977700 1 -- 172.21.15.31:0/4007619922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8107af0 msgr2=0x7fabb8112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.043+0000 7fabbd977700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8107af0 0x7fabb8112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.043+0000 7fabbd977700 1 -- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 msgr2=0x7fabb8110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.043+0000 7fabbd977700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 0x7fabb8110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.043+0000 7fabbd977700 1 -- 172.21.15.31:0/4007619922 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80095f0 con 0x7fabb8108400 2023-12-14T18:39:07.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.044+0000 7fabbd977700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8108400 0x7fabb810fac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba800f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.045+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba801d070 con 0x7fabb8108400 2023-12-14T18:39:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.045+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faba800fcf0 con 0x7fabb8108400 2023-12-14T18:39:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.045+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba8005630 con 0x7fabb8108400 2023-12-14T18:39:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.045+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb810a7c0 con 0x7fabb8108400 2023-12-14T18:39:07.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.045+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb810a630 con 0x7fabb8108400 2023-12-14T18:39:07.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.050+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faba8004830 con 0x7fabb8108400 2023-12-14T18:39:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.051+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c005320 con 0x7fabb8108400 2023-12-14T18:39:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.051+0000 7fabaf7fe700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faba40a1450 0x7faba40a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:07.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.051+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9229+0+0 (secure 0 0 0) 0x7faba80d1170 con 0x7fabb8108400 2023-12-14T18:39:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.058+0000 7fabbe178700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faba40a1450 0x7faba40a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.060+0000 7fabbe178700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faba40a1450 0x7faba40a3900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006fd0 tx=0x7fabb4008040 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.063+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7faba80d2060 con 0x7fabb8108400 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: Cluster is now healthy 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6836/588473942,v1:172.21.15.176:6837/588473942] up:boot 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6834/3194459448,v1:172.21.15.204:6835/3194459448] up:boot 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.176:6838/2586460262,v1:172.21.15.176:6839/2586460262] up:boot 2023-12-14T18:39:07.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6836/551995848,v1:172.21.15.204:6837/551995848] up:boot 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.204:6838/3176742602,v1:172.21.15.204:6839/3176742602] up:boot 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: mds.? [v2:172.21.15.31:6836/1116202557,v1:172.21.15.31:6837/1116202557] up:boot 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/264663962' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: fsmap 12 up:standby 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='mgr.15585 172.21.15.31:0/3486913537' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:39:07.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/3900769660' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:07.091 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:06 smithi204 ceph-mon[43866]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:07.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.403+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fab9c005190 con 0x7fabb8108400 2023-12-14T18:39:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.692+0000 7fabaf7fe700 1 -- 172.21.15.31:0/4007619922 <== mon.2 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) v1 ==== 169+0+0 (secure 0 0 0) 0x7faba8096f40 con 0x7fabb8108400 2023-12-14T18:39:07.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faba40a1450 msgr2=0x7faba40a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:07.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faba40a1450 0x7faba40a3900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fabb4006fd0 tx=0x7fabb4008040 comp rx=0 tx=0).stop 2023-12-14T18:39:07.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8108400 msgr2=0x7fabb810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8108400 0x7fabb810fac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7faba8009990 tx=0x7faba800f880 comp rx=0 tx=0).stop 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 shutdown_connections 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabb81133b0 0x7fabb8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7faba40a1450 0x7faba40a3900 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb8107af0 0x7fabb8112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 --2- 172.21.15.31:0/4007619922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fabb8108400 0x7fabb810fac0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.699+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 >> 172.21.15.31:0/4007619922 conn(0x7fabb806b7e0 msgr2=0x7fabb81172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:07.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.700+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 shutdown_connections 2023-12-14T18:39:07.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:07.704+0000 7fabc03dc700 1 -- 172.21.15.31:0/4007619922 wait complete. 2023-12-14T18:39:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-12-14T18:39:07.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-14T18:39:07.969 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:07 smithi031 ceph-mon[42085]: pgmap v354: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-12-14T18:39:07.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:07 smithi031 ceph-mon[42085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:39:07.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:07 smithi031 ceph-mon[42085]: osdmap e103: 12 total, 12 up, 12 in 2023-12-14T18:39:07.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:07 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/4007619922' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:07.970 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:07 smithi031 ceph-mon[42085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:07 smithi176 ceph-mon[43506]: pgmap v354: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-12-14T18:39:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:07 smithi176 ceph-mon[43506]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:39:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:07 smithi176 ceph-mon[43506]: osdmap e103: 12 total, 12 up, 12 in 2023-12-14T18:39:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:07 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/4007619922' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:07.992 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:07 smithi176 ceph-mon[43506]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.080+0000 7f8b935d4700 1 -- 172.21.15.31:0/2310920730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c108400 msgr2=0x7f8b8c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:08.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.080+0000 7f8b935d4700 1 --2- 172.21.15.31:0/2310920730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c108400 0x7f8b8c108870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0098f0 tx=0x7f8b7c009c00 comp rx=0 tx=0).stop 2023-12-14T18:39:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.085+0000 7f8b935d4700 1 -- 172.21.15.31:0/2310920730 shutdown_connections 2023-12-14T18:39:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.085+0000 7f8b935d4700 1 --2- 172.21.15.31:0/2310920730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c108e90 0x7f8b8c1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.085+0000 7f8b935d4700 1 --2- 172.21.15.31:0/2310920730 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b8c107af0 0x7f8b8c107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.085+0000 7f8b935d4700 1 --2- 172.21.15.31:0/2310920730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c108400 0x7f8b8c108870 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.085+0000 7f8b935d4700 1 -- 172.21.15.31:0/2310920730 >> 172.21.15.31:0/2310920730 conn(0x7f8b8c06b7e0 msgr2=0x7f8b8c071650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:08.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.086+0000 7f8b935d4700 1 -- 172.21.15.31:0/2310920730 shutdown_connections 2023-12-14T18:39:08.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:07 smithi204 ceph-mon[43866]: pgmap v354: 129 pgs: 129 active+clean; 544 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 25 op/s 2023-12-14T18:39:08.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:07 smithi204 ceph-mon[43866]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:39:08.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:07 smithi204 ceph-mon[43866]: osdmap e103: 12 total, 12 up, 12 in 2023-12-14T18:39:08.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:07 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/4007619922' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.090 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:07 smithi204 ceph-mon[43866]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.088+0000 7f8b935d4700 1 -- 172.21.15.31:0/2310920730 wait complete. 2023-12-14T18:39:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.089+0000 7f8b935d4700 1 Processor -- start 2023-12-14T18:39:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 -- start start 2023-12-14T18:39:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 0x7f8b8c110ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b8c108400 0x7f8b8c111200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c108e90 0x7f8b8c10bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c10c530 con 0x7f8b8c108e90 2023-12-14T18:39:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c10c6a0 con 0x7f8b8c108400 2023-12-14T18:39:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.096+0000 7f8b935d4700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c10c860 con 0x7f8b8c107af0 2023-12-14T18:39:08.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.098+0000 7f8b91370700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 0x7f8b8c110ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:08.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 0x7f8b8c110ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.31:35844/0 (socket says 172.21.15.31:35844) 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 -- 172.21.15.31:0/960296370 learned_addr learned my addr 172.21.15.31:0/960296370 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 -- 172.21.15.31:0/960296370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b8c108400 msgr2=0x7f8b8c111200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b8c108400 0x7f8b8c111200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 -- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c108e90 msgr2=0x7f8b8c10bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c108e90 0x7f8b8c10bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 -- 172.21.15.31:0/960296370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c009520 con 0x7f8b8c107af0 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.099+0000 7f8b91370700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 0x7f8b8c110ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800b550 tx=0x7f8b8800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.100+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88011830 con 0x7f8b8c107af0 2023-12-14T18:39:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.100+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b88011e50 con 0x7f8b8c107af0 2023-12-14T18:39:08.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.100+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800f2e0 con 0x7f8b8c107af0 2023-12-14T18:39:08.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.101+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c118b60 con 0x7f8b8c107af0 2023-12-14T18:39:08.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.101+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c1189d0 con 0x7f8b8c107af0 2023-12-14T18:39:08.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.109+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b70005320 con 0x7f8b8c107af0 2023-12-14T18:39:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.110+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 24) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8b88011990 con 0x7f8b8c107af0 2023-12-14T18:39:08.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.110+0000 7f8b827fc700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8b780a14d0 0x7f8b780a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:39:08.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.110+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8641+0+0 (secure 0 0 0) 0x7f8b880cc6d0 con 0x7f8b8c107af0 2023-12-14T18:39:08.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.112+0000 7f8b90b6f700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8b780a14d0 0x7f8b780a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:39:08.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.113+0000 7f8b90b6f700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8b780a14d0 0x7f8b780a3980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b7c00b260 comp rx=0 tx=0).ready entity=mgr.15585 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:39:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.119+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b88095b10 con 0x7f8b8c107af0 2023-12-14T18:39:08.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.487+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b70005190 con 0x7f8b8c107af0 2023-12-14T18:39:08.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.731+0000 7f8b827fc700 1 -- 172.21.15.31:0/960296370 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v105) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8b88099130 con 0x7f8b8c107af0 2023-12-14T18:39:08.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.742+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8b780a14d0 msgr2=0x7f8b780a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:08.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.742+0000 7f8b935d4700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8b780a14d0 0x7f8b780a3980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b7c00b260 comp rx=0 tx=0).stop 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.742+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 msgr2=0x7f8b8c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.742+0000 7f8b935d4700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 0x7f8b8c110ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800b550 tx=0x7f8b8800b910 comp rx=0 tx=0).stop 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.743+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 shutdown_connections 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.743+0000 7f8b935d4700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c108e90 0x7f8b8c10bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.743+0000 7f8b935d4700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.31:6800/1869071623,v1:172.21.15.31:6801/1869071623] conn(0x7f8b780a14d0 0x7f8b780a3980 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.743+0000 7f8b935d4700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b8c107af0 0x7f8b8c110ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.743+0000 7f8b935d4700 1 --2- 172.21.15.31:0/960296370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8b8c108400 0x7f8b8c111200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:39:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.743+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 >> 172.21.15.31:0/960296370 conn(0x7f8b8c06b7e0 msgr2=0x7f8b8c1a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:39:08.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.746+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 shutdown_connections 2023-12-14T18:39:08.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-12-14T18:39:08.749+0000 7f8b935d4700 1 -- 172.21.15.31:0/960296370 wait complete. 2023-12-14T18:39:08.773 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data_ec' removed 2023-12-14T18:39:08.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:39:08.948 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:39:08.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:08 smithi031 ceph-mon[42085]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:39:08.963 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:08 smithi031 ceph-mon[42085]: osdmap e104: 12 total, 12 up, 12 in 2023-12-14T18:39:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:08 smithi031 ceph-mon[42085]: from='client.? 172.21.15.31:0/960296370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.964 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:08 smithi031 ceph-mon[42085]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.975 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:08 smithi176 ceph-mon[43506]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:39:08.975 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:08 smithi176 ceph-mon[43506]: osdmap e104: 12 total, 12 up, 12 in 2023-12-14T18:39:08.975 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:08 smithi176 ceph-mon[43506]: from='client.? 172.21.15.31:0/960296370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.975 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:08 smithi176 ceph-mon[43506]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:08.979 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:39:09.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:08 smithi204 ceph-mon[43866]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:39:09.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:08 smithi204 ceph-mon[43866]: osdmap e104: 12 total, 12 up, 12 in 2023-12-14T18:39:09.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:08 smithi204 ceph-mon[43866]: from='client.? 172.21.15.31:0/960296370' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:09.016 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:08 smithi204 ceph-mon[43866]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:39:09.026 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-14T18:39:09.027 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:39:09.047 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:39:09.064 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:39:09.099 INFO:tasks.cephadm:Stopping all daemons... 2023-12-14T18:39:09.099 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-14T18:39:09.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a 2023-12-14T18:39:09.464 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:09 smithi031 systemd[1]: Stopping Ceph mon.a for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:09.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a[42062]: 2023-12-14T18:39:09.538+0000 7f7071503700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:09.836 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a[42062]: 2023-12-14T18:39:09.538+0000 7f7071503700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-14T18:39:10.213 INFO:journalctl@ceph.mon.a.smithi031.stdout:Dec 14 18:39:09 smithi031 bash[209238]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-a 2023-12-14T18:39:10.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.a.service' 2023-12-14T18:39:10.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:10.347 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-14T18:39:10.347 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-14T18:39:10.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.b 2023-12-14T18:39:10.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:10 smithi176 systemd[1]: Stopping Ceph mon.b for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:10.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-b[43483]: 2023-12-14T18:39:10.581+0000 7ff060959700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:10.742 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-b[43483]: 2023-12-14T18:39:10.581+0000 7ff060959700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-14T18:39:11.242 INFO:journalctl@ceph.mon.b.smithi176.stdout:Dec 14 18:39:11 smithi176 bash[74371]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-b 2023-12-14T18:39:11.291 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.b.service' 2023-12-14T18:39:11.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:11.392 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-14T18:39:11.392 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-14T18:39:11.392 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.c 2023-12-14T18:39:11.728 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:11 smithi204 systemd[1]: Stopping Ceph mon.c for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:12.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-c[43828]: 2023-12-14T18:39:11.726+0000 7f0c15325700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:12.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-c[43828]: 2023-12-14T18:39:11.726+0000 7f0c15325700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-14T18:39:12.089 INFO:journalctl@ceph.mon.c.smithi204.stdout:Dec 14 18:39:11 smithi204 bash[75659]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mon-c 2023-12-14T18:39:12.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mon.c.service' 2023-12-14T18:39:12.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:12.281 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-14T18:39:12.281 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-14T18:39:12.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x 2023-12-14T18:39:12.582 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:39:12 smithi031 systemd[1]: Stopping Ceph mgr.x for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:12.583 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:39:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:39:12.553+0000 7f8877b33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:12.583 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:39:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x[42335]: 2023-12-14T18:39:12.553+0000 7f8877b33700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-14T18:39:12.964 INFO:journalctl@ceph.mgr.x.smithi031.stdout:Dec 14 18:39:12 smithi031 bash[209377]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-x 2023-12-14T18:39:13.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.x.service' 2023-12-14T18:39:13.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:13.110 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-14T18:39:13.110 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-14T18:39:13.110 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.y 2023-12-14T18:39:13.454 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:39:13 smithi176 systemd[1]: Stopping Ceph mgr.y for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:13.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:39:13 smithi176 bash[74512]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-y 2023-12-14T18:39:13.742 INFO:journalctl@ceph.mgr.y.smithi176.stdout:Dec 14 18:39:13 smithi176 systemd[1]: ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-14T18:39:13.781 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.y.service' 2023-12-14T18:39:13.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:13.818 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-14T18:39:13.818 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-14T18:39:13.818 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.z 2023-12-14T18:39:14.223 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:39:13 smithi204 systemd[1]: Stopping Ceph mgr.z for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:14.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:39:14 smithi204 bash[75799]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mgr-z 2023-12-14T18:39:14.589 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Dec 14 18:39:14 smithi204 systemd[1]: ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-14T18:39:14.599 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mgr.z.service' 2023-12-14T18:39:14.636 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:14.636 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-14T18:39:14.636 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-14T18:39:14.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.0 2023-12-14T18:39:14.964 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:39:14 smithi031 systemd[1]: Stopping Ceph osd.0 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:14.964 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:39:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0[54961]: 2023-12-14T18:39:14.895+0000 7f54bae8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:14.964 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:39:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0[54961]: 2023-12-14T18:39:14.895+0000 7f54bae8f700 -1 osd.0 105 *** Got signal Terminated *** 2023-12-14T18:39:14.964 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:39:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0[54961]: 2023-12-14T18:39:14.895+0000 7f54bae8f700 -1 osd.0 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:39:22.214 INFO:journalctl@ceph.osd.0.smithi031.stdout:Dec 14 18:39:21 smithi031 bash[209519]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-0 2023-12-14T18:39:25.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.0.service' 2023-12-14T18:39:25.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:25.273 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-14T18:39:25.273 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-14T18:39:25.274 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.1 2023-12-14T18:39:25.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:39:25 smithi176 systemd[1]: Stopping Ceph osd.1 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:25.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:39:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1[48490]: 2023-12-14T18:39:25.512+0000 7f4c1fd5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:25.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:39:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1[48490]: 2023-12-14T18:39:25.512+0000 7f4c1fd5f700 -1 osd.1 105 *** Got signal Terminated *** 2023-12-14T18:39:25.742 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:39:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1[48490]: 2023-12-14T18:39:25.512+0000 7f4c1fd5f700 -1 osd.1 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:39:31.492 INFO:journalctl@ceph.osd.1.smithi176.stdout:Dec 14 18:39:31 smithi176 bash[74657]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-1 2023-12-14T18:39:32.326 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.1.service' 2023-12-14T18:39:32.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:32.366 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-14T18:39:32.367 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-14T18:39:32.367 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.2 2023-12-14T18:39:32.839 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:39:32 smithi204 systemd[1]: Stopping Ceph osd.2 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:32.839 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:39:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2[49631]: 2023-12-14T18:39:32.637+0000 7f8de1789700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:32.840 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:39:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2[49631]: 2023-12-14T18:39:32.637+0000 7f8de1789700 -1 osd.2 105 *** Got signal Terminated *** 2023-12-14T18:39:32.840 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:39:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2[49631]: 2023-12-14T18:39:32.637+0000 7f8de1789700 -1 osd.2 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:39:38.589 INFO:journalctl@ceph.osd.2.smithi204.stdout:Dec 14 18:39:38 smithi204 bash[75944]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-2 2023-12-14T18:39:39.471 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.2.service' 2023-12-14T18:39:39.511 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:39.511 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-14T18:39:39.512 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-14T18:39:39.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.3 2023-12-14T18:39:39.964 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:39 smithi031 systemd[1]: Stopping Ceph osd.3 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:39.964 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:39 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:39:39.773+0000 7f08aa5ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:39.964 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:39 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:39:39.773+0000 7f08aa5ac700 -1 osd.3 105 *** Got signal Terminated *** 2023-12-14T18:39:39.964 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:39 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:39:39.773+0000 7f08aa5ac700 -1 osd.3 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:39:42.492 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:42 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:42.227+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:42.713 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:42 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:39:42.301+0000 7f08a6ba2700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:15.278802+0000 front 2023-12-14T18:39:15.278841+0000 (oldest deadline 2023-12-14T18:39:42.279453+0000) 2023-12-14T18:39:43.714 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:39:43.289+0000 7f08a6ba2700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:15.278802+0000 front 2023-12-14T18:39:15.278841+0000 (oldest deadline 2023-12-14T18:39:42.279453+0000) 2023-12-14T18:39:43.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:43 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:43.403+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:43.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:43 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:43.262+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:43.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:43.431+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:44.713 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:44.420+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:44.714 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:44 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3[60455]: 2023-12-14T18:39:44.333+0000 7f08a6ba2700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:15.278802+0000 front 2023-12-14T18:39:15.278841+0000 (oldest deadline 2023-12-14T18:39:42.279453+0000) 2023-12-14T18:39:44.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:44 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:44.281+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:44.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:44 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:44.391+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:45.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:45.034+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:45.714 INFO:journalctl@ceph.osd.3.smithi031.stdout:Dec 14 18:39:45 smithi031 bash[209781]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-3 2023-12-14T18:39:45.715 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:45.418+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:45.715 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:45 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:45.381+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:45.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:45 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:45.277+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:45.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:45.412+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:46.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:46.025+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:46.340 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:45 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:45.866+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:46.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:46.385+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:46.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:46.395+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:46.741 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:46 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:46.326+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:46.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:46.365+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:47.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.3.service' 2023-12-14T18:39:47.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:47.301 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-14T18:39:47.302 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-14T18:39:47.302 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.4 2023-12-14T18:39:47.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:47.070+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:47.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:46 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:46.865+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:47.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:47.427+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:47.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:47 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:47.366+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:47.742 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:39:47 smithi176 systemd[1]: Stopping Ceph osd.4 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:47.744 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:39:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4[53974]: 2023-12-14T18:39:47.542+0000 7f02ea75e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:47.744 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:39:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4[53974]: 2023-12-14T18:39:47.542+0000 7f02ea75e700 -1 osd.4 105 *** Got signal Terminated *** 2023-12-14T18:39:47.744 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:39:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4[53974]: 2023-12-14T18:39:47.542+0000 7f02ea75e700 -1 osd.4 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:39:47.746 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:47 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:47.353+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:47.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:47.408+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:48.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:48 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:48.106+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:48.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:47 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:47.860+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:48.713 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:48 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:48.419+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:48.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:48 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:48.381+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:48.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:48 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:48.310+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:48.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:48 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:48.367+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:49.157 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:48 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:48.904+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:49.591 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:49 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:49.156+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:49.591 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:49 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:49.362+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:49.713 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:49.333+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:49.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:49 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:49.378+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:49.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:49 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:49.324+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:50.318 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:50 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:50.115+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:50.319 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:49 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:49.875+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:50.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:50 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:50.317+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:50.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:50 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:50.364+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:50.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:50 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:50.326+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:50.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:50 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:50.358+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:51.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:50 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:50.922+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:51.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:51 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:51.155+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:51.340 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:51 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:51.155+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:51.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:51 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:51.345+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:51.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:51 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:51.413+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:51.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:51 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:51.405+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:51.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:51 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:51.352+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:52.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:51 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:51.749+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:52.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:52 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:52.176+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:52.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:52 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:52.176+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:52.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:51 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:51.932+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:52.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:52.312+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:52.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:52 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:52.381+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:52.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:52 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:52.375+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:52.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:52 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:52.385+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:53.131 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:52 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:52.765+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:53.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:53 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:53.127+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:53.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:53 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:53.127+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:53.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:52 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:52.936+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:53.340 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:52 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:52.936+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:53.399 INFO:journalctl@ceph.osd.4.smithi176.stdout:Dec 14 18:39:53 smithi176 bash[74912]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-4 2023-12-14T18:39:53.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:53 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:53.408+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:53.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:53 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:53.319+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:53.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:53 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:53.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:53.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:53 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:53.399+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:54.089 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:53 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:53.745+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:54.288 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.4.service' 2023-12-14T18:39:54.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:54.329 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-14T18:39:54.329 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-14T18:39:54.329 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.5 2023-12-14T18:39:54.338 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:54.093+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:54.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:54.093+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:54.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:53 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:53.977+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:54.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:53 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:53.977+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:54.712 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:54 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:54.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:54.712 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:54 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:54.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:39:54.713 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:54 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:54.271+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:54.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:54 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:54.397+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:54.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:54 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:54.397+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:39:54.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:54.413+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:54.839 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:54 smithi204 systemd[1]: Stopping Ceph osd.5 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:54.839 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:54.587+0000 7fc756bd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:54.840 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:54.587+0000 7fc756bd0700 -1 osd.5 105 *** Got signal Terminated *** 2023-12-14T18:39:54.840 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:54.587+0000 7fc756bd0700 -1 osd.5 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:39:54.992 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:54 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:54.711+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:55.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:55 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:55.107+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:55.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:55 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:55.107+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:55.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:54.996+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:55.340 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:54 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:54.996+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:55.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:55 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:55.264+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:55.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:55 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:55.424+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:55.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:55 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:55.424+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:39:55.741 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:55 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:55.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:55.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:55 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:55.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:39:55.838 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:55 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:55.424+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:56.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:55 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:55.745+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:56.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:56.154+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:56.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:56.154+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:56.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:56.023+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:56.340 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:56.023+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:56.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:56 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:56.268+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:56.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:56 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:56.268+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:39:56.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:56 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:56.470+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:56.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:56 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:56.470+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:39:56.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:56 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:56.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:56.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:56 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:56.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:39:56.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:56.469+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:57.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:56 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:56.791+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:57.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:57 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:57.189+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:57.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:57 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:57.189+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:57.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:56.993+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:57.340 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:56 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:56.993+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:57.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:57 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:57.432+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:57.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:57 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:57.432+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:39:57.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:57 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:57.231+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:57.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:57 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:57.231+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:39:57.741 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:57 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:57.414+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:57.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:57 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:57.414+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:39:57.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:57 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:57.497+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:58.230 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:57 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:57.946+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:58.231 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:57 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:57.946+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:58.241 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:57 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:57.823+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:58.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:58 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:58.192+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:58.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:58 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:58.192+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:39:58.490 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:58 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:58.229+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:58.490 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:58 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:58.229+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:58.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:58 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:58.375+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:58.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:58 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:58.375+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:39:58.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:58 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:58.375+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:39:58.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:58 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:58.489+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:39:58.963 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:58 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:58.481+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:58.964 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:58 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:58.481+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:39:59.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:58 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:58.870+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:39:59.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:59.206+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:18.700609+0000 front 2023-12-14T18:39:18.700533+0000 (oldest deadline 2023-12-14T18:39:44.000431+0000) 2023-12-14T18:39:59.339 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:39:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5[55004]: 2023-12-14T18:39:59.206+0000 7fc7521c4700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.101619+0000 front 2023-12-14T18:39:28.101736+0000 (oldest deadline 2023-12-14T18:39:51.001722+0000) 2023-12-14T18:39:59.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:58 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:58.960+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:39:59.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:58 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:58.960+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:39:59.451 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:59.183+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:39:59.451 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:39:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:39:59.183+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:39:59.713 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:59.450+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:39:59.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:59.450+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:39:59.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:39:59 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:39:59.450+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:39:59.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:59 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:59.421+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:39:59.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:59 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:59.421+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:39:59.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:39:59 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:39:59.421+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:39:59.838 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:39:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:39:59.493+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:00.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:59 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:59.898+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:00.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:39:59 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:39:59.898+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:00.277 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:59.949+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:00.277 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:59.949+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:00.277 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:39:59 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:39:59.949+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:00.589 INFO:journalctl@ceph.osd.5.smithi204.stdout:Dec 14 18:40:00 smithi204 bash[76196]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-5 2023-12-14T18:40:00.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:00 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:00.478+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:00.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:00.231+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:00.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:00.231+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:00.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:00.455+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:00.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:00.455+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:00.715 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:00 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:00.455+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:00.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:00 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:00.383+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:00.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:00 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:00.383+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:00.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:00 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:00.383+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:01.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:00 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:00.889+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:01.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:00 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:00.889+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:01.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:00 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:00.908+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:01.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:00 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:00.908+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:01.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:00 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:00.908+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:01.463 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:01 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:01.406+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:01.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:01 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:01.406+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:01.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:01 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:01.406+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:01.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:01 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:01.204+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:01.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:01 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:01.204+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:01.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:01 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:01.204+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:01.588 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:01 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:01.477+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:01.668 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.5.service' 2023-12-14T18:40:01.741 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:01.412+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:01.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:01.412+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:01.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:01.412+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:01.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:01.749 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-14T18:40:01.750 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-14T18:40:01.750 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.6 2023-12-14T18:40:02.214 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:01 smithi031 systemd[1]: Stopping Ceph osd.6 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:02.214 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:02.021+0000 7fe596483700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:02.214 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:02.021+0000 7fe596483700 -1 osd.6 105 *** Got signal Terminated *** 2023-12-14T18:40:02.214 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:02.021+0000 7fe596483700 -1 osd.6 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:40:02.241 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:01.885+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:02.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:01.885+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:02.338 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:01 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:01.921+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:02.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:01 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:01.921+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:02.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:01 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:01.921+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:02.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:02.413+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:02.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:02.413+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:02.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:02.413+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:02.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:02.214+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:02.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:02.214+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:02.714 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:02 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:02.214+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:02.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:02 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:02.440+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:02.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:02 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:02.440+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:02.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:02 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:02.440+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:02.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:02 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:02.437+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:02.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:02 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:02.437+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:03.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:02 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:02.917+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:03.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:02 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:02.917+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:03.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:02 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:02.945+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:03.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:02 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:02.945+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:03.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:02 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:02.945+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:03.441 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:03 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:03.186+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:03.441 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:03 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:03.187+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:03.441 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:03 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:03.187+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:03.713 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:03 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:03.440+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:03.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:03 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:03.440+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:03.714 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:03 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:03.440+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:03.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:03 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:03.477+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:03.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:03 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:03.477+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:03.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:03 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:03.477+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:03.838 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:03 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:03.392+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:03.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:03 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:03.392+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:04.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:03 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:03.887+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:04.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:03 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:03.887+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:04.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:03 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:03.981+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:04.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:03 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:03.981+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:04.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:03 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:03.981+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:04.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:04.455+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:04.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:04.455+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:04.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:04.455+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:04.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:04.207+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:04.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:04.207+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:04.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:04 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:04.207+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:04.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:04.394+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:04.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:04 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:04.394+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:04.897 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:04.491+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:04.898 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:04.491+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:04.898 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:04.491+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:05.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:04.896+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:05.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:04 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:04.896+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:05.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:05 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:05.021+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:05.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:05 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:05.021+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:05.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:05 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:05.021+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:05.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:05 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:05.179+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:05.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:05 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:05.179+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:05.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:05 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:05.179+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:05.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:05.451+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:05.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:05.451+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:05.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:05.451+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:05.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:05 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:05.414+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:05.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:05 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:05.414+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:05.963 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:05 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:05.463+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:05.964 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:05 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:05.463+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:05.964 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:05 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:05.463+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:06.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:05.880+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:06.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:05.880+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:06.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:05.880+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:06.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:06 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:06.019+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:06.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:06 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:06.019+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:06.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:06 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:06.019+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:06.463 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:06.434+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.822859+0000 front 2023-12-14T18:39:17.822882+0000 (oldest deadline 2023-12-14T18:39:42.522968+0000) 2023-12-14T18:40:06.463 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:06.434+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:30.124840+0000 front 2023-12-14T18:39:30.124709+0000 (oldest deadline 2023-12-14T18:39:54.224822+0000) 2023-12-14T18:40:06.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:06.434+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:34.225390+0000 front 2023-12-14T18:39:34.225374+0000 (oldest deadline 2023-12-14T18:39:58.925303+0000) 2023-12-14T18:40:06.464 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6[65834]: 2023-12-14T18:40:06.434+0000 7fe591276700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.727146+0000 front 2023-12-14T18:39:41.727193+0000 (oldest deadline 2023-12-14T18:40:05.827336+0000) 2023-12-14T18:40:06.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:06.209+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:06.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:06.209+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:06.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:06 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:06.209+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:06.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:06.408+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:06.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:06.408+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:06.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:06.408+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:06.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:06.408+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:06.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:06 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:06.370+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:06.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:06 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:06.370+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:07.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:06.893+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:07.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:06.893+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:07.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:06 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:06.893+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:07.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:07 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:07.000+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:07.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:07 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:07.000+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:07.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:07 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:07.000+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:07.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:07 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:07.205+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:07.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:07 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:07.205+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:07.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:07 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:07.205+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:07.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:07 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:07.205+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:07.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:07.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:07.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:07.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:07.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:07.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:07.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:07.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:07.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:07 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:07.355+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:07.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:07 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:07.355+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:08.184 INFO:journalctl@ceph.osd.6.smithi031.stdout:Dec 14 18:40:07 smithi031 bash[210047]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-6 2023-12-14T18:40:08.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:07.894+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:08.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:07.894+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:08.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:07 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:07.894+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:08.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:08 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:08.025+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:08.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:08 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:08.025+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:08.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:08 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:08.025+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:08.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:08.183+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:08.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:08.183+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:08.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:08.183+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:08.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:08 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:08.183+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:08.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:08.432+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:08.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:08.432+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:08.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:08.432+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:08.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:08.432+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:08.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:08 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:08.390+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:08.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:08 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:08.390+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:09.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:08.936+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:09.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:08.936+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:09.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:08 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:08.936+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:09.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:09 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:09.050+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:09.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:09 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:09.050+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:09.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:09 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:09.050+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:09.442 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:09.158+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:09.443 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:09.158+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:09.443 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:09.158+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:09.443 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:09 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:09.158+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:09.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.6.service' 2023-12-14T18:40:09.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:09.507 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-14T18:40:09.507 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-14T18:40:09.508 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.7 2023-12-14T18:40:09.742 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 systemd[1]: Stopping Ceph osd.7 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:09.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:09.463+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:09.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:09.463+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:09.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:09.463+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:09.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:09.463+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:09.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:09 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:09.353+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:09.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:09 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:09.353+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:10.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:09.746+0000 7fb24f6d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:10.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:09.746+0000 7fb24f6d0700 -1 osd.7 105 *** Got signal Terminated *** 2023-12-14T18:40:10.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:09.746+0000 7fb24f6d0700 -1 osd.7 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:40:10.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:09.935+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:10.243 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:09.935+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:10.243 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:09.935+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:10.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:10 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:10.062+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:10.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:10 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:10.062+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:10.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:10 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:10.062+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:10.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:10.135+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:10.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:10.135+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:10.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:10.135+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:10.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:10 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:10.135+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:10.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:10.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:10.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:10.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:10.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:10.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:10.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:10.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:10.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:10 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:10.370+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:10.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:10 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:10.370+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:11.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:10.939+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:11.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:10.939+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:11.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:10 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:10.939+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:11.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:11.076+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:11.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:11.076+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:11.335 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:11.076+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:11.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:11 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:11.133+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:11.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:11 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:11.133+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:11.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:11 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:11.133+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:11.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:11 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:11.133+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:11.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:11.334+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:11.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:11.334+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:11.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:11 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:11.334+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:11.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:11.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:11.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:11.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:11.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:11.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:11.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:11.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:12.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:11.937+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:12.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:11.937+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:12.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:11 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:11.937+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:12.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:12.086+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:12.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:12.086+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:12.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:12.086+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:12.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:12 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:12.086+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:12.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:12 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:12.096+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:12.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:12 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:12.096+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:12.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:12 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:12.096+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:12.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:12 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:12.332+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:12.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:12 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:12.332+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:12.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:12 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:12.332+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:12.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:12.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:12.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:12.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:12.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:12.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:12.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:12.395+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:13.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:12.983+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:13.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:12.983+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:13.242 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:12 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:12.983+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:13.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:13.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:13.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:13.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:13.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:13.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:13.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:13 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:13.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:13.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:13 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:13.089+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:13.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:13 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:13.089+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:13.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:13 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:13.089+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:13.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:13 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:13.334+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:13.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:13 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:13.334+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:13.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:13 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:13.334+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:13.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:13 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:13.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:13.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:13 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:13.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:13.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:13 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:13.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:13.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:13 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:13.397+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:14.323 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:14 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:14.049+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:14.323 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:14 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:14.049+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:14.323 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:14 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:14.049+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:14.440 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:14.018+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:28.097923+0000 front 2023-12-14T18:39:28.098088+0000 (oldest deadline 2023-12-14T18:39:50.998275+0000) 2023-12-14T18:40:14.440 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:14.018+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.699439+0000 front 2023-12-14T18:39:35.699599+0000 (oldest deadline 2023-12-14T18:39:59.799527+0000) 2023-12-14T18:40:14.440 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7[59428]: 2023-12-14T18:40:14.018+0000 7fb248cc0700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:44.500801+0000 front 2023-12-14T18:39:44.500639+0000 (oldest deadline 2023-12-14T18:40:05.600553+0000) 2023-12-14T18:40:14.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:14.117+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:14.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:14.117+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:14.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:14.117+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:14.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:14 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:14.117+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:14.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:14 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:14.322+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:14.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:14 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:14.322+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:14.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:14 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:14.322+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:14.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:14.438+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:14.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:14.438+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:14.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:14.438+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:14.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:14.438+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:14.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:14 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:14.438+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:15.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:15 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:15.085+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:15.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:15 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:15.085+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:15.339 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:15 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:15.085+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:15.340 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:15 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:15.322+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:15.340 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:15 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:15.322+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:15.340 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:15 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:15.322+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:15.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:15 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:15.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:15.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:15 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:15.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:15.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:15 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:15.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:15.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:15 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:15.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:15.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:15 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:15.097+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:15.742 INFO:journalctl@ceph.osd.7.smithi176.stdout:Dec 14 18:40:15 smithi176 bash[75176]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-7 2023-12-14T18:40:15.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:15 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:15.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:15.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:15 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:15.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:15.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:15 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:15.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:15.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:15 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:15.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:15.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:15 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:15.429+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:16.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:16 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:16.129+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:16.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:16 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:16.129+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:16.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:16 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:16.129+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:16.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:16 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:16.129+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:16.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:16 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:16.129+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:16.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:16.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:16.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:16.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:16.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:16.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:16.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:16.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:16.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:16.368+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:16.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:16.368+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:16.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:16.368+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:16.594 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:16 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:16.460+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:16.594 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:16 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:16.460+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:16.594 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:16 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:16.460+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:16.595 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:16 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:16.460+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:16.595 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:16 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:16.460+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:16.602 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.7.service' 2023-12-14T18:40:16.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:16.643 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-14T18:40:16.643 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-14T18:40:16.643 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.8 2023-12-14T18:40:17.089 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 systemd[1]: Stopping Ceph osd.8 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:17.089 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:16.868+0000 7fa360a7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:17.090 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:16.868+0000 7fa360a7f700 -1 osd.8 105 *** Got signal Terminated *** 2023-12-14T18:40:17.090 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:16.868+0000 7fa360a7f700 -1 osd.8 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:40:17.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:17.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:17.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:17.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:17.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:17.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:17.409 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:17.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:17.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:17.104+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:17.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:17.104+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:17.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:17.104+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:17.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:17.104+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:17.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:17 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:17.104+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:17.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:17 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:17.436+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:17.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:17 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:17.436+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:17.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:17 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:17.436+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:17.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:17 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:17.436+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:17.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:17 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:17.436+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:17.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:17.408+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:17.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:17.408+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:17.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:17 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:17.408+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:18.384 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:18.122+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:18.385 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:18.122+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:18.385 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:18.122+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:18.385 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:18.122+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:18.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:18.078+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:18.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:18.078+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:18.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:18.078+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:18.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:18.078+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:18.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:18 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:18.078+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:18.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:18.404+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:18.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:18.404+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:18.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:18.404+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:18.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:18.404+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:18.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:18 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:18.404+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:18.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:18.383+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:18.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:18.383+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:18.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:18.383+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:18.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:18 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:18.383+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:52.214169+0000 front 2023-12-14T18:39:52.214264+0000 (oldest deadline 2023-12-14T18:40:18.014716+0000) 2023-12-14T18:40:19.359 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:19.097+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:19.359 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:19.097+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:19.359 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:19.097+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:19.359 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:19.097+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:19.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:19.115+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:19.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:19.115+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:19.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:19.115+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:19.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:19.115+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:19.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:19 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:19.115+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:19.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:19 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:19.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:19.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:19 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:19.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:19.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:19 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:19.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:19.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:19 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:19.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:19.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:19 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:19.385+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:19.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:19.358+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:19.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:19.358+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:19.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:19.358+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:19.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:19 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:19.358+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:52.214169+0000 front 2023-12-14T18:39:52.214264+0000 (oldest deadline 2023-12-14T18:40:18.014716+0000) 2023-12-14T18:40:20.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:20.082+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:20.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:20.082+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:20.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:20.082+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:20.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:20.082+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:20.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:20 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:20.082+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:20.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:20.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:20.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:20.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:20.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:20.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:20.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:20.142+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:20.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:20.330+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:20.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:20.330+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:20.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:20.330+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:20.590 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:20.330+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:52.214169+0000 front 2023-12-14T18:39:52.214264+0000 (oldest deadline 2023-12-14T18:40:18.014716+0000) 2023-12-14T18:40:20.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:20 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:20.352+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:20.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:20 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:20.352+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:20.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:20 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:20.352+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:20.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:20 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:20.352+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:20.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:20 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:20.352+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:21.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:21.062+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:21.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:21.062+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:21.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:21.062+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:21.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:21.062+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:21.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:21 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:21.062+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:21.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:21.374+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.709221+0000 front 2023-12-14T18:39:17.709123+0000 (oldest deadline 2023-12-14T18:39:43.009143+0000) 2023-12-14T18:40:21.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:21.374+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.811668+0000 front 2023-12-14T18:39:35.811683+0000 (oldest deadline 2023-12-14T18:40:01.711893+0000) 2023-12-14T18:40:21.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:21.374+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:41.712768+0000 front 2023-12-14T18:39:41.712619+0000 (oldest deadline 2023-12-14T18:40:10.512941+0000) 2023-12-14T18:40:21.589 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8[60503]: 2023-12-14T18:40:21.374+0000 7fa35b071700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:52.214169+0000 front 2023-12-14T18:39:52.214264+0000 (oldest deadline 2023-12-14T18:40:18.014716+0000) 2023-12-14T18:40:21.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:21.175+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:21.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:21.175+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:21.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:21.175+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:21.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:21 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:21.175+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:21.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:21 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:21.351+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:21.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:21 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:21.351+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:21.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:21 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:21.351+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:21.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:21 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:21.351+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:21.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:21 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:21.351+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:22.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:22.057+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:22.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:22.057+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:22.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:22.057+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:22.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:22.057+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:22.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:22 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:22.057+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:22.566 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:22 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:22.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:22.566 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:22 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:22.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:22.566 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:22 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:22.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:22.566 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:22 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:22.129+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:22.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:22 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:22.324+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:22.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:22 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:22.324+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:22.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:22 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:22.324+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:22.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:22 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:22.324+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:22.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:22 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:22.324+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:22.839 INFO:journalctl@ceph.osd.8.smithi204.stdout:Dec 14 18:40:22 smithi204 bash[76460]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-8 2023-12-14T18:40:23.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:23.025+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:23.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:23.025+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:23.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:23.025+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:23.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:23.025+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:23.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:23 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:23.025+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:23.539 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:23 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:23.123+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:23.539 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:23 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:23.123+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:23.539 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:23 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:23.123+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:23.540 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:23 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:23.123+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:23.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:23 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:23.331+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:23.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:23 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:23.331+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:23.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:23 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:23.331+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:23.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:23 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:23.331+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:23.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:23 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:23.331+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:24.039 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.8.service' 2023-12-14T18:40:24.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:24.081 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-14T18:40:24.081 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-14T18:40:24.081 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.9 2023-12-14T18:40:24.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.027+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:24.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.027+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:24.335 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.027+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:24.336 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.027+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:24.336 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.027+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:24.336 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 systemd[1]: Stopping Ceph osd.9 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:24.336 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.334+0000 7f764b8eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:24.336 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.334+0000 7f764b8eb700 -1 osd.9 105 *** Got signal Terminated *** 2023-12-14T18:40:24.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:24.164+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:24.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:24.164+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:24.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:24.164+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:24.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:24.164+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:24.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:24 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:24.164+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:24.713 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:24 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:24.334+0000 7f764b8eb700 -1 osd.9 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:40:24.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:24 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:24.348+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:24.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:24 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:24.348+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:24.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:24 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:24.348+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:24.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:24 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:24.348+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:24.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:24 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:24.348+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:24.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:24 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:24.348+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:25.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:25.058+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:25.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:25.058+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:25.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:25.058+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:25.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:25.058+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:25.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:25.058+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:25.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:25 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:25.058+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:56.243344+0000 front 2023-12-14T18:39:56.243179+0000 (oldest deadline 2023-12-14T18:40:25.043400+0000) 2023-12-14T18:40:25.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:25.209+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:25.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:25.209+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:25.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:25.209+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:25.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:25.209+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:25.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:25.209+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:25.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:25.303+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:25.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:25.303+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:25.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:25.303+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:25.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:25.303+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:25.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:25.303+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:25.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:25 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:25.303+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:26.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:26 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:26.015+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:26.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:26 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:26.015+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:26.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:26 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:26.015+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:26.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:26 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:26.015+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:26.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:26 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:26.015+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:26.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:26 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:26.015+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:56.243344+0000 front 2023-12-14T18:39:56.243179+0000 (oldest deadline 2023-12-14T18:40:25.043400+0000) 2023-12-14T18:40:26.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:26 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:26.217+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:26.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:26 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:26.217+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:26.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:26 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:26.217+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:26.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:26 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:26.217+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:26.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:26 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:26.217+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:26.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:26 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:26.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:26.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:26 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:26.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:26.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:26 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:26.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:26.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:26 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:26.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:26.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:26 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:26.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:26.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:26 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:26.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:27.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:27 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:27.064+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:27.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:27 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:27.064+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:27.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:27 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:27.064+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:27.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:27 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:27.064+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:27.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:27 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:27.064+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:27.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:27 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:27.064+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:56.243344+0000 front 2023-12-14T18:39:56.243179+0000 (oldest deadline 2023-12-14T18:40:25.043400+0000) 2023-12-14T18:40:27.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:27 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:27.194+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:27.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:27 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:27.194+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:27.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:27 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:27.194+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:27.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:27 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:27.194+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:27.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:27 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:27.194+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:27.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:27 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:27.301+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:27.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:27 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:27.301+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:27.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:27 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:27.301+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:27.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:27 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:27.301+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:27.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:27 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:27.301+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:27.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:27 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:27.301+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:28.463 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:28 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:28.083+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:28.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:28 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:28.083+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:28.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:28 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:28.083+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:28.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:28 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:28.083+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:28.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:28 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:28.083+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:28.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:28 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:28.083+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:56.243344+0000 front 2023-12-14T18:39:56.243179+0000 (oldest deadline 2023-12-14T18:40:25.043400+0000) 2023-12-14T18:40:28.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:28 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:28.211+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:28.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:28 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:28.211+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:28.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:28 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:28.211+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:28.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:28 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:28.211+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:28.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:28 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:28.211+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:28.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:28 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:28.280+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:28.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:28 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:28.280+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:28.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:28 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:28.280+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:28.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:28 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:28.280+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:28.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:28 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:28.280+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:28.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:28 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:28.280+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:29.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:29.089+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.437196+0000 front 2023-12-14T18:39:19.437146+0000 (oldest deadline 2023-12-14T18:39:45.337334+0000) 2023-12-14T18:40:29.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:29.089+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.438598+0000 front 2023-12-14T18:39:29.438469+0000 (oldest deadline 2023-12-14T18:39:55.338590+0000) 2023-12-14T18:40:29.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:29.089+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.339253+0000 front 2023-12-14T18:39:35.339079+0000 (oldest deadline 2023-12-14T18:40:00.639117+0000) 2023-12-14T18:40:29.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:29.089+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.940482+0000 front 2023-12-14T18:39:42.940426+0000 (oldest deadline 2023-12-14T18:40:07.040678+0000) 2023-12-14T18:40:29.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:29.089+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:51.642154+0000 front 2023-12-14T18:39:51.642289+0000 (oldest deadline 2023-12-14T18:40:14.542097+0000) 2023-12-14T18:40:29.465 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:29 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9[71345]: 2023-12-14T18:40:29.089+0000 7f76466de700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:56.243344+0000 front 2023-12-14T18:39:56.243179+0000 (oldest deadline 2023-12-14T18:40:25.043400+0000) 2023-12-14T18:40:29.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:29.215+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:29.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:29.215+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:29.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:29.215+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:29.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:29.215+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:29.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:29.215+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:29.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:29.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:29.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:29.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:29.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:29.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:29.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:29 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:29.292+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:30.464 INFO:journalctl@ceph.osd.9.smithi031.stdout:Dec 14 18:40:30 smithi031 bash[210301]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-9 2023-12-14T18:40:30.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:30.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:30.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:30.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:30.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:30.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:30.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:30.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:30.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:30.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:30.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:30 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:30.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:30.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:30.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:30.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:30.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:30.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:30.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:30.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:30 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:30.314+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:31.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.9.service' 2023-12-14T18:40:31.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:31.519 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-14T18:40:31.519 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-14T18:40:31.520 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.10 2023-12-14T18:40:31.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:31 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:31.256+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:31.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:31 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:31.256+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:31.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:31 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:31.256+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:31.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:31 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:31.256+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:31.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:31 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:31.256+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:31.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:31 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:31.256+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.321+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:31.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 systemd[1]: Stopping Ceph osd.10 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:32.242 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.768+0000 7f16324c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:32.242 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.768+0000 7f16324c2700 -1 osd.10 105 *** Got signal Terminated *** 2023-12-14T18:40:32.242 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:31 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:31.768+0000 7f16324c2700 -1 osd.10 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:40:32.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:32.231+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:32.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:32.231+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:32.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:32.231+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:32.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:32.231+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:32.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:32.231+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:32.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:32 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:32.231+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:32.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:32.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:32.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:32.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:32.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:32.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:32.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:32 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:32.317+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:33.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:33 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:33.212+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:33.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:33 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:33.212+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:33.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:33 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:33.212+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:33.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:33 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:33.212+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:33.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:33 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:33.212+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:33.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:33 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:33.212+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:33.741 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:33.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:33.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:33.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:33.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:33.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:33.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:33 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:33.335+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:34.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:34.208+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:34.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:34.208+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:34.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:34.208+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:34.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:34.208+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:34.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:34.208+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:34.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:34 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:34.208+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:34.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:34.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:34.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:34.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:34.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:34.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:34.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:34 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:34.377+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:35.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:35.192+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:35.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:35.192+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:35.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:35.192+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:35.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:35.192+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:35.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:35.192+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:35.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:35 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:35.192+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:35.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:35.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:35.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:35.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:35.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:35.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:35.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:35 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:35.426+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:36.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:36 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:36.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:36.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:36 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:36.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:36.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:36 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:36.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:36.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:36 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:36.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:36.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:36 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:36.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:36.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:36 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:36.224+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:36.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:17.368651+0000 front 2023-12-14T18:39:17.368635+0000 (oldest deadline 2023-12-14T18:39:42.068634+0000) 2023-12-14T18:40:36.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:29.070264+0000 front 2023-12-14T18:39:29.070202+0000 (oldest deadline 2023-12-14T18:39:54.370433+0000) 2023-12-14T18:40:36.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:36.071380+0000 front 2023-12-14T18:39:36.071531+0000 (oldest deadline 2023-12-14T18:39:58.371386+0000) 2023-12-14T18:40:36.742 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6814 osd.3 since back 2023-12-14T18:39:42.373140+0000 front 2023-12-14T18:39:42.373085+0000 (oldest deadline 2023-12-14T18:40:05.873085+0000) 2023-12-14T18:40:36.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.973902+0000 front 2023-12-14T18:39:49.973932+0000 (oldest deadline 2023-12-14T18:40:13.473912+0000) 2023-12-14T18:40:36.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:58.774902+0000 front 2023-12-14T18:39:58.775021+0000 (oldest deadline 2023-12-14T18:40:23.474817+0000) 2023-12-14T18:40:36.743 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:36 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10[64914]: 2023-12-14T18:40:36.466+0000 7f162aab0700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.576042+0000 front 2023-12-14T18:40:04.575980+0000 (oldest deadline 2023-12-14T18:40:28.675860+0000) 2023-12-14T18:40:37.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:37 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:37.191+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:37.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:37 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:37.191+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:37.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:37 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:37.191+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:37.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:37 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:37.191+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:37.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:37 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:37.191+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:37.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:37 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:37.191+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:37.741 INFO:journalctl@ceph.osd.10.smithi176.stdout:Dec 14 18:40:37 smithi176 bash[75429]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-10 2023-12-14T18:40:38.542 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.10.service' 2023-12-14T18:40:38.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:38.583 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-14T18:40:38.584 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-14T18:40:38.584 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.11 2023-12-14T18:40:38.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.154+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:38.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.154+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:38.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.154+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:38.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.154+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:38.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.154+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:38.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.154+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:39.089 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 systemd[1]: Stopping Ceph osd.11 for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:39.089 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.848+0000 7f03937de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:39.089 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.848+0000 7f03937de700 -1 osd.11 105 *** Got signal Terminated *** 2023-12-14T18:40:39.090 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:38 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:38.848+0000 7f03937de700 -1 osd.11 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:40:39.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:39.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:39.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:39.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:39.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:39.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:39.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:39 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:39.153+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2023-12-14T18:40:09.318101+0000 front 2023-12-14T18:40:09.318176+0000 (oldest deadline 2023-12-14T18:40:38.718693+0000) 2023-12-14T18:40:40.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:40.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:40.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:40.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:40.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:40.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:40.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:40 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:40.109+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2023-12-14T18:40:09.318101+0000 front 2023-12-14T18:40:09.318176+0000 (oldest deadline 2023-12-14T18:40:38.718693+0000) 2023-12-14T18:40:41.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:41.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:41.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:41.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:41.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:41.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:41.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:41 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:41.131+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2023-12-14T18:40:09.318101+0000 front 2023-12-14T18:40:09.318176+0000 (oldest deadline 2023-12-14T18:40:38.718693+0000) 2023-12-14T18:40:42.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:42.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:42.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:42.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:42.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:42.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:42.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:42 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:42.118+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2023-12-14T18:40:09.318101+0000 front 2023-12-14T18:40:09.318176+0000 (oldest deadline 2023-12-14T18:40:38.718693+0000) 2023-12-14T18:40:43.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2023-12-14T18:39:19.809783+0000 front 2023-12-14T18:39:19.809688+0000 (oldest deadline 2023-12-14T18:39:45.709653+0000) 2023-12-14T18:40:43.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6804 osd.1 since back 2023-12-14T18:39:27.311597+0000 front 2023-12-14T18:39:27.311521+0000 (oldest deadline 2023-12-14T18:39:52.611457+0000) 2023-12-14T18:40:43.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2023-12-14T18:39:35.512436+0000 front 2023-12-14T18:39:35.512540+0000 (oldest deadline 2023-12-14T18:39:59.612634+0000) 2023-12-14T18:40:43.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6812 osd.4 since back 2023-12-14T18:39:49.315779+0000 front 2023-12-14T18:39:49.315799+0000 (oldest deadline 2023-12-14T18:40:15.215845+0000) 2023-12-14T18:40:43.589 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2023-12-14T18:39:59.316960+0000 front 2023-12-14T18:39:59.316915+0000 (oldest deadline 2023-12-14T18:40:24.017048+0000) 2023-12-14T18:40:43.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.31:6822 osd.6 since back 2023-12-14T18:40:04.017655+0000 front 2023-12-14T18:40:04.017759+0000 (oldest deadline 2023-12-14T18:40:29.317507+0000) 2023-12-14T18:40:43.590 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:43 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11[65946]: 2023-12-14T18:40:43.094+0000 7f038fdd4700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.176:6820 osd.7 since back 2023-12-14T18:40:09.318101+0000 front 2023-12-14T18:40:09.318176+0000 (oldest deadline 2023-12-14T18:40:38.718693+0000) 2023-12-14T18:40:44.839 INFO:journalctl@ceph.osd.11.smithi204.stdout:Dec 14 18:40:44 smithi204 bash[76716]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-osd-11 2023-12-14T18:40:45.879 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@osd.11.service' 2023-12-14T18:40:45.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:45.960 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-14T18:40:45.960 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-14T18:40:45.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.a 2023-12-14T18:40:46.257 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:40:46 smithi031 systemd[1]: Stopping Ceph mds.a for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:46.713 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:40:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-a[76579]: 2023-12-14T18:40:46.254+0000 7f6f2dc39700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:46.713 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:40:46 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-a[76579]: 2023-12-14T18:40:46.254+0000 7f6f2dc39700 -1 mds.a *** got signal Terminated *** 2023-12-14T18:40:48.187 INFO:journalctl@ceph.mds.a.smithi031.stdout:Dec 14 18:40:47 smithi031 bash[210559]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-a 2023-12-14T18:40:48.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.a.service' 2023-12-14T18:40:48.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:48.238 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-14T18:40:48.238 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-14T18:40:48.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.d 2023-12-14T18:40:48.463 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:40:48 smithi031 systemd[1]: Stopping Ceph mds.d for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:48.964 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:40:48 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d[75661]: 2023-12-14T18:40:48.520+0000 7f43c4a94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:48.964 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:40:48 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d[75661]: 2023-12-14T18:40:48.520+0000 7f43c4a94700 -1 mds.d *** got signal Terminated *** 2023-12-14T18:40:53.098 INFO:journalctl@ceph.mds.d.smithi031.stdout:Dec 14 18:40:52 smithi031 bash[210706]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-d 2023-12-14T18:40:53.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.d.service' 2023-12-14T18:40:53.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:53.148 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-14T18:40:53.148 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-14T18:40:53.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.g 2023-12-14T18:40:53.446 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:40:53 smithi031 systemd[1]: Stopping Ceph mds.g for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:53.713 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:40:53 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-g[76190]: 2023-12-14T18:40:53.444+0000 7fe2d02eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:53.714 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:40:53 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-g[76190]: 2023-12-14T18:40:53.444+0000 7fe2d02eb700 -1 mds.g *** got signal Terminated *** 2023-12-14T18:40:57.963 INFO:journalctl@ceph.mds.g.smithi031.stdout:Dec 14 18:40:57 smithi031 bash[210846]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-g 2023-12-14T18:40:58.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.g.service' 2023-12-14T18:40:58.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:40:58.134 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-14T18:40:58.134 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-14T18:40:58.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.j 2023-12-14T18:40:58.454 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:40:58 smithi031 systemd[1]: Stopping Ceph mds.j for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:40:58.713 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:40:58 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-j[75275]: 2023-12-14T18:40:58.451+0000 7ff6327b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:40:58.714 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:40:58 smithi031 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-j[75275]: 2023-12-14T18:40:58.451+0000 7ff6327b0700 -1 mds.j *** got signal Terminated *** 2023-12-14T18:41:01.652 INFO:journalctl@ceph.mds.j.smithi031.stdout:Dec 14 18:41:01 smithi031 bash[210986]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-j 2023-12-14T18:41:01.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.j.service' 2023-12-14T18:41:01.698 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:01.698 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-14T18:41:01.698 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-14T18:41:01.698 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.b 2023-12-14T18:41:01.992 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:41:01 smithi176 systemd[1]: Stopping Ceph mds.b for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:01.992 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:41:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b[68675]: 2023-12-14T18:41:01.939+0000 7f9664380700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:01.992 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:41:01 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b[68675]: 2023-12-14T18:41:01.939+0000 7f9664380700 -1 mds.b *** got signal Terminated *** 2023-12-14T18:41:05.227 INFO:journalctl@ceph.mds.b.smithi176.stdout:Dec 14 18:41:04 smithi176 bash[75691]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-b 2023-12-14T18:41:05.233 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.b.service' 2023-12-14T18:41:05.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:05.271 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-14T18:41:05.272 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-14T18:41:05.272 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.e 2023-12-14T18:41:05.492 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:41:05 smithi176 systemd[1]: Stopping Ceph mds.e for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:05.991 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:41:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-e[67986]: 2023-12-14T18:41:05.538+0000 7fb13a62e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:05.992 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:41:05 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-e[67986]: 2023-12-14T18:41:05.538+0000 7fb13a62e700 -1 mds.e *** got signal Terminated *** 2023-12-14T18:41:09.069 INFO:journalctl@ceph.mds.e.smithi176.stdout:Dec 14 18:41:08 smithi176 bash[75829]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-e 2023-12-14T18:41:09.074 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.e.service' 2023-12-14T18:41:09.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:09.109 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-14T18:41:09.109 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-14T18:41:09.110 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.h 2023-12-14T18:41:09.396 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:41:09 smithi176 systemd[1]: Stopping Ceph mds.h for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:09.741 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:41:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h[68329]: 2023-12-14T18:41:09.395+0000 7f84a221d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:09.742 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:41:09 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h[68329]: 2023-12-14T18:41:09.395+0000 7f84a221d700 -1 mds.h *** got signal Terminated *** 2023-12-14T18:41:13.180 INFO:journalctl@ceph.mds.h.smithi176.stdout:Dec 14 18:41:12 smithi176 bash[75968]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-h 2023-12-14T18:41:13.185 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.h.service' 2023-12-14T18:41:13.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:13.218 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-14T18:41:13.218 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-14T18:41:13.219 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.k 2023-12-14T18:41:13.487 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:41:13 smithi176 systemd[1]: Stopping Ceph mds.k for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:13.741 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:41:13 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-k[67645]: 2023-12-14T18:41:13.486+0000 7fe80dc11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:13.742 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:41:13 smithi176 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-k[67645]: 2023-12-14T18:41:13.486+0000 7fe80dc11700 -1 mds.k *** got signal Terminated *** 2023-12-14T18:41:15.742 INFO:journalctl@ceph.mds.k.smithi176.stdout:Dec 14 18:41:15 smithi176 bash[76107]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-k 2023-12-14T18:41:15.798 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.k.service' 2023-12-14T18:41:15.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:15.832 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-14T18:41:15.832 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-14T18:41:15.833 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.c 2023-12-14T18:41:16.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:41:15 smithi204 systemd[1]: Stopping Ceph mds.c for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:16.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:41:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c[69520]: 2023-12-14T18:41:16.090+0000 7fee50128700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:16.339 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:41:16 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c[69520]: 2023-12-14T18:41:16.090+0000 7fee50128700 -1 mds.c *** got signal Terminated *** 2023-12-14T18:41:20.589 INFO:journalctl@ceph.mds.c.smithi204.stdout:Dec 14 18:41:20 smithi204 bash[76977]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-c 2023-12-14T18:41:20.640 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.c.service' 2023-12-14T18:41:20.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:20.678 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-14T18:41:20.678 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-14T18:41:20.678 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.f 2023-12-14T18:41:20.960 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:41:20 smithi204 systemd[1]: Stopping Ceph mds.f for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:21.338 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:41:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f[69178]: 2023-12-14T18:41:20.958+0000 7f717ba77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:21.338 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:41:20 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f[69178]: 2023-12-14T18:41:20.958+0000 7f717ba77700 -1 mds.f *** got signal Terminated *** 2023-12-14T18:41:25.339 INFO:journalctl@ceph.mds.f.smithi204.stdout:Dec 14 18:41:24 smithi204 bash[77117]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-f 2023-12-14T18:41:25.415 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.f.service' 2023-12-14T18:41:25.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:25.449 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-14T18:41:25.449 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-14T18:41:25.449 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.i 2023-12-14T18:41:25.718 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:41:25 smithi204 systemd[1]: Stopping Ceph mds.i for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:26.089 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:41:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: 2023-12-14T18:41:25.716+0000 7f6a069ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:26.089 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:41:25 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i[69861]: 2023-12-14T18:41:25.718+0000 7f6a069ca700 -1 mds.i *** got signal Terminated *** 2023-12-14T18:41:29.311 INFO:journalctl@ceph.mds.i.smithi204.stdout:Dec 14 18:41:28 smithi204 bash[77254]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-i 2023-12-14T18:41:29.317 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.i.service' 2023-12-14T18:41:29.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:29.351 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-14T18:41:29.351 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-14T18:41:29.352 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.l 2023-12-14T18:41:29.589 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:41:29 smithi204 systemd[1]: Stopping Ceph mds.l for 37e25396-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:41:30.089 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:41:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-l[68836]: 2023-12-14T18:41:29.616+0000 7f38d7196700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:41:30.089 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:41:29 smithi204 ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-l[68836]: 2023-12-14T18:41:29.616+0000 7f38d7196700 -1 mds.l *** got signal Terminated *** 2023-12-14T18:41:34.089 INFO:journalctl@ceph.mds.l.smithi204.stdout:Dec 14 18:41:33 smithi204 bash[77392]: ceph-37e25396-9aad-11ee-95a5-87774f69a715-mds-l 2023-12-14T18:41:34.122 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-37e25396-9aad-11ee-95a5-87774f69a715@mds.l.service' 2023-12-14T18:41:34.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:41:34.156 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-14T18:41:34.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-14T18:41:36.598 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-14T18:41:38.253 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-14T18:41:40.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:41:40.059 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:41:40.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:41:40.127 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-14T18:41:40.129 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi031/crash 2023-12-14T18:41:40.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash -- . 2023-12-14T18:41:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-14T18:41:40.178 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2023-12-14T18:41:40.181 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi176/crash 2023-12-14T18:41:40.182 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash -- . 2023-12-14T18:41:40.218 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-14T18:41:40.218 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2023-12-14T18:41:40.220 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi204/crash 2023-12-14T18:41:40.222 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash -- . 2023-12-14T18:41:40.278 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/37e25396-9aad-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-14T18:41:40.278 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2023-12-14T18:41:40.279 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-14T18:41:40.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/37e25396-9aad-11ee-95a5-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-12-14T18:41:40.319 INFO:teuthology.orchestra.run.smithi031.stderr:grep: /var/log/ceph/37e25396-9aad-11ee-95a5-87774f69a715/ceph.log: No such file or directory 2023-12-14T18:41:40.322 INFO:tasks.cephadm:Compressing logs... 2023-12-14T18:41:40.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:41:40.365 DEBUG:teuthology.orchestra.run.smithi176:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:41:40.368 DEBUG:teuthology.orchestra.run.smithi204:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:41:40.396 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-14T18:41:40.406 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-14T18:41:40.417 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-14T18:46:17.689 INFO:tasks.cephadm:Archiving logs... 2023-12-14T18:46:17.690 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi031/log 2023-12-14T18:46:17.692 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-14T18:46:30.967 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi176/log 2023-12-14T18:46:30.968 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-14T18:46:50.964 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi204/log 2023-12-14T18:46:50.965 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-14T18:47:21.107 INFO:tasks.cephadm:Removing cluster... 2023-12-14T18:47:21.107 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --force 2023-12-14T18:47:21.816 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --force 2023-12-14T18:47:22.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 37e25396-9aad-11ee-95a5-87774f69a715 --force 2023-12-14T18:47:23.321 INFO:tasks.cephadm:Removing cephadm ... 2023-12-14T18:47:23.322 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-14T18:47:23.345 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-14T18:47:23.367 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-14T18:47:23.405 INFO:tasks.cephadm:Teardown complete 2023-12-14T18:47:23.406 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-14T18:47:23.418 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-14T18:47:23.418 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-14T18:47:23.421 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-14T18:47:23.423 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-14T18:47:23.584 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-14T18:47:23.595 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-14T18:47:23.595 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:47:23.598 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:47:23.601 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:47:23.617 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-12-14T18:47:23.618 INFO:teuthology.orchestra.run.smithi176.stderr:bash: ntpq: command not found 2023-12-14T18:47:23.618 INFO:teuthology.orchestra.run.smithi204.stderr:bash: ntpq: command not found 2023-12-14T18:47:23.623 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:47:23.623 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-12-14T18:47:23.623 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 22 +523us[ +523us] +/- 80ms 2023-12-14T18:47:23.623 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -677us[ -671us] +/- 50ms 2023-12-14T18:47:23.623 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +509us[ +509us] +/- 73ms 2023-12-14T18:47:23.623 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:47:23.625 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:47:23.625 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2023-12-14T18:47:23.625 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +491us[ +488us] +/- 80ms 2023-12-14T18:47:23.625 INFO:teuthology.orchestra.run.smithi176.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -695us[ -697us] +/- 51ms 2023-12-14T18:47:23.625 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +498us[ +495us] +/- 73ms 2023-12-14T18:47:23.625 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:47:23.665 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:47:23.665 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2023-12-14T18:47:23.665 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +461us[ +434us] +/- 80ms 2023-12-14T18:47:23.665 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 6 377 18 -700us[ -728us] +/- 51ms 2023-12-14T18:47:23.665 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +483us[ +456us] +/- 73ms 2023-12-14T18:47:23.665 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:47:23.667 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-14T18:47:23.679 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-14T18:47:23.679 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-14T18:47:23.695 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-14T18:47:23.846 DEBUG:teuthology.orchestra.run.smithi176:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-14T18:47:23.915 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-14T18:47:23.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:47:24.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:47:24.036 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:47:24.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:47:24.069 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:47:24.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:47:24.099 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-14T18:47:24.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-14T18:47:24.194 INFO:teuthology.task.internal:Duration was 2426.832544 seconds 2023-12-14T18:47:24.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-14T18:47:24.240 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-14T18:47:24.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-14T18:47:24.243 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-14T18:47:24.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-14T18:47:24.296 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:47:24.336 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:47:24.343 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:47:24.763 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-14T18:47:24.763 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2023-12-14T18:47:24.763 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-14T18:47:24.807 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2023-12-14T18:47:24.807 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-14T18:47:24.833 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2023-12-14T18:47:24.834 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-14T18:47:24.856 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-14T18:47:24.856 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:47:24.860 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:47:24.876 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:47:24.939 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-14T18:47:24.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-14T18:47:24.942 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-14T18:47:24.945 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-14T18:47:25.067 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-14T18:47:25.102 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-14T18:47:25.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-14T18:47:25.137 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-14T18:47:25.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-14T18:47:25.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-14T18:47:25.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:47:25.324 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:47:25.327 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:47:25.354 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2023-12-14T18:47:25.359 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2023-12-14T18:47:25.367 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2023-12-14T18:47:25.386 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:47:25.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:47:25.406 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:47:25.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:47:25.444 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:47:25.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:47:25.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-14T18:47:25.472 INFO:teuthology.task.internal:Transferring archived files... 2023-12-14T18:47:25.473 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi031 2023-12-14T18:47:25.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-14T18:47:25.518 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi176 2023-12-14T18:47:25.518 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-14T18:47:25.555 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431/remote/smithi204 2023-12-14T18:47:25.555 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-14T18:47:25.591 INFO:teuthology.task.internal:Removing archive directory... 2023-12-14T18:47:25.591 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-14T18:47:25.593 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-14T18:47:25.596 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-14T18:47:25.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-14T18:47:25.658 INFO:teuthology.task.internal:Not uploading archives. 2023-12-14T18:47:25.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-14T18:47:25.764 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-14T18:47:25.764 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-14T18:47:25.766 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-14T18:47:25.768 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-14T18:47:25.783 INFO:teuthology.orchestra.run.smithi176.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 14 18:47 /home/ubuntu/cephtest 2023-12-14T18:47:25.785 INFO:teuthology.orchestra.run.smithi031.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 14 18:47 /home/ubuntu/cephtest 2023-12-14T18:47:25.785 INFO:teuthology.orchestra.run.smithi204.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 14 18:47 /home/ubuntu/cephtest 2023-12-14T18:47:25.787 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-14T18:47:25.802 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-14T18:47:25.867 INFO:teuthology.nuke:Checking targets against current locks 2023-12-14T18:47:25.895 DEBUG:teuthology.nuke:shortname: smithi031 2023-12-14T18:47:25.896 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:47:25.918 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:30.958178', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBAqEFIrERJ0E4b5IMkedtnZUWsmlYIIy7lk3DB4N5nmW7/t2499qhZeWhvObY0t8ffVcjTUsLT6gglmQ8Mj6RU='} 2023-12-14T18:47:25.922 DEBUG:teuthology.nuke:shortname: smithi176 2023-12-14T18:47:25.922 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:47:25.942 DEBUG:teuthology.nuke:shortname: smithi204 2023-12-14T18:47:25.943 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:47:25.949 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:30.960024', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEq2mpC03HKtWtJka6RjSZNr5uY9crXaq6+micl/UkHRcCYKzJAOoM8HKHc5xTs+7DOgHl4jUJunghqSEf3XmLE='} 2023-12-14T18:47:25.966 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:30.961444', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9DUm0HJt+g7/4B/7t7kUrNVx02zoDrxD3L4PZzs2A8y7LAoS09DOMhwuSXDTBK1APhaQkPKkTatUGeV1osDAg='} 2023-12-14T18:47:25.978 INFO:teuthology.orchestra.console.smithi031:Power off 2023-12-14T18:47:25.978 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-14T18:47:26.004 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-12-14T18:47:26.004 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:26.016 INFO:teuthology.orchestra.console.smithi176:Power off 2023-12-14T18:47:26.017 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-14T18:47:26.031 INFO:teuthology.orchestra.console.smithi204:Power off 2023-12-14T18:47:26.031 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-14T18:47:26.038 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-12-14T18:47:26.039 DEBUG:teuthology.orchestra.console.smithi176:power off output: Chassis Power Control: Down/Off 2023-12-14T18:47:26.039 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:26.053 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2023-12-14T18:47:26.053 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:26.060 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2023-12-14T18:47:26.073 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-12-14T18:47:30.040 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:30.061 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:30.074 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:30.150 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-12-14T18:47:30.169 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2023-12-14T18:47:30.183 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-12-14T18:47:34.156 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:34.171 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:34.185 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:34.266 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-12-14T18:47:34.280 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2023-12-14T18:47:34.294 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2023-12-14T18:47:34.366 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-12-14T18:47:34.381 INFO:teuthology.orchestra.console.smithi176:Power off completed 2023-12-14T18:47:38.297 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:47:38.409 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2023-12-14T18:47:38.510 INFO:teuthology.orchestra.console.smithi204:Power off completed 2023-12-14T18:47:38.617 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} duration: 2426.832544326782 failure_reason: 'Command failed (workunit test fs/misc/layout_vxattrs.sh) on smithi031 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=08caab1e99f04e72a3cdaaf5804e0c70 status: fail success: false 2023-12-14T18:47:38.617 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-14T18:47:38.707 INFO:teuthology.run:FAIL